Multithreading MtxVec Code

MtxVec supports multithreading at two levels: internal (automatic) and user-level (explicit). Both work best with block processing as a foundation.

Three-step performance optimization:

  1. Vectorize — use MtxVec methods instead of scalar loops (3-10x speedup)
  2. Block process — add BlockInit/BlockNext/BlockEnd (1.5-3x on top of step 1)
  3. Multithread — parallelize the blocked code (2-4x on top of step 2)

Skipping steps loses performance: Threading without blocking does not result in a speed-up, because parallelism requires each CPU core to hold its working data in its own private cache (L1 or L2) — block processing is what gives each core that local, independent working set, and is therefore unconditionally "required" for threading to scale with core count.

User-level multithreading with DoForLoop:

using Dew.Math.Units;

void MyLoopBody(int IdxMin, int IdxMax, object[] Context)
{
    var vxb = new Vector();
    var vyb = new Vector();
    vxb.BlockInit(vx);   // create per-thread block view
    vyb.BlockInit(vy);

    while (!vxb.BlockEnd()) {
        for (int i = IdxMin; i <= IdxMax; i++) {
            result[i] = vxb.DotProd(vyb);
        }
        vxb.BlockNext();
        vyb.BlockNext();
    }
}

// Launch threads — blocks until all finish
MtxForLoop.DoForLoop(0, N - 1, MyLoopBody, null, null);

Key rules for threaded code:

  1. Each thread works on its own data range (IdxMin..IdxMax) — no shared writes
  2. Use Vector/Matrix (value types) or CreateIt/FreeIt inside thread bodies
  3. Do not create TVec/TMtx with constructors inside threads — use CreateIt or value types
  4. Block processing ensures each thread's data fits in its core's L1 cache

Internal threading: MtxVec automatically threads some operations (Sin, Cos, Exp, Ln, FFT, BLAS). The Controller global variable controls this:

// Query/set thread counts per subsystem
Controller.FFTThreadCount = 4;
Controller.BLASThreadCount = 4;
Controller.VMLThreadCount = 4;
Controller.IPPThreadCount = 4;

// Disable all internal threading (when doing your own threading)
Controller.ThreadingMode = TThreadingMode.tmNone;

// Other controller properties
int cacheSize = Controller.CPUCacheSize;      // L1 cache size in bytes
int coreCount = Controller.CPUCoreCount;      // number of cores

// DenormalsAreZero — forces tiny numbers to zero
Controller.DenormalsAreZero = true;  // up to 50x speedup if denormals occur

// Thread spin-wait (trade CPU usage for lower latency)
Controller.ThreadWaitBeforeSleep = 10;  // ms to spin before sleeping

FFT threading tips:

  1. Use not-in-place FFT versions (avoid internal copy)
  2. Use FFTFromReal/IFFTToReal when possible (2x faster than complex-to-complex)
  3. Prefer power-of-two sizes (3-4x faster than non-power-of-two)
  4. Disable internal FFT threading when doing your own threading
  5. Limit the number of different FFT sizes to reduce memory usage

Turbo mode warning: Modern CPUs run single-core up to 50% faster than all-core. A well-optimized single-threaded FFT can beat a poorly threaded one.