Multithreading MtxVec Code

MtxVec supports multithreading at two levels: internal (automatic) and user-level (explicit). Both work best with block processing as a foundation.

Three-step performance optimization:

1. Vectorize — use MtxVec methods instead of scalar loops (3-10x speedup)
2. Block process — add BlockInit/BlockNext/BlockEnd (1.5-3x on top of step 1)
3. Multithread — parallelize the blocked code (2-4x on top of step 2)

Skipping steps loses performance: Threading without blocking does not result in a speed-up, because parallelism requires each CPU core to hold its working data in its own private cache (L1 or L2) — block processing is what gives each core that local, independent working set, and is therefore unconditionally "required" for threading to scale with core count.

User-level multithreading with DoForLoop:

#include "Units.MtxVec.h"   // DoForLoop

using namespace Dew;
using namespace Dew::Math;
using namespace Dew::Math::Units::MtxVec;

// Called once per thread; each thread owns the range IdxMin..IdxMax
void MyLoopBody(int IdxMin, int IdxMax, const DewArray<tObject*>& Context, int ThreadIndex)
{
    TVec* vx = (TVec*)Context[0];
    TVec* vy = (TVec*)Context[1];
    TVec* result = (TVec*)Context[2];

    Vector vxb;
    Vector vyb;
    vxb.BlockInit(vx);   // create per-thread block view
    vyb.BlockInit(vy);

    while (!vxb.GetBlockEnd())
    {
        for (int i = IdxMin; i <= IdxMax; i++)
        {
            result->Values(i) += vxb.DotProd(vyb);   // accumulate across blocks
        }
        vxb.BlockNext();
        vyb.BlockNext();
    }
}

void ThreadedDotProducts(const Vector& vx, const Vector& vy, const Vector& result, int N)
{
    result.Size(N);
    result.SetZero();
    // Launch threads — blocks until all finish
    DoForLoop(0, N - 1, MyLoopBody, nullptr, DewArray<tObject*>{vx, vy, result});
}

Key rules for threaded code:

1. Each thread works on its own data range (IdxMin..IdxMax) — no shared writes
2. Use Vector/Matrix (value types) or CreateIt/FreeIt inside thread bodies
3. Do not create TVec/TMtx with constructors inside threads — use CreateIt or value types

In C++ the loop body is a plain function with the signature void (int IdxMin, int IdxMax, const DewArray<tObject*>& Context, int ThreadIndex), which DoForLoop accepts as a TForLoopRangeEvent. To run a member function, bind the object to a static function whose first parameter is void*: TForLoopRangeEvent(this, &MyClass::LoopBody). Objects reach the body through Context as tObject*. Passing nullptr for Threads uses the library's shared thread pool.

Internal threading: MtxVec automatically threads some operations (Sin, Cos, Exp, Ln, FFT, BLAS). The Controller global variable controls this:

// Query/set thread counts per subsystem
Controller->SetFFTThreadCount(4);
Controller->SetBlasThreadCount(4);
Controller->SetVmlThreadCount(4);
Controller->SetIppThreadCount(4);

// Disable internal threading (when doing your own threading)
Controller->SetThreadingMode(TMklThreadingMode::ttSerial);

// Other controller properties
int cacheSize = Controller->GetCpuCacheL1SizeInBytes();   // L1 cache size in bytes
int coreCount = Controller->GetCpuCores();                // number of cores
int frequency = Controller->GetCpuFrequencyInMhz();       // CPU frequency in MHz

// DenormalsAreZero — forces tiny numbers to zero
Controller->SetDenormalsAreZero(true);

// Thread spin-wait (trade CPU usage for lower latency)
Controller->SetThreadWaitBeforeSleep(10);   // ms to spin before sleeping (ttThroughput mode)

FFT threading tips:

1. Use not-in-place FFT versions (avoid internal copy)
2. Use FFTFromReal/IFFTToReal when possible (2x faster than complex-to-complex)
3. Prefer power-of-two sizes (3-4x faster than non-power-of-two)
4. Disable internal FFT threading when doing your own threading
5. Limit the number of different FFT sizes to reduce memory usage

Turbo mode warning: Modern CPUs run single-core up to 50% faster than all-core. A well-optimized single-threaded FFT can beat a poorly threaded one.