snapraid/raid/NOTES

SnapRAID RAID library performance notes
=======================================

Scope
-----

These notes summarize performance experiments, implementation choices,
rejected ideas, recovery optimizations, and non-obvious conclusions
reached while reviewing and tuning the SnapRAID RAID library.

The focus is the 1..6 parity implementation, especially x86 AVX2,
AVX512BW, GFNI, SSSE3, and AArch64/AArch32 NEON backends.

This is intentionally a record of what has already been tried and what was
learned from it, so that the same ideas are not repeatedly re-investigated.

Detailed project-by-project comparisons with other RAID and erasure-coding
engines are kept in OTHERS. This file keeps only conclusions that affect
SnapRAID implementation or optimization policy.

The main rule is to optimize the real SnapRAID workload, not only isolated
inner loops.


Implementation overview
-----------------------

Generation:

* The 1..6 parity generation paths are considered effectively closed from
  a performance point of view after coefficient-residency optimization and
  the separation of the AES G23 and common x2 hot paths.

* Non-GFNI AES generation uses dedicated x2-only and G23 implementations.
  When the number of data disks does not reach the first G23 x3 transition,
  AES dispatches directly to the same branch-free x2 implementation structure
  used by RAID, with the active AES polynomial and Cauchy tables.

* The G23 implementation is reserved for larger arrays that actually require
  x3 transitions. This avoids carrying G23 boundary control flow through the
  overwhelmingly common small-array generation path.

* Relevant scalar, SSE/SSSE3, AVX2, AVX512BW, AArch64/AArch32 NEON, RAID GFNI
  affine, and AES GFNI paths are implemented.

* Specialized GEN2 and GEN3 paths keep useful multiplication tables or
  coefficient representations in vector registers when register pressure
  permits it.

* This residency optimization preserves the preferred chunk-major,
  multi-output traversal. It does not add another pass over source blocks.

* RAID GFNI Horner Q is implemented where it is useful and where register
  pressure allows it.

* AVX512 128-byte processing, non-GFNI AVX512BW Horner, standalone
  VPTERNLOGQ transformations, VPBLENDVB, cached parity stores, and large
  pre-expanded GFNI constants have been investigated. Some are rejected, while
  VPTERNLOGQ is now used where it is part of a larger profitable GEN2/GEN3
  restructuring.

Recovery:

* The normal one-data-disk recovery with P available is already handled by
  raid_rec1of1(), which is essentially an XOR generation pass with pointer
  remapping.

* Generic REC1 non-P SIMD kernels have been locally optimized and spatially
  unrolled where useful.

* Generic REC2 non-GFNI kernels hoist coefficient tables and reuse delta nibble
  splits.

* NEON REC2 processes two 16-byte blocks per iteration, for a 32-byte recovery
  chunk on the optimized AArch64 path.

* RECX keeps delta values in SIMD registers on the optimized paths and reuses
  pre-split low/high nibbles instead of repeatedly reconstructing them.

* Non-GFNI RECX precomputes shuffle-table pointers in C before entering the hot
  data loop.

* RAID GFNI RECX precomputes the 8-byte affine representation of every inverse
  matrix coefficient in C before entering the hot data loop.

* A recurring RECX local cost is repeated coefficient table/matrix loading
  or broadcasting inside the data loop.

* Specialized SIMD P+Q rec2of2 recovery is a distinct optimization
  opportunity. The scalar int8 implementation already has the formula.

Architecture and validation:

* AArch32 has a separate NEON backend for generation and recovery. Cross-build,
  QEMU user-mode testing, and CI coverage are available for the ARM32 path.

* Matrix inversion has dedicated multithreaded stress testing. With at most six
  parity levels, inversion setup is not a significant performance target.


Real workload assumptions
-------------------------

* SnapRAID processes large blocks: typically 256 KiB per data/parity block.

* A normal stripe therefore touches a large amount of memory. With 12 data
  disks, one 256 KiB stripe already means about 3 MiB of data input before
  considering parity output.

* This large-block behavior is very important. Optimizations that reread a
  source buffer, increase the number of passes, or enlarge lookup tables can be
  worse even if the inner instruction sequence looks better.

* This is also the first filter for ideas taken from other RAID or
  erasure-coding engines. A design optimized for page-sized recovery,
  cache-resident tiles, or generic small shards may make a different memory
  tradeoff and should not be copied mechanically.

* Prefer reducing work inside the existing single-pass traversal before
  considering a traversal change. Saving a broadcast or table load is useful;
  saving registers by rereading a complete 256 KiB source block usually is not.

* SnapRAID uses a queue before parity data is written to disk. Real measurements
  with actual disk writes show that very small queues are worse. Performance
  improves clearly up to about queue depth 32, and still improves after that,
  but with diminishing returns.

* With 256 KiB blocks, queue depth 32 means this much parity output can be in
  flight:

      1 parity  =  8 MiB
      2 parity  = 16 MiB
      3 parity  = 24 MiB
      6 parity  = 48 MiB

  Therefore a parity block produced by the RAID kernel is normally not consumed
  immediately.

* A memory read of a 256 KiB block that is no longer cached is still very cheap
  compared with real storage I/O. Trying to keep many queued parity blocks hot
  in cache is therefore not necessarily useful.


MOVNT / non-temporal stores
---------------------------

* Generation backends use MOVNTDQ / VMOVNTDQ for parity output where applicable.

* The synthetic speedtest naturally favors non-temporal stores because generated
  parity is not consumed afterwards. This benchmark therefore measures producer
  throughput, not the complete producer -> queue -> writer pipeline.

* However, the real SnapRAID queue makes the workload much closer to the
  non-temporal case than to an immediate-consumer case.

* Cached stores could potentially help only with very short queues, where the
  writer consumes parity while it is still hot. Queue depths around 1..4 are the
  most plausible area where this could happen.

* Those very short queues are not representative of the real fastest disk
  pipeline. Real measurements already show that queues up to about 32 provide
  substantial improvements.

* At realistic queue depths, cached parity output would occupy many MiB of cache
  and could evict useful input data. MOVNT avoids this cache pollution and
  avoids normal cache allocation/write-allocate behavior for parity output.

* Conclusion: keep MOVNT for the normal generation path. Replacing MOVNT with
  cached stores is not considered an open optimization.


Typical recovery case
---------------------

* The common recovery case is one failed data disk while parity P is still
  available.

* raid_rec() selects the first available parity. Therefore, for a single data
  failure with P available, recovery uses P.

* REC1 implementations special-case ip[0] == 0 and call raid_rec1of1().

* raid_rec1of1() remaps pointers so that P is used as an XOR input and the
  failed data buffer is used as the GEN1 output, then calls
  raid_gen(nd, 1, ...).

* Therefore the common single-disk recovery path does NOT call
  raid_delta_gen().

* The data traffic is already close to the minimum:

      read P
      read all surviving data blocks
      XOR them
      write the recovered data block once

* There are no temporary syndrome buffers, no GF multiplication, and no matrix
  inversion in this common path.

* This means that large architectural recovery changes mostly optimize rare
  failure cases, not the usual single-disk recovery.

* Two simultaneous data disk failures are already an exceptional case in normal
  use. Three or more simultaneous data failures are even less important as
  optimization targets.


Recovery speedtest interpretation
---------------------------------

The recovery speed tests contain separate rows for the important fast paths and
for the generic recovery paths.

REC1OF1:

* The row uses ip[0] == 0.

* Every REC1 wrapper therefore enters raid_rec1of1() instead of executing its
  normal GF recovery body.

* The speed test explicitly selects the generation function used by the
  raid_rec1of1() call with raid_gen_force(1, ...).

* There is no GFNI GEN1 because PAR1 is only XOR. Therefore:

      rec1of1 avx2     -> raid_gen1_avx2
      rec1of1 gfni     -> raid_gen1_avx2

      rec1of1 avx512   -> raid_gen1_avx512bw
      rec1of1 gfni512  -> raid_gen1_avx512bw

  This is intentional. The GFNI recovery wrapper is selected, but the fast path
  naturally reduces to the best available XOR generator.

REC1:

* The generic REC1 row uses ip + 1 to avoid the PAR1 fast path.

* It therefore measures recovery from Q or a later parity and executes the
  backend-specific REC1 GF multiplication kernel.

REC2OF2:

* The row uses ip = { 0, 1 }, meaning P and Q.

* raid_rec2_int8() has an existing specialized P+Q fast path and calls
  raid_rec2of2_int8().

* SIMD REC2 implementations do not have the equivalent specialized P+Q
  fast path. They execute the generic 2x2 recovery algorithm with P and Q.

* This makes the row useful as a direct benchmark for deciding whether a
  specialized SIMD rec2of2 implementation is worthwhile.

REC2:

* The generic REC2 row uses ip + 1, avoiding the P+Q fast case.

* It measures the normal generic 2x2 recovery path.

Delta generation:

* Each recovery benchmark explicitly forces a matching generation backend with
  raid_gen_force() before entering the timed recovery function.

* This is important because raid_delta_gen() internally calls raid_gen().

* The benchmark therefore avoids accidentally comparing a recovery kernel while
  using an unrelated generation backend for its delta step.


raid_delta_gen() fusion
-----------------------

* A fused recovery design was studied, where selected syndromes would be
  accumulated directly instead of:

      raid_delta_gen()
      temporary parity output
      reload temporary parity
      solve recovery matrix

* The fused design could also compute only the parity rows actually selected by
  ip[].

* This matters because raid_delta_gen() currently computes:

      np = ip[nr - 1] + 1

  and raid_gen() can generate only a prefix of parity rows.

* Example:

      nr = 3
      ip = { 0, 2, 5 }

  requires GEN6 today even though only P, R and U are needed.

* A fused RECX could calculate exactly those three syndromes.

* The idea is technically valid and could reduce temporary memory traffic and
  unnecessary parity work.

* However, the common rec1of1/P case already bypasses raid_delta_gen() entirely.

* Because the main benefit applies to less common multiple-failure or missing-P
  cases, this direction was intentionally deprioritized.

* Decision: do not pursue raid_delta_gen() + REC fusion for now.


Generation: AVX512 width and unrolling
--------------------------------------

* Wider processing is not automatically faster.

* AVX512 128-byte processing was tested and can be slower than the existing
  64-byte path.

* Therefore do not assume that x2/x4 spatial unrolling used by some RAID
  engines is beneficial for SnapRAID.

* The block size is already large, memory traffic is substantial, and wider
  unrolling can increase register pressure, front-end work, and AVX512 power or
  frequency effects without reducing total memory traffic.

* Standalone VPTERNLOGQ transformations were tested on Zen 5 and did not improve
  performance.

* This is not a general rejection of VPTERNLOGQ. AVX512BW GEN2/GEN3
  kernels use VPTERNLOGQ where it is part of a larger table-residency
  restructuring that reduces other work. Judge the complete kernel, not an
  instruction in isolation.

* Conclusion: keep the conservative AVX512 processing width unless a
  specific benchmark proves otherwise.


Generation: Horner
------------------

* Classic RAID6 P+Q generation uses the Horner recurrence for Q with the
  standard RAID polynomial/generator.

* Non-GFNI AES Cauchy generation also evaluates Q with Horner's method. Its
  coefficient sequence uses multiplication by 2 almost everywhere, with
  multiplication by 3 only at the fixed G23 coset boundaries. The optimized
  implementation of this recurrence is described in the next section.

* A Horner-style AVX512BW non-GFNI implementation was tested on Zen 5 and did
  not improve performance.

* Therefore Horner is NOT considered an open optimization for the AVX512BW
  non-GFNI generation path.

* GFNI is different.

* For RAID GFNI generation, Horner Q is useful because multiplying by the fixed
  generator can use one fixed affine matrix instead of a different matrix for
  every data coefficient.

* RAID GFNI Horner Q is implemented as follows:

      AVX2 RAID GFNI GEN2..GEN5:
          Horner Q with a fixed multiply-by-2 affine matrix resident in a
          register.

      AVX2 RAID GFNI GEN6:
          keep the direct-coefficient implementation because register pressure
          is too high to keep the extra fixed matrix efficiently.

      AVX512 RAID GFNI GEN2..GEN6:
          Horner Q with the fixed matrix resident in a register.

      AES GFNI:
          unchanged, direct VGF2P8MULB coefficient multiplication.

* D0/final coefficient 1 is handled directly where appropriate.

* Conclusion: GFNI Horner Q is implemented where useful. Non-GFNI AVX512BW
  Horner was tested and rejected.


Generation: AES G23 x2/G23 split
--------------------------------

The AES Extended Cauchy Q sequence is evaluated in descending Horner order.

Most transitions multiply the Q accumulator by 2. Multiplication by 3 is
required only at the four boundaries between the five G23 cosets:

    d = 203, 152, 101, 50

Therefore arrays with at most 51 data disks never execute an x3 transition.
For nd = 51 the first Horner step is d = 49. The first x3 transition appears
only at nd = 52, where the first processed step is d = 50.

The dispatch condition is centralized in:

    raid_g23_x2_only(nd)

which currently means:

    nd <= 51

The value 51 is not merely a performance tuning threshold. It is the largest
data-disk count for which the AES G23 Q recurrence contains only x2
transitions. The threshold may safely be lowered for performance experiments,
but it cannot be increased without making the x2-only implementation
incorrect.

Earlier implementations kept x2 and G23 behavior in a single generator and
tested G23 state inside or around the hot data loop. This was correct, but it
hurt generated code even when no x3 transition was actually executed.

The current implementation therefore uses two separate paths:

    *_x2()
        branch-free Horner recurrence using only Q *= 2

    *_g23()
        segmented Horner recurrence using Q *= 2 between boundaries
        and Q *= 3 at the fixed G23 boundaries

AES wrappers dispatch before entering the hot kernel:

    if (raid_g23_x2_only(nd))
        *_x2(...);
    else
        *_g23(...);

RAID always selects the x2 path.

The G23 implementation computes the first relevant boundary from:

    raid_g23_boundary(l - 1)

where l = nd - 1.

Using l - 1 is important. Q is initialized directly from D[l], so D[l] does
not perform a Horner multiplication. The first transition occurs before adding
D[l - 1].

Between boundaries, the G23 implementations run dedicated x2-only loops.
The x3 transition is handled separately at the boundary. Implementations that
process two data disks per iteration must stop pairing before a boundary so
that a pair never crosses an x3 transition.

For GEN3 and higher, D0 is normally handled separately to avoid unnecessary
Cauchy-table work. Since the lowest G23 boundary is d = 50, the final D0
transition is always Q *= 2.

This separation was introduced for performance, not correctness. A generic
G23 implementation with no active boundary is mathematically equivalent to
the x2 path, but its additional control-flow structure can still produce
significantly worse machine code.

This was confirmed on an Intel family 6 model 165 CPU with GCC 11.2.0,
where forcing the generic G23 control flow on small arrays caused a measurable
throughput loss across SIMD backends even without executing any x3 transition.


Generation: coefficient residency
---------------------------------

Recent GEN2/GEN3 work established coefficient residency as a practical
generation optimization, not only a theoretical one.

* Non-GFNI SIMD GEN2/GEN3 kernels preload useful low/high multiplication tables
  into vector registers when the register file has enough space.

* The resident set can be partial. When there are more data disks than can be
  covered by spare registers, later disks fall back to loading their
  coefficient tables normally.

* GEN3 has higher register pressure because both Q and R coefficients are
  needed. Residency is therefore deliberately bounded by the available
  register file rather than forcing spills or another source pass.

* Appropriate GFNI generation paths also preload coefficient representations
  when doing so reduces repeated setup without increasing source traffic.

* AVX512BW can use VPTERNLOGQ as part of these resident-table kernels to combine
  the accumulator with low/high products. The useful optimization is the whole
  scheduling and residency change, not VPTERNLOGQ by itself.

* The source chunk is still loaded once and contributes to all parity outputs
  before moving on. The optimization therefore preserves the most important
  large-block memory property.

General conclusion:

* First use spare registers to remove repeated coefficient setup while
  preserving one-pass multi-output generation.

* Do not trade coefficient residency for extra 256 KiB source passes.

* Generation success does not imply that RECX can use the same register map.
  RECX must also keep multiple deltas and reconstruction state live, so its
  coefficient-residency opportunity remains narrower.

Overall generation conclusion:

* For the 1..6 parity implementation, no major generation optimization remains
  with a strong reason to be investigated.

* Further generation changes should require new hardware evidence, a changed
  workload, or measurements that identify a concrete bottleneck.


GFNI field implementation
-------------------------

* AES GFNI multiplication maps naturally to VGF2P8MULB because the instruction
  uses the AES polynomial.

* Standard RAID polynomial 0x11d cannot use VGF2P8MULB directly.

* RAID GFNI therefore uses VGF2P8AFFINEQB with precomputed affine matrices.

* The global affine matrix table stores 8 bytes per coefficient.

* A generic exhaustive software test validates all coefficient/input pairs
  against the normal RAID GF multiplication table.

* Dedicated hardware affine-matrix validators were intentionally not added.
  Normal generation/recovery tests exercise the hardware path.

* RAID and AES GFNI recovery/generation have polynomial-specific names:

      *_gfni_raid
      *_gfni_aes

  with backend tags:

      gfni
      gfni512

* This naming and registration split is intentional and should be preserved.


GFNI RECX coefficient precomputation
------------------------------------

RAID GFNI RECX previously performed this dependency chain for every coefficient
inside every data chunk:

      load coefficient byte from V[]
      use it to index raid_gfaffine_raid[]
      load the 8-byte affine matrix
      broadcast the matrix
      VGF2P8AFFINEQB

The current implementation precomputes:

      uint8_t C[N * N][8]

and copies the complete 8-byte affine representation for every inverse matrix
coefficient before entering the data loop.

The hot loop therefore becomes:

      direct load/broadcast from C[]
      VGF2P8AFFINEQB

This removes the coefficient-byte -> indexed affine-table dependency from the
hot loop while keeping the compact 8-byte representation.

Important details:

* All 8 bytes must be copied. An affine matrix coefficient is not one byte.

* Using a byte array and memcpy() keeps the representation explicit and avoids
  strict-aliasing assumptions.

* AVX2 and AVX512 RAID GFNI RECX both use this precomputation.

* AES GFNI does not need this conversion because VGF2P8MULB consumes the
  coefficient byte directly.

* The remaining repeated operation is the per-chunk broadcast of the 8-byte
  affine matrix.


GFNI pre-expanded affine tables
-------------------------------

* Some high-performance generic erasure-code engines use pre-expanded GFNI
  constants, for example a 32-byte representation ready for vector use.

* For SnapRAID this would increase the RAID affine table from about 2 KiB to
  about 8 KiB.

* With large 256 KiB processing blocks, the small 8-byte entries plus
  broadcast have a better cache-footprint tradeoff.

* No convincing reason was found to trade four times the table footprint for
  fewer broadcasts.

* Decision: keep compact 8-byte affine matrices.


External implementation comparisons
-----------------------------------

Detailed comparisons are in OTHERS. The implementations studied include
specialized RAID engines, filesystem RAID, and general Reed-Solomon libraries.

The important conclusions are general rather than project-specific.

Classify an external advantage before trying to copy it:

    algorithm advantage
        the method performs genuinely less mathematical work

    implementation advantage
        the same work is scheduled, vectorized, or prepared better

    workload-specific advantage
        the method wins because the external workload has different block,
        cache, parity, update, or latency assumptions

This distinction prevents a good optimization for another engine from becoming
an automatic SnapRAID TODO.

Memory traversal:

* Several modern high-throughput engines confirm that loading one source chunk
  and updating multiple outputs from it is a strong general strategy.

* Other engines deliberately process cache-resident regions and repeatedly
  update cached parity. That can be correct when parity is consumed again
  immediately, but it is a different memory model from SnapRAID generation.

* Whole-region operations that keep one coefficient resident can be efficient
  for generic erasure coding. For SnapRAID, the same loop orientation can
  require rereading large source blocks and is normally a poor trade.

* The preferred order remains: preserve the one-pass source traversal, then use
  spare registers, hoisting, and precomputation to reduce local coefficient
  cost.

Specialization:

* SnapRAID can specialize aggressively because parity is limited to 1..6 and
  the matrix structure is known.

* Generic engines may use large generated N-input/M-output kernel families,
  general decoding matrices, structured solvers, or broader runtime machinery.
  These are valuable for their scale but do not automatically reduce work in
  the N <= 6 SnapRAID domain.

* The most useful external ideas are usually local: fuse multiple outputs,
  precompute coefficient representations, reuse source splits, special-case
  coefficient 0/1 where simple, and exploit algebraic fast paths.

Cache and stores:

* Store policy follows output lifetime. Cached stores are appropriate when an
  algorithm immediately reloads parity; non-temporal stores are appropriate
  when final parity is queued and consumed later.

* Therefore cached stores in another fast implementation are not evidence that
  SnapRAID should change its MOVNT policy.

Recovery:

* General matrix inversion or structured Cauchy solvers become important at
  larger scale. For at most six missing blocks, moving 256 KiB regions is far
  more important than reducing the tiny matrix-setup cost.

* Algorithmic failure-case specializations are different: if a fixed recovery
  case provably needs fewer GF multiplications, it remains worth measuring.
  The specialized P+Q rec2of2 case is an example.

Runtime and architecture:

* Some engines benchmark implementations at startup, while others dispatch only
  from CPU capabilities. Runtime calibration remains low priority for SnapRAID
  because the kernel set is small and the workload is stable.

* Other engines support additional SIMD families such as SVE, RVV, or VSX.
  These are useful porting references, not hot-path optimization tasks
  without a target platform requirement.


Memory layout, displacement and stride noise
--------------------------------------------

* The memory layout is not a neutral detail.

* Fixed displacement between buffers was observed to help.

* STRIDE_NOISE behavior differs by CPU.

* On Intel, stride noise had little effect in the observed tests.

* On Zen 5, disabling stride noise caused a large throughput reduction.

* This indicates that hardware prefetch/cache behavior is already strongly
  affected by the chosen buffer layout.

* Therefore new software-prefetch or traversal changes should not be assumed to
  help. They must be benchmarked against the existing displacement/stride
  behavior.

* This is another reason to avoid adding extra passes over 256 KiB blocks.


REC1 optimizations
------------------

The important normal REC1 case is already raid_rec1of1() with P and is close to
the minimum possible memory traffic.

For generic REC1, used when P is unavailable:

* AVX2 processes 64 bytes per iteration using two independent 32-byte chains.

* SSSE3 x86-64 processes 64 bytes per iteration using four independent 16-byte
  chains. The x86-32 path keeps the smaller register-compatible fallback.

* NEON processes 64 bytes per iteration using four independent 16-byte chains.

The arithmetic work per byte is mostly unchanged. The purpose of the unrolling
is:

* reduce loop-control overhead
* expose more independent work
* improve instruction scheduling around shuffle/table operations

This is a local optimization, not an architectural change.

REC1 status:

* rec1of1/P is essentially closed.

* Generic non-P REC1 may still receive small local tuning if measurements show a
  clear benefit.

* No major REC1 algorithmic optimization is currently expected.


REC2 optimizations
------------------

Generic non-GFNI REC2 table/broadcast hoisting is implemented.

AVX2 x86-64:

* keep the 8 low/high shuffle tables for V0..V3 resident in ymm8..ymm15
* keep the low-nibble mask resident
* split d0/d1 once per chunk
* reuse the split values for both recovered outputs

AVX512BW:

* keep the 8 low/high shuffle tables resident in zmm16..zmm23
* keep the low-nibble mask resident
* split d0/d1 once per chunk
* reuse the split values for both recovered outputs

SSSE3 x86-64:

* keep all four coefficient low/high tables in XMM registers
* split both deltas once
* reuse the split values for both outputs

NEON:

* keep the four low/high coefficient tables in v20..v27
* keep the low-nibble mask in v28
* process two independent 16-byte blocks per iteration
* split both deltas once per block
* reuse the split values for both outputs
* the 32-byte unrolled kernel uses 25 vector registers

Important portability result:

* x86-32 has only the low eight XMM/YMM registers available to these old ISA
  paths.

* Optimizations that depend on xmm8..xmm15 or ymm8..ymm15 must remain restricted
  to CONFIG_X86_64.

REC2 status:

* Generic REC2 local table/split reuse is implemented.

* Scalar int8 already has specialized rec2of2 P+Q recovery.

* Specialized SIMD rec2of2 P+Q is a possible further optimization.


Specialized rec2of2 P+Q
-----------------------

For failures of two data disks recovered specifically with P and Q, the system
can be reduced to:

      Dy = C0 * Pd ^ C1 * Qd
      Dx = Pd ^ Dy

This needs only two GF multiplications for Dy, followed by one XOR for Dx,
instead of the generic 2x2 reconstruction with four GF multiplications.

The scalar int8 implementation already uses this specialization.

SIMD REC2 backends use the generic 2x2 matrix solution even when
ip = { 0, 1 }.

Potential targets include:

* SSSE3
* AVX2
* AVX512BW
* AVX2 GFNI
* AVX512 GFNI
* NEON

The new rec2of2 benchmark row exists specifically to measure whether these
specializations are worth adding.

Practical priority remains low because simultaneous loss of two data disks is an
exceptional real-world case.


RECX delta handling
-------------------

RECX previously had avoidable temporary-buffer and repeated split work.

The optimized structure keeps the delta values for each data
chunk in SIMD registers.

For non-GFNI paths:

* each delta is split into low/high nibbles once
* the split values remain in SIMD registers while all recovered outputs are
  calculated
* the same split values are reused across output rows
* stack temporary delta buffers are avoided on the optimized paths

For GFNI paths:

* the byte deltas remain directly in SIMD registers
* no nibble split is needed

This closes the old questions:

      can pd[k] stay in a register longer?
      can low/high nibble splits be reused across outputs?

For the optimized paths, the answer is yes and this work is implemented.


RECX shuffle-table pointer precomputation
----------------------------------------

Non-GFNI RECX previously performed coefficient address calculation inside the
hot loop:

      load coefficient byte from V[]
      multiply the coefficient index by 32
      add the raid_gfmulpshufb base address
      load low/high shuffle tables

The implementation precomputes in C:

      T[j] = &raid_gfmulpshufb[V[j]][0][0];

The hot assembly then only performs:

      load T[j]
      load or broadcast low table at T[j] + 0
      load or broadcast high table at T[j] + 16

This has two advantages:

1. It removes coefficient-index address arithmetic from the hot assembly.

2. It keeps C object layout and address calculation in C, leaving the assembly
   focused on SIMD operations.

Optimized users include:

* AVX2 RECX x86-64
* AVX512BW RECX
* SSSE3 RECX x86-64
* NEON RECX

For x86-32 AVX2/SSSE3 fallbacks, T[] is not unnecessarily built when the old
fallback body does not use it.

The remaining cost is that the low/high tables are still loaded or broadcast for
every coefficient of every data chunk.


RECX register state
-------------------

The RECX structure already uses registers aggressively.

AVX512BW:

* up to six delta values are kept as low/high pairs in zmm0..zmm11
* zmm12..zmm15 are reconstruction temporaries/accumulators
* zmm31 keeps the low-nibble mask
* coefficient shuffle tables are still broadcast from T[] during reconstruction

AVX512 RAID GFNI:

* up to six byte deltas are kept in zmm0..zmm5
* temporary/accumulator registers are used for reconstruction
* coefficient affine matrices come directly from precomputed C[]
* one 8-byte affine matrix is still broadcast per coefficient per chunk

AVX2 RAID GFNI:

* the two 32-byte halves of up to six delta blocks occupy most of the available
  YMM register file
* C[] removes indexed affine-table lookup
* register pressure leaves less room for keeping many coefficient matrices
  resident

AVX2/SSSE3 non-GFNI:

* the optimized x86-64 paths keep pre-split deltas in vector registers
* register pressure is significantly higher than AVX512
* coefficient table residency is therefore much more constrained

NEON:

* pre-split deltas are kept in vector registers
* T[] provides direct pointers to low/high multiplication tables
* reconstruction still loads each coefficient table pair as needed


RECX coefficient residency
--------------------------

The main RECX local optimization question is:

      can repeated coefficient loads/broadcasts be reduced by keeping useful
      coefficient tables or affine matrices resident in spare vector registers?

This is most plausible on AVX512 because of the larger register file.

Possible experiments:

* keep one output row of coefficients resident across data chunks

* keep a useful prefix of coefficient tables resident when N permits it

* on AVX512 RAID GFNI, keep several 8-byte affine matrices broadcast in ZMM
  registers instead of broadcasting them again for every chunk

* on AVX512BW, keep some low/high shuffle-table pairs resident

Constraints:

* Do not add a second pass over the 256 KiB block.

* Do not spill deltas or accumulators merely to save coefficient broadcasts.

* Do not add large N-specific kernel families unless measurements justify the
  complexity.

* AVX2 and SSSE3 register pressure is already high, so the likely opportunity is
  much smaller there.

* NEON has more architectural registers than old x86 SIMD but RECX already
  uses a significant part of them; any residency experiment should be
  measured carefully.

This is a narrow local problem. Delta residency, nibble reuse, coefficient
address precomputation, and GFNI affine lookup precomputation are already
implemented.


Recovery optimization policy
----------------------------

For recovery, avoid new architecture-level changes unless a very strong case
appears.

Preferred optimizations are basic/local:

* hoist loop-invariant values
* keep masks/tables/matrices resident when register pressure allows it
* avoid repeated loads or broadcasts
* initialize accumulators from the first useful term
* keep address calculations in C when doing so simplifies inline assembly
* exploit coefficient 0 or 1 only when it stays simple and does not add harmful
  branching or complexity
* use limited spatial unrolling only when it exposes ILP without excessive
  register pressure

Avoid unless new measurements provide a concrete reason:

* raid_delta_gen() fusion
* extra full passes over source blocks
* aggressive spatial unrolling
* software prefetch without clear evidence
* changing MOVNT policy
* major loop-orientation changes
* large pre-expanded lookup tables
* broad runtime calibration machinery


Recovery matrix setup and inversion
-----------------------------------

* Recovery matrices are at most 6x6 for the 1..6 parity implementation.

* The arithmetic cost of constructing and inverting such a matrix is tiny
  compared with reading and writing 256 KiB data regions.

* More elaborate structured Cauchy solvers, inversion caches, or asymptotically
  faster coding methods are therefore not useful performance targets for the
  1..6 parity limit.

* A dedicated multithreaded inversion stress test exists to validate the matrix
  code independently of the data-path SIMD kernels.

* At parity counts far beyond six, matrix construction and structured
  solvers should be re-evaluated together with the whole memory traversal.
  They should not be optimized in isolation.


Inline assembly and portability constraints
-------------------------------------------

The SIMD code intentionally uses multiple small or large asm volatile blocks and
keeps SIMD state live across them.

Important project constraints:

* Assembly blocks that dereference pointers internally must model memory effects
  appropriately. RECX pointer-based blocks use a "memory" clobber.

* The x86 SSE/AVX end helpers declare SIMD clobbers needed by the project ABI
  strategy, including the Windows x64 callee-saved XMM registers.

* The NEON end helper declares v0..v31 clobbered so that GCC/Clang preserve
  AAPCS64 callee-saved v8..v15 when required.

* The end-of-function dummy-clobber mechanism is a documented project technique.
  It is not a fully explicit extended-asm dataflow model, so future compiler
  portability changes should be reviewed carefully.

* Do not assume x86-32 has xmm8..xmm15 or ymm8..ymm15.

* AArch32 NEON is a separate backend. Do not assume the AArch64 register or ABI
  model when changing ARM32 assembly.

* RAID thread-safety requirements are documented. Avoid introducing mutable
  global calibration or dispatch state without preserving those requirements.

* RAID block sizes passed to the SIMD code are multiples of 64, so the
  32-byte and 64-byte unrolled loops do not need tail handling.

These are established implementation constraints, not correctness findings.


Scaling beyond 6 parity levels
------------------------------

The implementation deliberately targets at most six parity levels.

* At this scale, fixed-parity specialization keeps matrix setup small and
  enables tight kernels with explicit register allocation.

* General-purpose erasure-code engines demonstrate other designs, including
  generic matrix kernels, generated multi-input/multi-output kernels, and
  algorithms intended for much larger shard counts.

* Those designs solve a different scaling problem. They should not be adopted
  merely to generalize a domain where the fixed 1..6 structure is useful.

* If the parity limit is increased substantially, generic intrinsics or
  generated kernels should be compared with handwritten specialization.
  Memory traversal
  must remain a first-class design constraint.

* Scaling beyond six parity levels is primarily a functional and architectural
  change, not an unresolved performance problem of the 1..6 implementation.


Short list of ideas already tried or rejected
---------------------------------------------

* AVX512 128-byte generation processing:
  tried, can be slower.

* Non-GFNI AVX512BW Horner:
  tried on Zen 5, no improvement.

* Standalone VPTERNLOGQ substitutions:
  tried on Zen 5, no improvement. This does not reject integrated use as part
  of the profitable GEN2/GEN3 resident-table kernels.

* VPBLENDVB x2:
  tried, no benefit / worse.

* MOVNT -> cached stores:
  not appropriate for the normal queued disk-write workload; keep MOVNT.

* GFNI 32-byte pre-expanded affine constants:
  rejected due to larger cache footprint.

* Split outputs into multiple full source passes:
  rejected because 256 KiB source blocks would be reread.

* Whole-region/output-major rewrites intended mainly to keep one coefficient
  resident:
  poor fit when they add source rereads. Use register residency inside the
  existing traversal instead.

* RECX source-major rewrite:
  low priority; hot tiny delta working set and register-pressure tradeoff.

* raid_delta_gen() + REC fusion:
  technically valid but intentionally deprioritized because the normal
  rec1of1/P case bypasses raid_delta_gen() already.

* Software prefetch:
  no strong reason to add it given the existing long streams and proven
  sensitivity to displacement/stride tuning.

* Runtime backend microbenchmark selection:
  possible, but low value relative to complexity.

* Large generic N-input/M-output kernel families:
  useful for general erasure coding, but not justified for the fixed
  1..6 parity domain without a concrete benchmark need.

* Structured or asymptotically faster matrix solvers:
  unnecessary at N <= 6 because matrix setup is tiny compared with data
  movement.