snapraid/raid/NOTES

SnapRAID RAID library performance notes
=======================================

Scope
-----

These notes summarize performance experiments, implementation choices, rejected
ideas, and non-obvious conclusions reached while reviewing and tuning the
SnapRAID RAID library.

The focus is the current 1..6 parity implementation, especially x86 AVX2,
AVX512BW and GFNI backends.

This is intentionally a record of what has already been tried and what was
learned from it, so that the same ideas are not repeatedly re-investigated.


Real workload assumptions
-------------------------

* SnapRAID processes large blocks: typically 256 KiB per data/parity block.

* A normal stripe therefore touches a large amount of memory. With 12 data
  disks, one 256 KiB stripe already means about 3 MiB of data input before
  considering parity output.

* This large-block behavior is very important. Optimizations that reread a
  source buffer, increase the number of passes, or enlarge lookup tables can be
  worse even if the inner instruction sequence looks better.

* SnapRAID uses a queue before parity data is written to disk. Real measurements
  with actual disk writes show that very small queues are worse. Performance
  improves clearly up to about queue depth 32, and still improves after that,
  but with diminishing returns.

* With 256 KiB blocks, queue depth 32 means this much parity output can be in
  flight:

      1 parity  =  8 MiB
      2 parity  = 16 MiB
      3 parity  = 24 MiB
      6 parity  = 48 MiB

  Therefore the parity block produced by the RAID kernel is normally not
  consumed immediately.

* A memory read of a 256 KiB block that is no longer cached is still very cheap
  compared with real storage I/O. Trying to keep many queued parity blocks hot
  in cache is therefore not necessarily useful.


MOVNT / non-temporal stores
---------------------------

* Generation backends use MOVNTDQ / VMOVNTDQ for parity output.

* The synthetic speedtest naturally favors non-temporal stores because the
  generated parity is not consumed afterwards. This benchmark therefore
  measures producer throughput, not the complete producer -> queue -> writer
  pipeline.

* However, the real SnapRAID queue makes the workload much closer to the
  non-temporal case than to an immediate-consumer case.

* Cached stores could potentially help only with very short queues, where the
  writer consumes the parity while it is still hot. Queue depths around 1..4
  are the most plausible area where this could happen.

* Those very short queues are not representative of the real fastest disk
  pipeline. Real measurements already show that queues up to about 32 provide
  substantial improvements.

* At realistic queue depths, cached parity output would occupy many MiB of
  cache and could evict useful input data. MOVNT avoids this cache pollution and
  avoids normal cache allocation/write-allocate behavior for the parity output.

* Conclusion: keep MOVNT for the normal generation path. We do not consider
  replacing MOVNT with cached stores an open optimization anymore.


Typical recovery case
---------------------

* The common recovery case is one failed data disk, while parity P is still
  available.

* raid_rec() selects the first available parity. Therefore, for a single data
  failure with P available, recovery uses P.

* raid_rec1_avx2()/raid_rec1_avx512bw() special-case ip[0] == 0 and call
  raid_rec1of1().

* raid_rec1of1() is already effectively fused. It remaps pointers so that P is
  used as one of the XOR inputs and the failed data buffer is used as GEN1
  output, then calls raid_gen(nd, 1, ...).

* Therefore the common single-disk recovery path does NOT call
  raid_delta_gen().

* The data traffic is already close to the minimum:

      read P
      read all surviving data blocks
      XOR them
      write the recovered data block once

* There are no temporary syndrome buffers, no GF multiplication, and no
  matrix inversion in this common path.

* This means that large architectural recovery changes mostly optimize rare
  failure cases, not the usual single-disk recovery.

* Two simultaneous data disk failures are already an exceptional case in
  normal use. Three or more simultaneous data failures are even less important
  as optimization targets.


raid_delta_gen() fusion
-----------------------

* A fused recovery design was studied, where the selected syndromes would be
  accumulated directly instead of:

      raid_delta_gen()
      temporary parity output
      reload temporary parity
      solve recovery matrix

* The fused design could also compute only the parity rows actually selected by
  ip[].

* This matters because raid_delta_gen() currently computes:

      np = ip[nr - 1] + 1

  and raid_gen() can generate only a prefix of parity rows.

* Example:

      nr = 3
      ip = { 0, 2, 5 }

  requires GEN6 today even though only P, R and U are needed.

* A fused RECX could calculate exactly those three syndromes.

* The idea is technically valid and could reduce temporary memory traffic and
  unnecessary parity work.

* However, the common rec1of1/P case already bypasses raid_delta_gen() entirely.

* Because the main benefit would apply to less common multiple-failure or
  missing-P cases, this direction was intentionally deprioritized.

* Decision: do not pursue raid_delta_gen() + REC fusion for now.


Generation: AVX512 width and unrolling
--------------------------------------

* Wider processing is not automatically faster.

* AVX512 128-byte processing was tested and can be slower than the existing
  64-byte path.

* Therefore do not assume that x2/x4 spatial unrolling, like some Linux RAID6
  kernels use, is beneficial for SnapRAID.

* The block size is already large, memory traffic is substantial, and wider
  unrolling can increase register pressure, front-end work, and AVX512 power/
  frequency effects without reducing total memory traffic.

* VPTERNLOGQ based alternatives were also tested on Zen 5 and did not improve
  performance.

* Conclusion: keep the current conservative spatial width unless a specific
  benchmark proves otherwise.


Generation: Horner
------------------

* Linux RAID6 uses the classic Horner recurrence for Q with the standard RAID
  polynomial/generator.

* A Horner-style AVX512BW non-GFNI implementation was tested on Zen 5 and did
  not improve performance.

* Therefore Horner is NOT considered an open optimization for the AVX512BW
  non-GFNI generation path.

* GFNI is different.

* For RAID GFNI generation, Horner Q is useful because multiplying by the fixed
  generator can use one fixed affine matrix instead of a different matrix for
  every data coefficient.

* RAID GFNI Horner Q was implemented as follows:

      AVX2 RAID GFNI GEN2..GEN5:
          Horner Q with fixed multiply-by-2 affine matrix resident in a register.

      AVX2 RAID GFNI GEN6:
          keep the older direct-coefficient implementation because register
          pressure is too high to keep the extra fixed matrix efficiently.

      AVX512 RAID GFNI GEN2..GEN6:
          Horner Q with the fixed matrix resident in a register.

      AES GFNI:
          unchanged, direct VGF2P8MULB coefficient multiplication.

* D0/final coefficient 1 is handled directly where appropriate.

* Conclusion: GFNI Horner Q is implemented where useful. Non-GFNI AVX512BW
  Horner was tested and rejected.


GFNI field implementation
-------------------------

* AES GFNI multiplication maps naturally to VGF2P8MULB because the instruction
  uses the AES polynomial.

* Standard RAID polynomial 0x11d cannot use VGF2P8MULB directly.

* RAID GFNI therefore uses VGF2P8AFFINEQB with precomputed affine matrices.

* The affine matrix table stores 8 bytes per coefficient.

* A generic exhaustive software test validates all coefficient/input pairs
  against the normal RAID GF multiplication table.

* Dedicated hardware affine-matrix validators were intentionally not added.
  Normal generation/recovery tests exercise the hardware path.

* RAID and AES GFNI recovery/generation have polynomial-specific names:

      *_gfni_raid
      *_gfni_aes

  with backend tags:

      gfni
      gfni512

* This naming and registration split is intentional and should be preserved.


GFNI pre-expanded affine tables
-------------------------------

* Intel ISA-L can use pre-expanded GFNI constants, for example a 32-byte
  representation that is ready for vector use.

* For SnapRAID this would increase the RAID affine table from about 2 KiB to
  about 8 KiB.

* With large 256 KiB processing blocks, the current small 8-byte entries plus
  broadcast have a better cache-footprint tradeoff.

* No convincing reason was found to trade four times the table footprint for
  fewer broadcasts.

* Decision: keep compact 8-byte affine matrices.


Intel ISA-L comparison
----------------------

The following ISA-L ideas were examined.

1. Horner Q for GFNI RAID generation

   Useful idea. Implemented for RAID GFNI where register pressure allows it.

2. AVX512 GFNI 128-byte processing

   Tested. It can be slower. Do not use wider processing simply because more
   ZMM registers are available.

3. Source-major generic recovery loops

   Initially interesting because one source could feed several outputs.

   For the current RECX structure, the delta/syndrome working set is tiny and
   hot. Reordering loops does not obviously save important memory traffic and
   can increase register pressure.

   This is considered low priority / not useful enough.

4. Split output processing, for example 3+3 rows

   Bad fit for SnapRAID 256 KiB blocks because it requires rereading the source
   range in another pass.

   Avoid optimizations that save registers by adding full source passes.

5. Pre-expanded 32-byte GFNI constants

   Rejected because of the larger table/cache footprint described above.

6. VPBLENDVB based x2 techniques

   Tested. No benefit / worse performance.

7. Cached parity stores

   ISA-L style normal stores are not automatically suitable for SnapRAID.
   SnapRAID has large output blocks and a real queued disk-write pipeline.
   MOVNT remains the better design choice for this workload.

8. Multiple-row GFNI macros

   Interesting in isolation, but not enough reason to reread 256 KiB source
   blocks or enlarge table footprint.

9. Broad runtime dispatch/calibration

   ISA-L has broad dispatch machinery. SnapRAID already dispatches based on CPU
   features. A runtime microbenchmark selector is possible, but is not currently
   considered worth the extra complexity for the known kernels.

10. Incremental parity APIs

   Useful in other applications, but not a missing optimization in the current
   full-block SnapRAID parity-generation hot path.

Overall ISA-L conclusion:

* The important transferable idea was GFNI Horner Q, and that is already
  implemented.

* The other ISA-L techniques examined either lose on the current benchmark,
  increase source rereads, increase cache footprint, or do not match the real
  SnapRAID workload.


Linux kernel RAID6 comparison
-----------------------------

The Linux RAID6 implementation was studied as another optimization reference.

Relevant Linux characteristics include:

* x1/x2/x4 spatial kernels
* wider AVX512 processing
* explicit prefetch
* independent chains for ILP
* classic Horner Q
* runtime generator selection
* xor_syndrome/incremental interfaces
* specialized P+Q two-disk recovery

Conclusions for SnapRAID:

* Wider x2/x4 AVX512 processing is not automatically useful. 128-byte variants
  were tested and can be slower.

* Horner AVX512BW non-GFNI was tested on Zen 5 and did not improve.

* VPTERNLOGQ alternatives were tested on Zen 5 and did not improve.

* Explicit software prefetch is not currently considered a strong candidate.
  The workload already consists of long regular streams, and the existing
  memory-layout tuning is important.

* Runtime microbenchmark-based backend selection would add complexity and is
  low priority.

* xor_syndrome/incremental parity is a different API/use case and does not
  directly improve the current full-block generation path.

* The one Linux-style algorithmic recovery optimization still worth keeping as
  a TODO is specialized P+Q two-data recovery (rec2of2), but its practical
  priority is low because two simultaneous data failures are rare.


Memory layout, displacement and stride noise
--------------------------------------------

* The memory layout is not a neutral detail.

* Fixed displacement between buffers was observed to help.

* STRIDE_NOISE behavior differs by CPU.

* On Intel, stride noise had little effect in the observed tests.

* On Zen 5, disabling stride noise caused a large throughput reduction.

* This indicates that hardware prefetch/cache behavior is already strongly
  affected by the chosen buffer layout.

* Therefore new software-prefetch or traversal changes should not be assumed to
  help. They must be benchmarked against the existing displacement/stride
  behavior.

* This is another reason to avoid adding extra passes over 256 KiB blocks.


Non-GFNI REC2 table hoisting
----------------------------

A concrete successful recovery micro-optimization was implemented.

Before:

* multiplication shuffle tables could be broadcast repeatedly inside the
  32/64-byte loop
* delta low/high nibbles could be recomputed for different outputs

After:

AVX2 x86-64:

* keep the 8 low/high shuffle tables for V0..V3 resident in ymm8..ymm15
* keep the low-nibble mask resident
* split d0/d1 once per chunk
* reuse the split values for both recovered outputs

AVX512BW:

* keep the 8 low/high shuffle tables resident in zmm16..zmm23
* keep the low-nibble mask resident
* split d0/d1 once per chunk
* reuse them for both outputs

Important portability result:

* The first AVX2 version used ymm8..ymm15 unconditionally.

* That is invalid for 32-bit x86, where only ymm0..ymm7 are available.

* The final implementation keeps the optimized resident-table path only under
  CONFIG_X86_64 and preserves the old REC2 body for CONFIG_X86_32.

* AVX512BW is already x86-64-only, so zmm16..zmm23 are safe there.

This is the preferred class of future recovery optimization:
same algorithm, same stride, same memory traffic, fewer repeated loads/broadcasts
and better register reuse.


Recovery optimization policy from now on
-----------------------------------------

For recovery, avoid new architecture-level changes unless a very strong case
appears.

Preferred optimizations are basic/local:

* hoist loop-invariant values
* preload coefficient tables into registers
* keep masks/matrices resident when register pressure allows it
* reuse low/high nibble splits
* avoid repeated loads of the same hot temporary value
* avoid repeated broadcasts
* initialize accumulators from the first useful term when possible instead of
  zeroing and XORing unnecessarily
* exploit coefficient 0 or 1 only when it is simple and does not add harmful
  branching/complexity

Avoid for now:

* raid_delta_gen() fusion
* extra full passes over source blocks
* aggressive spatial unrolling
* software prefetch without clear evidence
* changing MOVNT policy
* major loop-orientation changes
* large pre-expanded lookup tables


REC1 status
-----------

* The important normal case is already raid_rec1of1() with P.

* That path is essentially XOR-only and close to the minimum possible memory
  traffic.

* There is little algorithmic work left to remove.

* More elaborate REC1 work is mainly relevant when P itself is unavailable and
  Q/R/S/T/U must be used.

* Those cases are much less common than the normal single-disk + P recovery.


REC2 status
-----------

* Generic non-GFNI REC2 table/broadcast hoisting is implemented.

* A specialized rec2of2 P+Q formula remains a TODO:

      Dy = C0 * Pd ^ C1 * Qd
      Dx = Pd ^ Dy

  instead of the generic 2x2 solution with four GF multiplications.

* This should be implemented, if desired, for AVX2, AVX512 and GFNI.

* Practical priority is low because simultaneous loss of two data disks is an
  exceptional case.


RECX status
-----------

* RECX is the main remaining place to look for basic register-reuse
  optimizations.

* The interesting questions are local:

      can pd[k] stay in a register longer?
      can low/high nibble splits be reused across outputs?
      can some multiplication tables remain resident?
      can repeated broadcasts be removed?

* AVX512 has enough registers to make some of these experiments plausible.

* AVX2 is much more constrained, so any attempt must be careful not to trade
  a few broadcasts for spills or a second source pass.

* Source-major loop inversion is not currently considered a priority because
  the pd working set is tiny/hot and the register-pressure tradeoff is not
  obviously favorable.


Generation status
-----------------

For current 1..6 parity generation, performance work is considered largely
closed.

Already covered:

* scalar/int paths
* SSE/SSSE3/AVX2/AVX512 paths
* GFNI RAID affine generation
* GFNI AES direct multiplication
* RAID GFNI Horner Q
* AVX512 width experiments
* VPTERNLOGQ experiments
* VPBLENDVB experiments
* MOVNT policy
* memory displacement/stride behavior
* ISA-L comparisons
* Linux RAID6 comparisons

No major generation optimization remains that currently has a strong reason to
be tested.


More than 6 parity levels
-------------------------

* Supporting more than 6 parity levels remains a future feature.

* A generic implementation using SSSE3/AVX intrinsics could be compared with
  the current handwritten assembler style.

* This is primarily a functional extension, not an unresolved performance issue
  for the current 1..6 parity library.


Current practical priority
--------------------------

1. Recovery only: continue looking for small invariant-hoisting/register-reuse
   optimizations.

2. RECX AVX512BW is the most natural place to inspect for repeated
   load/split/broadcast work.

3. Keep specialized rec2of2 P+Q as a correctness/performance TODO, but with low
   real-world priority.

4. Do not reopen generation-width, MOVNT, Horner AVX512BW, VPTERNLOGQ,
   VPBLENDVB, large GFNI-table, source-reread, or raid_delta_gen-fusion ideas
   unless new hardware or new measurements provide a concrete reason.


Short list of ideas already tried or rejected
---------------------------------------------

* AVX512 128-byte processing:
  tried, can be slower.

* AVX512BW Horner:
  tried on Zen 5, no improvement.

* VPTERNLOGQ optimization:
  tried on Zen 5, no improvement.

* VPBLENDVB x2:
  tried, no benefit / worse.

* MOVNT -> cached stores:
  not appropriate for the normal queued disk-write workload; keep MOVNT.

* GFNI 32-byte pre-expanded affine constants:
  rejected due to larger cache footprint.

* split GFNI outputs into multiple passes:
  rejected because 256 KiB source blocks would be reread.

* RECX source-major rewrite:
  low priority; hot tiny pd working set and register-pressure tradeoff.

* raid_delta_gen() + REC fusion:
  technically valid but intentionally deprioritized because the normal
  rec1of1/P case bypasses raid_delta_gen() already.

* software prefetch:
  no strong reason to add it given the existing long streams and proven
  sensitivity to displacement/stride tuning.

* runtime backend microbenchmark selection:
  possible, but low value relative to complexity.

* Linux x2/x4 style wide spatial kernels:
  not assumed useful; wider AVX512 already tested negatively.

* ISA-L cached-store behavior:
  not a good match for the real SnapRAID parity queue.

* ISA-L source reread / split-output strategies:
  poor fit for 256 KiB blocks.

* specialized P+Q rec2of2:
  still open, but low practical priority.


End state
---------

For the current 1..6 parity RAID library:

    Generation:       effectively closed
    GFNI generation:  effectively closed
    Store policy:     closed, keep MOVNT
    Memory layout:    tuned; do not casually change
    Recovery:         only basic/local micro-optimizations remain
    rec2of2 P+Q:      open TODO, low real-world priority
    >6 parity:        future feature

The most important workload fact to remember is that the normal recovery is a
single failed data disk with P available, which goes through raid_rec1of1() and
is already a direct XOR-style recovery with minimal extra work.