Add NOTES

raid/NOTES

@@ -0,0 +1,652 @@
+SnapRAID RAID library performance notes
+=======================================
+
+Scope
+-----
+
+These notes summarize performance experiments, implementation choices, rejected
+ideas, and non-obvious conclusions reached while reviewing and tuning the
+SnapRAID RAID library.
+
+The focus is the current 1..6 parity implementation, especially x86 AVX2,
+AVX512BW and GFNI backends.
+
+This is intentionally a record of what has already been tried and what was
+learned from it, so that the same ideas are not repeatedly re-investigated.
+
+
+Real workload assumptions
+-------------------------
+
+* SnapRAID processes large blocks: typically 256 KiB per data/parity block.
+
+* A normal stripe therefore touches a large amount of memory. With 12 data
+  disks, one 256 KiB stripe already means about 3 MiB of data input before
+  considering parity output.
+
+* This large-block behavior is very important. Optimizations that reread a
+  source buffer, increase the number of passes, or enlarge lookup tables can be
+  worse even if the inner instruction sequence looks better.
+
+* SnapRAID uses a queue before parity data is written to disk. Real measurements
+  with actual disk writes show that very small queues are worse. Performance
+  improves clearly up to about queue depth 32, and still improves after that,
+  but with diminishing returns.
+
+* With 256 KiB blocks, queue depth 32 means this much parity output can be in
+  flight:
+
+      1 parity  =  8 MiB
+      2 parity  = 16 MiB
+      3 parity  = 24 MiB
+      6 parity  = 48 MiB
+
+  Therefore the parity block produced by the RAID kernel is normally not
+  consumed immediately.
+
+* A memory read of a 256 KiB block that is no longer cached is still very cheap
+  compared with real storage I/O. Trying to keep many queued parity blocks hot
+  in cache is therefore not necessarily useful.
+
+
+MOVNT / non-temporal stores
+---------------------------
+
+* Generation backends use MOVNTDQ / VMOVNTDQ for parity output.
+
+* The synthetic speedtest naturally favors non-temporal stores because the
+  generated parity is not consumed afterwards. This benchmark therefore
+  measures producer throughput, not the complete producer -> queue -> writer
+  pipeline.
+
+* However, the real SnapRAID queue makes the workload much closer to the
+  non-temporal case than to an immediate-consumer case.
+
+* Cached stores could potentially help only with very short queues, where the
+  writer consumes the parity while it is still hot. Queue depths around 1..4
+  are the most plausible area where this could happen.
+
+* Those very short queues are not representative of the real fastest disk
+  pipeline. Real measurements already show that queues up to about 32 provide
+  substantial improvements.
+
+* At realistic queue depths, cached parity output would occupy many MiB of
+  cache and could evict useful input data. MOVNT avoids this cache pollution and
+  avoids normal cache allocation/write-allocate behavior for the parity output.
+
+* Conclusion: keep MOVNT for the normal generation path. We do not consider
+  replacing MOVNT with cached stores an open optimization anymore.
+
+
+Typical recovery case
+---------------------
+
+* The common recovery case is one failed data disk, while parity P is still
+  available.
+
+* raid_rec() selects the first available parity. Therefore, for a single data
+  failure with P available, recovery uses P.
+
+* raid_rec1_avx2()/raid_rec1_avx512bw() special-case ip[0] == 0 and call
+  raid_rec1of1().
+
+* raid_rec1of1() is already effectively fused. It remaps pointers so that P is
+  used as one of the XOR inputs and the failed data buffer is used as GEN1
+  output, then calls raid_gen(nd, 1, ...).
+
+* Therefore the common single-disk recovery path does NOT call
+  raid_delta_gen().
+
+* The data traffic is already close to the minimum:
+
+      read P
+      read all surviving data blocks
+      XOR them
+      write the recovered data block once
+
+* There are no temporary syndrome buffers, no GF multiplication, and no
+  matrix inversion in this common path.
+
+* This means that large architectural recovery changes mostly optimize rare
+  failure cases, not the usual single-disk recovery.
+
+* Two simultaneous data disk failures are already an exceptional case in
+  normal use. Three or more simultaneous data failures are even less important
+  as optimization targets.
+
+
+raid_delta_gen() fusion
+-----------------------
+
+* A fused recovery design was studied, where the selected syndromes would be
+  accumulated directly instead of:
+
+      raid_delta_gen()
+      temporary parity output
+      reload temporary parity
+      solve recovery matrix
+
+* The fused design could also compute only the parity rows actually selected by
+  ip[].
+
+* This matters because raid_delta_gen() currently computes:
+
+      np = ip[nr - 1] + 1
+
+  and raid_gen() can generate only a prefix of parity rows.
+
+* Example:
+
+      nr = 3
+      ip = { 0, 2, 5 }
+
+  requires GEN6 today even though only P, R and U are needed.
+
+* A fused RECX could calculate exactly those three syndromes.
+
+* The idea is technically valid and could reduce temporary memory traffic and
+  unnecessary parity work.
+
+* However, the common rec1of1/P case already bypasses raid_delta_gen() entirely.
+
+* Because the main benefit would apply to less common multiple-failure or
+  missing-P cases, this direction was intentionally deprioritized.
+
+* Decision: do not pursue raid_delta_gen() + REC fusion for now.
+
+
+Generation: AVX512 width and unrolling
+--------------------------------------
+
+* Wider processing is not automatically faster.
+
+* AVX512 128-byte processing was tested and can be slower than the existing
+  64-byte path.
+
+* Therefore do not assume that x2/x4 spatial unrolling, like some Linux RAID6
+  kernels use, is beneficial for SnapRAID.
+
+* The block size is already large, memory traffic is substantial, and wider
+  unrolling can increase register pressure, front-end work, and AVX512 power/
+  frequency effects without reducing total memory traffic.
+
+* VPTERNLOGQ based alternatives were also tested on Zen 5 and did not improve
+  performance.
+
+* Conclusion: keep the current conservative spatial width unless a specific
+  benchmark proves otherwise.
+
+
+Generation: Horner
+------------------
+
+* Linux RAID6 uses the classic Horner recurrence for Q with the standard RAID
+  polynomial/generator.
+
+* A Horner-style AVX512BW non-GFNI implementation was tested on Zen 5 and did
+  not improve performance.
+
+* Therefore Horner is NOT considered an open optimization for the AVX512BW
+  non-GFNI generation path.
+
+* GFNI is different.
+
+* For RAID GFNI generation, Horner Q is useful because multiplying by the fixed
+  generator can use one fixed affine matrix instead of a different matrix for
+  every data coefficient.
+
+* RAID GFNI Horner Q was implemented as follows:
+
+      AVX2 RAID GFNI GEN2..GEN5:
+          Horner Q with fixed multiply-by-2 affine matrix resident in a register.
+
+      AVX2 RAID GFNI GEN6:
+          keep the older direct-coefficient implementation because register
+          pressure is too high to keep the extra fixed matrix efficiently.
+
+      AVX512 RAID GFNI GEN2..GEN6:
+          Horner Q with the fixed matrix resident in a register.
+
+      AES GFNI:
+          unchanged, direct VGF2P8MULB coefficient multiplication.
+
+* D0/final coefficient 1 is handled directly where appropriate.
+
+* Conclusion: GFNI Horner Q is implemented where useful. Non-GFNI AVX512BW
+  Horner was tested and rejected.
+
+
+GFNI field implementation
+-------------------------
+
+* AES GFNI multiplication maps naturally to VGF2P8MULB because the instruction
+  uses the AES polynomial.
+
+* Standard RAID polynomial 0x11d cannot use VGF2P8MULB directly.
+
+* RAID GFNI therefore uses VGF2P8AFFINEQB with precomputed affine matrices.
+
+* The affine matrix table stores 8 bytes per coefficient.
+
+* A generic exhaustive software test validates all coefficient/input pairs
+  against the normal RAID GF multiplication table.
+
+* Dedicated hardware affine-matrix validators were intentionally not added.
+  Normal generation/recovery tests exercise the hardware path.
+
+* RAID and AES GFNI recovery/generation have polynomial-specific names:
+
+      *_gfni_raid
+      *_gfni_aes
+
+  with backend tags:
+
+      gfni
+      gfni512
+
+* This naming and registration split is intentional and should be preserved.
+
+
+GFNI pre-expanded affine tables
+-------------------------------
+
+* Intel ISA-L can use pre-expanded GFNI constants, for example a 32-byte
+  representation that is ready for vector use.
+
+* For SnapRAID this would increase the RAID affine table from about 2 KiB to
+  about 8 KiB.
+
+* With large 256 KiB processing blocks, the current small 8-byte entries plus
+  broadcast have a better cache-footprint tradeoff.
+
+* No convincing reason was found to trade four times the table footprint for
+  fewer broadcasts.
+
+* Decision: keep compact 8-byte affine matrices.
+
+
+Intel ISA-L comparison
+----------------------
+
+The following ISA-L ideas were examined.
+
+1. Horner Q for GFNI RAID generation
+
+   Useful idea. Implemented for RAID GFNI where register pressure allows it.
+
+2. AVX512 GFNI 128-byte processing
+
+   Tested. It can be slower. Do not use wider processing simply because more
+   ZMM registers are available.
+
+3. Source-major generic recovery loops
+
+   Initially interesting because one source could feed several outputs.
+
+   For the current RECX structure, the delta/syndrome working set is tiny and
+   hot. Reordering loops does not obviously save important memory traffic and
+   can increase register pressure.
+
+   This is considered low priority / not useful enough.
+
+4. Split output processing, for example 3+3 rows
+
+   Bad fit for SnapRAID 256 KiB blocks because it requires rereading the source
+   range in another pass.
+
+   Avoid optimizations that save registers by adding full source passes.
+
+5. Pre-expanded 32-byte GFNI constants
+
+   Rejected because of the larger table/cache footprint described above.
+
+6. VPBLENDVB based x2 techniques
+
+   Tested. No benefit / worse performance.
+
+7. Cached parity stores
+
+   ISA-L style normal stores are not automatically suitable for SnapRAID.
+   SnapRAID has large output blocks and a real queued disk-write pipeline.
+   MOVNT remains the better design choice for this workload.
+
+8. Multiple-row GFNI macros
+
+   Interesting in isolation, but not enough reason to reread 256 KiB source
+   blocks or enlarge table footprint.
+
+9. Broad runtime dispatch/calibration
+
+   ISA-L has broad dispatch machinery. SnapRAID already dispatches based on CPU
+   features. A runtime microbenchmark selector is possible, but is not currently
+   considered worth the extra complexity for the known kernels.
+
+10. Incremental parity APIs
+
+   Useful in other applications, but not a missing optimization in the current
+   full-block SnapRAID parity-generation hot path.
+
+Overall ISA-L conclusion:
+
+* The important transferable idea was GFNI Horner Q, and that is already
+  implemented.
+
+* The other ISA-L techniques examined either lose on the current benchmark,
+  increase source rereads, increase cache footprint, or do not match the real
+  SnapRAID workload.
+
+
+Linux kernel RAID6 comparison
+-----------------------------
+
+The Linux RAID6 implementation was studied as another optimization reference.
+
+Relevant Linux characteristics include:
+
+* x1/x2/x4 spatial kernels
+* wider AVX512 processing
+* explicit prefetch
+* independent chains for ILP
+* classic Horner Q
+* runtime generator selection
+* xor_syndrome/incremental interfaces
+* specialized P+Q two-disk recovery
+
+Conclusions for SnapRAID:
+
+* Wider x2/x4 AVX512 processing is not automatically useful. 128-byte variants
+  were tested and can be slower.
+
+* Horner AVX512BW non-GFNI was tested on Zen 5 and did not improve.
+
+* VPTERNLOGQ alternatives were tested on Zen 5 and did not improve.
+
+* Explicit software prefetch is not currently considered a strong candidate.
+  The workload already consists of long regular streams, and the existing
+  memory-layout tuning is important.
+
+* Runtime microbenchmark-based backend selection would add complexity and is
+  low priority.
+
+* xor_syndrome/incremental parity is a different API/use case and does not
+  directly improve the current full-block generation path.
+
+* The one Linux-style algorithmic recovery optimization still worth keeping as
+  a TODO is specialized P+Q two-data recovery (rec2of2), but its practical
+  priority is low because two simultaneous data failures are rare.
+
+
+Memory layout, displacement and stride noise
+--------------------------------------------
+
+* The memory layout is not a neutral detail.
+
+* Fixed displacement between buffers was observed to help.
+
+* STRIDE_NOISE behavior differs by CPU.
+
+* On Intel, stride noise had little effect in the observed tests.
+
+* On Zen 5, disabling stride noise caused a large throughput reduction.
+
+* This indicates that hardware prefetch/cache behavior is already strongly
+  affected by the chosen buffer layout.
+
+* Therefore new software-prefetch or traversal changes should not be assumed to
+  help. They must be benchmarked against the existing displacement/stride
+  behavior.
+
+* This is another reason to avoid adding extra passes over 256 KiB blocks.
+
+
+Non-GFNI REC2 table hoisting
+----------------------------
+
+A concrete successful recovery micro-optimization was implemented.
+
+Before:
+
+* multiplication shuffle tables could be broadcast repeatedly inside the
+  32/64-byte loop
+* delta low/high nibbles could be recomputed for different outputs
+
+After:
+
+AVX2 x86-64:
+
+* keep the 8 low/high shuffle tables for V0..V3 resident in ymm8..ymm15
+* keep the low-nibble mask resident
+* split d0/d1 once per chunk
+* reuse the split values for both recovered outputs
+
+AVX512BW:
+
+* keep the 8 low/high shuffle tables resident in zmm16..zmm23
+* keep the low-nibble mask resident
+* split d0/d1 once per chunk
+* reuse them for both outputs
+
+Important portability result:
+
+* The first AVX2 version used ymm8..ymm15 unconditionally.
+
+* That is invalid for 32-bit x86, where only ymm0..ymm7 are available.
+
+* The final implementation keeps the optimized resident-table path only under
+  CONFIG_X86_64 and preserves the old REC2 body for CONFIG_X86_32.
+
+* AVX512BW is already x86-64-only, so zmm16..zmm23 are safe there.
+
+This is the preferred class of future recovery optimization:
+same algorithm, same stride, same memory traffic, fewer repeated loads/broadcasts
+and better register reuse.
+
+
+Recovery optimization policy from now on
+-----------------------------------------
+
+For recovery, avoid new architecture-level changes unless a very strong case
+appears.
+
+Preferred optimizations are basic/local:
+
+* hoist loop-invariant values
+* preload coefficient tables into registers
+* keep masks/matrices resident when register pressure allows it
+* reuse low/high nibble splits
+* avoid repeated loads of the same hot temporary value
+* avoid repeated broadcasts
+* initialize accumulators from the first useful term when possible instead of
+  zeroing and XORing unnecessarily
+* exploit coefficient 0 or 1 only when it is simple and does not add harmful
+  branching/complexity
+
+Avoid for now:
+
+* raid_delta_gen() fusion
+* extra full passes over source blocks
+* aggressive spatial unrolling
+* software prefetch without clear evidence
+* changing MOVNT policy
+* major loop-orientation changes
+* large pre-expanded lookup tables
+
+
+REC1 status
+-----------
+
+* The important normal case is already raid_rec1of1() with P.
+
+* That path is essentially XOR-only and close to the minimum possible memory
+  traffic.
+
+* There is little algorithmic work left to remove.
+
+* More elaborate REC1 work is mainly relevant when P itself is unavailable and
+  Q/R/S/T/U must be used.
+
+* Those cases are much less common than the normal single-disk + P recovery.
+
+
+REC2 status
+-----------
+
+* Generic non-GFNI REC2 table/broadcast hoisting is implemented.
+
+* A specialized rec2of2 P+Q formula remains a TODO:
+
+      Dy = C0 * Pd ^ C1 * Qd
+      Dx = Pd ^ Dy
+
+  instead of the generic 2x2 solution with four GF multiplications.
+
+* This should be implemented, if desired, for AVX2, AVX512 and GFNI.
+
+* Practical priority is low because simultaneous loss of two data disks is an
+  exceptional case.
+
+
+RECX status
+-----------
+
+* RECX is the main remaining place to look for basic register-reuse
+  optimizations.
+
+* The interesting questions are local:
+
+      can pd[k] stay in a register longer?
+      can low/high nibble splits be reused across outputs?
+      can some multiplication tables remain resident?
+      can repeated broadcasts be removed?
+
+* AVX512 has enough registers to make some of these experiments plausible.
+
+* AVX2 is much more constrained, so any attempt must be careful not to trade
+  a few broadcasts for spills or a second source pass.
+
+* Source-major loop inversion is not currently considered a priority because
+  the pd working set is tiny/hot and the register-pressure tradeoff is not
+  obviously favorable.
+
+
+Generation status
+-----------------
+
+For current 1..6 parity generation, performance work is considered largely
+closed.
+
+Already covered:
+
+* scalar/int paths
+* SSE/SSSE3/AVX2/AVX512 paths
+* GFNI RAID affine generation
+* GFNI AES direct multiplication
+* RAID GFNI Horner Q
+* AVX512 width experiments
+* VPTERNLOGQ experiments
+* VPBLENDVB experiments
+* MOVNT policy
+* memory displacement/stride behavior
+* ISA-L comparisons
+* Linux RAID6 comparisons
+
+No major generation optimization remains that currently has a strong reason to
+be tested.
+
+
+More than 6 parity levels
+-------------------------
+
+* Supporting more than 6 parity levels remains a future feature.
+
+* A generic implementation using SSSE3/AVX intrinsics could be compared with
+  the current handwritten assembler style.
+
+* This is primarily a functional extension, not an unresolved performance issue
+  for the current 1..6 parity library.
+
+
+Current practical priority
+--------------------------
+
+1. Recovery only: continue looking for small invariant-hoisting/register-reuse
+   optimizations.
+
+2. RECX AVX512BW is the most natural place to inspect for repeated
+   load/split/broadcast work.
+
+3. Keep specialized rec2of2 P+Q as a correctness/performance TODO, but with low
+   real-world priority.
+
+4. Do not reopen generation-width, MOVNT, Horner AVX512BW, VPTERNLOGQ,
+   VPBLENDVB, large GFNI-table, source-reread, or raid_delta_gen-fusion ideas
+   unless new hardware or new measurements provide a concrete reason.
+
+
+Short list of ideas already tried or rejected
+---------------------------------------------
+
+* AVX512 128-byte processing:
+  tried, can be slower.
+
+* AVX512BW Horner:
+  tried on Zen 5, no improvement.
+
+* VPTERNLOGQ optimization:
+  tried on Zen 5, no improvement.
+
+* VPBLENDVB x2:
+  tried, no benefit / worse.
+
+* MOVNT -> cached stores:
+  not appropriate for the normal queued disk-write workload; keep MOVNT.
+
+* GFNI 32-byte pre-expanded affine constants:
+  rejected due to larger cache footprint.
+
+* split GFNI outputs into multiple passes:
+  rejected because 256 KiB source blocks would be reread.
+
+* RECX source-major rewrite:
+  low priority; hot tiny pd working set and register-pressure tradeoff.
+
+* raid_delta_gen() + REC fusion:
+  technically valid but intentionally deprioritized because the normal
+  rec1of1/P case bypasses raid_delta_gen() already.
+
+* software prefetch:
+  no strong reason to add it given the existing long streams and proven
+  sensitivity to displacement/stride tuning.
+
+* runtime backend microbenchmark selection:
+  possible, but low value relative to complexity.
+
+* Linux x2/x4 style wide spatial kernels:
+  not assumed useful; wider AVX512 already tested negatively.
+
+* ISA-L cached-store behavior:
+  not a good match for the real SnapRAID parity queue.
+
+* ISA-L source reread / split-output strategies:
+  poor fit for 256 KiB blocks.
+
+* specialized P+Q rec2of2:
+  still open, but low practical priority.
+
+
+End state
+---------
+
+For the current 1..6 parity RAID library:
+
+    Generation:       effectively closed
+    GFNI generation:  effectively closed
+    Store policy:     closed, keep MOVNT
+    Memory layout:    tuned; do not casually change
+    Recovery:         only basic/local micro-optimizations remain
+    rec2of2 P+Q:      open TODO, low real-world priority
+    >6 parity:        future feature
+
+The most important workload fact to remember is that the normal recovery is a
+single failed data disk with P available, which goes through raid_rec1of1() and
+is already a direct XOR-style recovery with minimal extra work.