diff --git a/raid/NOTES b/raid/NOTES new file mode 100644 index 00000000..c5fa2eae --- /dev/null +++ b/raid/NOTES @@ -0,0 +1,1076 @@ +SnapRAID RAID library performance notes +======================================= + +Scope +----- + +These notes summarize performance experiments, implementation choices, rejected +ideas, current recovery optimizations, and non-obvious conclusions reached while +reviewing and tuning the SnapRAID RAID library. + +The focus is the current 1..6 parity implementation, especially x86 AVX2, +AVX512BW, GFNI, SSSE3, and AArch64 NEON backends. + +This is intentionally a record of what has already been tried and what was +learned from it, so that the same ideas are not repeatedly re-investigated. + +The main rule is to optimize the real SnapRAID workload, not only isolated inner +loops. + + +Current implementation snapshot +------------------------------- + +Generation: + +* Current 1..6 parity generation is considered effectively closed from a + performance point of view. + +* Relevant scalar, SSE/SSSE3, AVX2, AVX512BW, NEON, RAID GFNI affine, and AES + GFNI paths are implemented. + +* RAID GFNI Horner Q is implemented where it is useful and where register + pressure allows it. + +* AVX512 128-byte processing, AVX512BW Horner, VPTERNLOGQ, VPBLENDVB, cached + parity stores, large pre-expanded GFNI constants, and other major alternatives + have already been investigated or rejected. + +Recovery: + +* The normal one-data-disk recovery with P available is already handled by + raid_rec1of1(), which is essentially an XOR generation pass with pointer + remapping. + +* Generic REC1 non-P SIMD kernels have been locally optimized and spatially + unrolled where useful. + +* Generic REC2 non-GFNI kernels hoist coefficient tables and reuse delta nibble + splits. + +* NEON REC2 processes two 16-byte blocks per iteration, for a 32-byte recovery + chunk. + +* RECX keeps delta values in SIMD registers on the optimized paths and reuses + pre-split low/high nibbles instead of repeatedly reconstructing them. + +* Non-GFNI RECX precomputes shuffle-table pointers in C before entering the hot + data loop. + +* RAID GFNI RECX precomputes the 8-byte affine representation of every inverse + matrix coefficient in C before entering the hot data loop. + +* The main remaining RECX local cost is repeated coefficient table/matrix + loading or broadcasting inside the data loop. + +* A specialized SIMD P+Q rec2of2 path remains open. The scalar int8 + implementation already has the specialized formula. + + +Real workload assumptions +------------------------- + +* SnapRAID processes large blocks: typically 256 KiB per data/parity block. + +* A normal stripe therefore touches a large amount of memory. With 12 data + disks, one 256 KiB stripe already means about 3 MiB of data input before + considering parity output. + +* This large-block behavior is very important. Optimizations that reread a + source buffer, increase the number of passes, or enlarge lookup tables can be + worse even if the inner instruction sequence looks better. + +* SnapRAID uses a queue before parity data is written to disk. Real measurements + with actual disk writes show that very small queues are worse. Performance + improves clearly up to about queue depth 32, and still improves after that, + but with diminishing returns. + +* With 256 KiB blocks, queue depth 32 means this much parity output can be in + flight: + + 1 parity = 8 MiB + 2 parity = 16 MiB + 3 parity = 24 MiB + 6 parity = 48 MiB + + Therefore a parity block produced by the RAID kernel is normally not consumed + immediately. + +* A memory read of a 256 KiB block that is no longer cached is still very cheap + compared with real storage I/O. Trying to keep many queued parity blocks hot + in cache is therefore not necessarily useful. + + +MOVNT / non-temporal stores +--------------------------- + +* Generation backends use MOVNTDQ / VMOVNTDQ for parity output where applicable. + +* The synthetic speedtest naturally favors non-temporal stores because generated + parity is not consumed afterwards. This benchmark therefore measures producer + throughput, not the complete producer -> queue -> writer pipeline. + +* However, the real SnapRAID queue makes the workload much closer to the + non-temporal case than to an immediate-consumer case. + +* Cached stores could potentially help only with very short queues, where the + writer consumes parity while it is still hot. Queue depths around 1..4 are the + most plausible area where this could happen. + +* Those very short queues are not representative of the real fastest disk + pipeline. Real measurements already show that queues up to about 32 provide + substantial improvements. + +* At realistic queue depths, cached parity output would occupy many MiB of cache + and could evict useful input data. MOVNT avoids this cache pollution and avoids + normal cache allocation/write-allocate behavior for parity output. + +* Conclusion: keep MOVNT for the normal generation path. Replacing MOVNT with + cached stores is not considered an open optimization. + + +Typical recovery case +--------------------- + +* The common recovery case is one failed data disk while parity P is still + available. + +* raid_rec() selects the first available parity. Therefore, for a single data + failure with P available, recovery uses P. + +* REC1 implementations special-case ip[0] == 0 and call raid_rec1of1(). + +* raid_rec1of1() remaps pointers so that P is used as an XOR input and the + failed data buffer is used as the GEN1 output, then calls raid_gen(nd, 1, ...). + +* Therefore the common single-disk recovery path does NOT call + raid_delta_gen(). + +* The data traffic is already close to the minimum: + + read P + read all surviving data blocks + XOR them + write the recovered data block once + +* There are no temporary syndrome buffers, no GF multiplication, and no matrix + inversion in this common path. + +* This means that large architectural recovery changes mostly optimize rare + failure cases, not the usual single-disk recovery. + +* Two simultaneous data disk failures are already an exceptional case in normal + use. Three or more simultaneous data failures are even less important as + optimization targets. + + +Recovery speedtest interpretation +--------------------------------- + +The recovery speed tests contain separate rows for the important fast paths and +for the generic recovery paths. + +REC1OF1: + +* The row uses ip[0] == 0. + +* Every REC1 wrapper therefore enters raid_rec1of1() instead of executing its + normal GF recovery body. + +* The speed test explicitly selects the generation function used by the + raid_rec1of1() call with raid_gen_force(1, ...). + +* There is no GFNI GEN1 because PAR1 is only XOR. Therefore: + + rec1of1 avx2 -> raid_gen1_avx2 + rec1of1 gfni -> raid_gen1_avx2 + + rec1of1 avx512 -> raid_gen1_avx512bw + rec1of1 gfni512 -> raid_gen1_avx512bw + + This is intentional. The GFNI recovery wrapper is selected, but the fast path + naturally reduces to the best available XOR generator. + +REC1: + +* The generic REC1 row uses ip + 1 to avoid the PAR1 fast path. + +* It therefore measures recovery from Q or a later parity and executes the + backend-specific REC1 GF multiplication kernel. + +REC2OF2: + +* The row uses ip = { 0, 1 }, meaning P and Q. + +* raid_rec2_int8() has an existing specialized P+Q fast path and calls + raid_rec2of2_int8(). + +* Current SIMD REC2 implementations do not have the equivalent specialized P+Q + fast path. They execute the generic 2x2 recovery algorithm with P and Q. + +* This makes the row useful as a direct benchmark for deciding whether a + specialized SIMD rec2of2 implementation is worthwhile. + +REC2: + +* The generic REC2 row uses ip + 1, avoiding the P+Q fast case. + +* It measures the normal generic 2x2 recovery path. + +Delta generation: + +* Each recovery benchmark explicitly forces a matching generation backend with + raid_gen_force() before entering the timed recovery function. + +* This is important because raid_delta_gen() internally calls raid_gen(). + +* The benchmark therefore avoids accidentally comparing a recovery kernel while + using an unrelated generation backend for its delta step. + + +raid_delta_gen() fusion +----------------------- + +* A fused recovery design was studied, where selected syndromes would be + accumulated directly instead of: + + raid_delta_gen() + temporary parity output + reload temporary parity + solve recovery matrix + +* The fused design could also compute only the parity rows actually selected by + ip[]. + +* This matters because raid_delta_gen() currently computes: + + np = ip[nr - 1] + 1 + + and raid_gen() can generate only a prefix of parity rows. + +* Example: + + nr = 3 + ip = { 0, 2, 5 } + + requires GEN6 today even though only P, R and U are needed. + +* A fused RECX could calculate exactly those three syndromes. + +* The idea is technically valid and could reduce temporary memory traffic and + unnecessary parity work. + +* However, the common rec1of1/P case already bypasses raid_delta_gen() entirely. + +* Because the main benefit applies to less common multiple-failure or missing-P + cases, this direction was intentionally deprioritized. + +* Decision: do not pursue raid_delta_gen() + REC fusion for now. + + +Generation: AVX512 width and unrolling +-------------------------------------- + +* Wider processing is not automatically faster. + +* AVX512 128-byte processing was tested and can be slower than the existing + 64-byte path. + +* Therefore do not assume that x2/x4 spatial unrolling, like some Linux RAID6 + kernels use, is beneficial for SnapRAID. + +* The block size is already large, memory traffic is substantial, and wider + unrolling can increase register pressure, front-end work, and AVX512 power or + frequency effects without reducing total memory traffic. + +* VPTERNLOGQ based alternatives were also tested on Zen 5 and did not improve + performance. + +* Conclusion: keep the current conservative AVX512 generation width unless a + specific benchmark proves otherwise. + + +Generation: Horner +------------------ + +* Linux RAID6 uses the classic Horner recurrence for Q with the standard RAID + polynomial/generator. + +* A Horner-style AVX512BW non-GFNI implementation was tested on Zen 5 and did + not improve performance. + +* Therefore Horner is NOT considered an open optimization for the AVX512BW + non-GFNI generation path. + +* GFNI is different. + +* For RAID GFNI generation, Horner Q is useful because multiplying by the fixed + generator can use one fixed affine matrix instead of a different matrix for + every data coefficient. + +* RAID GFNI Horner Q is implemented as follows: + + AVX2 RAID GFNI GEN2..GEN5: + Horner Q with fixed multiply-by-2 affine matrix resident in a register. + + AVX2 RAID GFNI GEN6: + keep the direct-coefficient implementation because register pressure + is too high to keep the extra fixed matrix efficiently. + + AVX512 RAID GFNI GEN2..GEN6: + Horner Q with the fixed matrix resident in a register. + + AES GFNI: + unchanged, direct VGF2P8MULB coefficient multiplication. + +* D0/final coefficient 1 is handled directly where appropriate. + +* Conclusion: GFNI Horner Q is implemented where useful. Non-GFNI AVX512BW + Horner was tested and rejected. + + +GFNI field implementation +------------------------- + +* AES GFNI multiplication maps naturally to VGF2P8MULB because the instruction + uses the AES polynomial. + +* Standard RAID polynomial 0x11d cannot use VGF2P8MULB directly. + +* RAID GFNI therefore uses VGF2P8AFFINEQB with precomputed affine matrices. + +* The global affine matrix table stores 8 bytes per coefficient. + +* A generic exhaustive software test validates all coefficient/input pairs + against the normal RAID GF multiplication table. + +* Dedicated hardware affine-matrix validators were intentionally not added. + Normal generation/recovery tests exercise the hardware path. + +* RAID and AES GFNI recovery/generation have polynomial-specific names: + + *_gfni_raid + *_gfni_aes + + with backend tags: + + gfni + gfni512 + +* This naming and registration split is intentional and should be preserved. + + +GFNI RECX coefficient precomputation +------------------------------------ + +RAID GFNI RECX previously performed this dependency chain for every coefficient +inside every data chunk: + + load coefficient byte from V[] + use it to index raid_gfaffine_raid[] + load the 8-byte affine matrix + broadcast the matrix + VGF2P8AFFINEQB + +The current implementation precomputes: + + uint8_t C[N * N][8] + +and copies the complete 8-byte affine representation for every inverse matrix +coefficient before entering the data loop. + +The hot loop therefore becomes: + + direct load/broadcast from C[] + VGF2P8AFFINEQB + +This removes the coefficient-byte -> indexed affine-table dependency from the +hot loop while keeping the compact 8-byte representation. + +Important details: + +* All 8 bytes must be copied. An affine matrix coefficient is not one byte. + +* Using a byte array and memcpy() keeps the representation explicit and avoids + strict-aliasing assumptions. + +* AVX2 and AVX512 RAID GFNI RECX both use this precomputation. + +* AES GFNI does not need this conversion because VGF2P8MULB consumes the + coefficient byte directly. + +* The remaining repeated operation is the per-chunk broadcast of the 8-byte + affine matrix. + + +GFNI pre-expanded affine tables +------------------------------- + +* Intel ISA-L can use pre-expanded GFNI constants, for example a 32-byte + representation that is ready for vector use. + +* For SnapRAID this would increase the RAID affine table from about 2 KiB to + about 8 KiB. + +* With large 256 KiB processing blocks, the current small 8-byte entries plus + broadcast have a better cache-footprint tradeoff. + +* No convincing reason was found to trade four times the table footprint for + fewer broadcasts. + +* Decision: keep compact 8-byte affine matrices. + + +Intel ISA-L comparison +---------------------- + +The following ISA-L ideas were examined. + +1. Horner Q for GFNI RAID generation + + Useful idea. Implemented for RAID GFNI where register pressure allows it. + +2. AVX512 GFNI 128-byte processing + + Tested. It can be slower. Do not use wider processing simply because more + ZMM registers are available. + +3. Source-major generic recovery loops + + Initially interesting because one source could feed several outputs. + + For the current RECX structure, the delta/syndrome working set is tiny and + hot. Reordering loops does not obviously save important memory traffic and + can increase register pressure. + + This is considered low priority / not useful enough. + +4. Split output processing, for example 3+3 rows + + Bad fit for SnapRAID 256 KiB blocks because it requires rereading the source + range in another pass. + + Avoid optimizations that save registers by adding full source passes. + +5. Pre-expanded 32-byte GFNI constants + + Rejected because of the larger table/cache footprint described above. + +6. VPBLENDVB based x2 techniques + + Tested. No benefit / worse performance. + +7. Cached parity stores + + ISA-L style normal stores are not automatically suitable for SnapRAID. + SnapRAID has large output blocks and a real queued disk-write pipeline. + MOVNT remains the better design choice for this workload. + +8. Multiple-row GFNI macros + + Interesting in isolation, but not enough reason to reread 256 KiB source + blocks or enlarge table footprint. + +9. Broad runtime dispatch/calibration + + ISA-L has broad dispatch machinery. SnapRAID already dispatches based on CPU + features. A runtime microbenchmark selector is possible, but is not currently + considered worth the extra complexity for the known kernels. + +10. Incremental parity APIs + + Useful in other applications, but not a missing optimization in the current + full-block SnapRAID parity-generation hot path. + +Overall ISA-L conclusion: + +* The important transferable idea was GFNI Horner Q, and that is already + implemented. + +* The other ISA-L techniques examined either lose on the current benchmark, + increase source rereads, increase cache footprint, or do not match the real + SnapRAID workload. + + +Linux kernel RAID6 comparison +----------------------------- + +The Linux RAID6 implementation was studied as another optimization reference. + +Relevant Linux characteristics include: + +* x1/x2/x4 spatial kernels +* wider AVX512 processing +* explicit prefetch +* independent chains for ILP +* classic Horner Q +* runtime generator selection +* xor_syndrome/incremental interfaces +* specialized P+Q two-disk recovery + +Conclusions for SnapRAID: + +* Wider x2/x4 AVX512 processing is not automatically useful. 128-byte variants + were tested and can be slower. + +* Horner AVX512BW non-GFNI was tested on Zen 5 and did not improve. + +* VPTERNLOGQ alternatives were tested on Zen 5 and did not improve. + +* Explicit software prefetch is not currently considered a strong candidate. + The workload already consists of long regular streams, and the existing + memory-layout tuning is important. + +* Runtime microbenchmark-based backend selection would add complexity and is low + priority. + +* xor_syndrome/incremental parity is a different API/use case and does not + directly improve the current full-block generation path. + +* The main Linux-style algorithmic recovery optimization still worth keeping as + a TODO is specialized P+Q two-data recovery for SIMD backends. + + +Memory layout, displacement and stride noise +-------------------------------------------- + +* The memory layout is not a neutral detail. + +* Fixed displacement between buffers was observed to help. + +* STRIDE_NOISE behavior differs by CPU. + +* On Intel, stride noise had little effect in the observed tests. + +* On Zen 5, disabling stride noise caused a large throughput reduction. + +* This indicates that hardware prefetch/cache behavior is already strongly + affected by the chosen buffer layout. + +* Therefore new software-prefetch or traversal changes should not be assumed to + help. They must be benchmarked against the existing displacement/stride + behavior. + +* This is another reason to avoid adding extra passes over 256 KiB blocks. + + +REC1 optimizations +------------------ + +The important normal REC1 case is already raid_rec1of1() with P and is close to +the minimum possible memory traffic. + +For generic REC1, used when P is unavailable: + +* AVX2 processes 64 bytes per iteration using two independent 32-byte chains. + +* SSSE3 x86-64 processes 64 bytes per iteration using four independent 16-byte + chains. The x86-32 path keeps the smaller register-compatible fallback. + +* NEON processes 64 bytes per iteration using four independent 16-byte chains. + +The arithmetic work per byte is mostly unchanged. The purpose of the unrolling +is: + +* reduce loop-control overhead +* expose more independent work +* improve instruction scheduling around shuffle/table operations + +This is a local optimization, not an architectural change. + +REC1 status: + +* rec1of1/P is essentially closed. + +* Generic non-P REC1 may still receive small local tuning if measurements show a + clear benefit. + +* No major REC1 algorithmic optimization is currently expected. + + +REC2 optimizations +------------------ + +Generic non-GFNI REC2 table/broadcast hoisting is implemented. + +AVX2 x86-64: + +* keep the 8 low/high shuffle tables for V0..V3 resident in ymm8..ymm15 +* keep the low-nibble mask resident +* split d0/d1 once per chunk +* reuse the split values for both recovered outputs + +AVX512BW: + +* keep the 8 low/high shuffle tables resident in zmm16..zmm23 +* keep the low-nibble mask resident +* split d0/d1 once per chunk +* reuse the split values for both recovered outputs + +SSSE3 x86-64: + +* keep all four coefficient low/high tables in XMM registers +* split both deltas once +* reuse the split values for both outputs + +NEON: + +* keep the four low/high coefficient tables in v20..v27 +* keep the low-nibble mask in v28 +* process two independent 16-byte blocks per iteration +* split both deltas once per block +* reuse the split values for both outputs +* the 32-byte unrolled kernel uses 25 vector registers + +Important portability result: + +* x86-32 has only the low eight XMM/YMM registers available to these old ISA + paths. + +* Optimizations that depend on xmm8..xmm15 or ymm8..ymm15 must remain restricted + to CONFIG_X86_64. + +REC2 status: + +* Generic REC2 local table/split reuse is implemented. + +* Scalar int8 already has specialized rec2of2 P+Q recovery. + +* Specialized SIMD rec2of2 P+Q remains open. + + +Specialized rec2of2 P+Q +----------------------- + +For failures of two data disks recovered specifically with P and Q, the system +can be reduced to: + + Dy = C0 * Pd ^ C1 * Qd + Dx = Pd ^ Dy + +This needs only two GF multiplications for Dy, followed by one XOR for Dx, +instead of the generic 2x2 reconstruction with four GF multiplications. + +The scalar int8 implementation already uses this specialization. + +Current SIMD REC2 backends still use the generic 2x2 matrix solution even when +ip = { 0, 1 }. + +Potential targets include: + +* SSSE3 +* AVX2 +* AVX512BW +* AVX2 GFNI +* AVX512 GFNI +* NEON + +The new rec2of2 benchmark row exists specifically to measure whether these +specializations are worth adding. + +Practical priority remains low because simultaneous loss of two data disks is an +exceptional real-world case. + + +RECX delta handling +------------------- + +RECX previously had avoidable temporary-buffer and repeated split work. + +The current optimized structure keeps the delta values for the current data +chunk in SIMD registers. + +For non-GFNI paths: + +* each delta is split into low/high nibbles once +* the split values remain in SIMD registers while all recovered outputs are + calculated +* the same split values are reused across output rows +* stack temporary delta buffers are avoided on the optimized paths + +For GFNI paths: + +* the byte deltas remain directly in SIMD registers +* no nibble split is needed + +This closes the old questions: + + can pd[k] stay in a register longer? + can low/high nibble splits be reused across outputs? + +For the current optimized paths, the answer is yes and this work is implemented. + + +RECX shuffle-table pointer precomputation +---------------------------------------- + +Non-GFNI RECX previously performed coefficient address calculation inside the +hot loop: + + load coefficient byte from V[] + multiply the coefficient index by 32 + add the raid_gfmulpshufb base address + load low/high shuffle tables + +The current implementation precomputes in C: + + T[j] = &raid_gfmulpshufb[V[j]][0][0]; + +The hot assembly then only performs: + + load T[j] + load or broadcast low table at T[j] + 0 + load or broadcast high table at T[j] + 16 + +This has two advantages: + +1. It removes coefficient-index address arithmetic from the hot assembly. + +2. It keeps C object layout and address calculation in C, leaving the assembly + focused on SIMD operations. + +Current users include: + +* AVX2 RECX x86-64 +* AVX512BW RECX +* SSSE3 RECX x86-64 +* NEON RECX + +For x86-32 AVX2/SSSE3 fallbacks, T[] is not unnecessarily built when the old +fallback body does not use it. + +The remaining cost is that the low/high tables are still loaded or broadcast for +every coefficient of every data chunk. + + +RECX current register state +--------------------------- + +The current RECX structure already uses registers aggressively. + +AVX512BW: + +* up to six delta values are kept as low/high pairs in zmm0..zmm11 +* zmm12..zmm15 are reconstruction temporaries/accumulators +* zmm31 keeps the low-nibble mask +* coefficient shuffle tables are still broadcast from T[] during reconstruction + +AVX512 RAID GFNI: + +* up to six byte deltas are kept in zmm0..zmm5 +* temporary/accumulator registers are used for reconstruction +* coefficient affine matrices come directly from precomputed C[] +* one 8-byte affine matrix is still broadcast per coefficient per chunk + +AVX2 RAID GFNI: + +* the two 32-byte halves of up to six delta blocks occupy most of the available + YMM register file +* C[] removes indexed affine-table lookup +* register pressure leaves less room for keeping many coefficient matrices + resident + +AVX2/SSSE3 non-GFNI: + +* the optimized x86-64 paths keep pre-split deltas in vector registers +* register pressure is significantly higher than AVX512 +* coefficient table residency is therefore much more constrained + +NEON: + +* pre-split deltas are kept in vector registers +* T[] provides direct pointers to low/high multiplication tables +* reconstruction still loads each coefficient table pair as needed + + +RECX remaining local optimization +--------------------------------- + +The main RECX question that remains genuinely open is: + + can repeated coefficient loads/broadcasts be reduced by keeping useful + coefficient tables or affine matrices resident in spare vector registers? + +This is most plausible on AVX512 because of the larger register file. + +Possible experiments: + +* keep one output row of coefficients resident across data chunks + +* keep a useful prefix of coefficient tables resident when N permits it + +* on AVX512 RAID GFNI, keep several 8-byte affine matrices broadcast in ZMM + registers instead of broadcasting them again for every chunk + +* on AVX512BW, keep some low/high shuffle-table pairs resident + +Constraints: + +* Do not add a second pass over the 256 KiB block. + +* Do not spill deltas or accumulators merely to save coefficient broadcasts. + +* Do not add large N-specific kernel families unless measurements justify the + complexity. + +* AVX2 and SSSE3 register pressure is already high, so the likely opportunity is + much smaller there. + +* NEON has more architectural registers than old x86 SIMD but current RECX + already uses a significant part of them; any residency experiment should be + measured carefully. + +This is now a narrower problem than the old RECX TODO list. Delta residency, +nibble reuse, coefficient address precomputation, and GFNI affine lookup +precomputation are already implemented. + + +Recovery optimization policy from now on +----------------------------------------- + +For recovery, avoid new architecture-level changes unless a very strong case +appears. + +Preferred optimizations are basic/local: + +* hoist loop-invariant values +* keep masks/tables/matrices resident when register pressure allows it +* avoid repeated loads or broadcasts +* initialize accumulators from the first useful term +* keep address calculations in C when doing so simplifies inline assembly +* exploit coefficient 0 or 1 only when it stays simple and does not add harmful + branching or complexity +* use limited spatial unrolling only when it exposes ILP without excessive + register pressure + +Avoid for now: + +* raid_delta_gen() fusion +* extra full passes over source blocks +* aggressive spatial unrolling +* software prefetch without clear evidence +* changing MOVNT policy +* major loop-orientation changes +* large pre-expanded lookup tables +* broad runtime calibration machinery + + +Inline assembly and portability constraints +------------------------------------------- + +The SIMD code intentionally uses multiple small or large asm volatile blocks and +keeps SIMD state live across them. + +Important project constraints: + +* Assembly blocks that dereference pointers internally must model memory effects + appropriately. RECX pointer-based blocks use a "memory" clobber. + +* The x86 SSE/AVX end helpers declare SIMD clobbers needed by the project ABI + strategy, including the Windows x64 callee-saved XMM registers. + +* The NEON end helper declares v0..v31 clobbered so that GCC/Clang preserve + AAPCS64 callee-saved v8..v15 when required. + +* The end-of-function dummy-clobber mechanism is a documented project technique. + It is not a fully explicit extended-asm dataflow model, so future compiler + portability changes should be reviewed carefully. + +* Do not assume x86-32 has xmm8..xmm15 or ymm8..ymm15. + +* RAID block sizes passed to the SIMD code are multiples of 64, so current + 32-byte and 64-byte unrolled loops do not need tail handling. + +These are established implementation constraints, not current correctness +findings. + + +Generation status +----------------- + +For current 1..6 parity generation, performance work is considered largely +closed. + +Already covered: + +* scalar/int paths +* SSE/SSSE3/AVX2/AVX512 paths +* NEON paths +* GFNI RAID affine generation +* GFNI AES direct multiplication +* RAID GFNI Horner Q +* AVX512 width experiments +* VPTERNLOGQ experiments +* VPBLENDVB experiments +* MOVNT policy +* memory displacement/stride behavior +* ISA-L comparisons +* Linux RAID6 comparisons + +No major generation optimization remains that currently has a strong reason to +be tested. + + +More than 6 parity levels +------------------------- + +* Supporting more than 6 parity levels remains a future feature. + +* A generic implementation using SSSE3/AVX intrinsics could be compared with the + current handwritten assembler style. + +* This is primarily a functional extension, not an unresolved performance issue + for the current 1..6 parity library. + + +Current practical priority +-------------------------- + +1. Use the new rec2of2 benchmark to decide whether specialized SIMD P+Q recovery + is worth implementing. + +2. Recovery only: inspect RECX for coefficient table/matrix residency, especially + AVX512BW and AVX512 GFNI. + +3. Prefer small local improvements that reduce broadcasts or repeated loads + without changing memory traversal. + +4. Treat REC1 and generic REC2 as largely locally optimized unless benchmarks + expose a clear new bottleneck. + +5. Do not reopen generation width, MOVNT, non-GFNI AVX512BW Horner, + VPTERNLOGQ, VPBLENDVB, large GFNI tables, source rereads, or + raid_delta_gen() fusion unless new hardware or new measurements provide a + concrete reason. + + +Short list of ideas already implemented +--------------------------------------- + +* Common one-disk + P recovery through raid_rec1of1(). + +* Generic REC1 64-byte local unrolling for AVX2, x86-64 SSSE3, and NEON. + +* x86-64 non-GFNI REC2 coefficient-table residency. + +* AVX512BW REC2 coefficient-table residency. + +* NEON REC2 32-byte unrolling with two independent blocks. + +* RECX delta values kept in vector registers on optimized paths. + +* RECX low/high nibble splitting performed once and reused across outputs. + +* RECX stack temporary delta buffers removed on optimized paths. + +* Non-GFNI RECX shuffle-table addresses precomputed as T[] pointers in C. + +* RAID GFNI RECX affine coefficient representations precomputed as C[][8]. + +* RECX accumulators initialized from useful products where the current kernels + allow it. + +* Recovery speedtest rows for rec1of1 and rec2of2. + + +Short list of ideas already tried or rejected +--------------------------------------------- + +* AVX512 128-byte generation processing: + tried, can be slower. + +* AVX512BW Horner: + tried on Zen 5, no improvement. + +* VPTERNLOGQ optimization: + tried on Zen 5, no improvement. + +* VPBLENDVB x2: + tried, no benefit / worse. + +* MOVNT -> cached stores: + not appropriate for the normal queued disk-write workload; keep MOVNT. + +* GFNI 32-byte pre-expanded affine constants: + rejected due to larger cache footprint. + +* Split GFNI outputs into multiple passes: + rejected because 256 KiB source blocks would be reread. + +* RECX source-major rewrite: + low priority; hot tiny delta working set and register-pressure tradeoff. + +* raid_delta_gen() + REC fusion: + technically valid but intentionally deprioritized because the normal + rec1of1/P case bypasses raid_delta_gen() already. + +* Software prefetch: + no strong reason to add it given the existing long streams and proven + sensitivity to displacement/stride tuning. + +* Runtime backend microbenchmark selection: + possible, but low value relative to complexity. + +* Linux x2/x4 style wide generation kernels: + not assumed useful; wider AVX512 has already tested negatively. + +* ISA-L cached-store behavior: + not a good match for the real SnapRAID parity queue. + +* ISA-L source reread / split-output strategies: + poor fit for 256 KiB blocks. + + +Open items +---------- + +For the current 1..6 parity implementation, the open list is intentionally +short. + +1. Specialized SIMD rec2of2 P+Q recovery + + The scalar int8 specialization exists. + + Use the rec2of2 speedtest row to quantify whether SIMD specializations are + worthwhile. + +2. RECX coefficient residency + + Investigate whether AVX512BW and AVX512 GFNI can keep useful coefficient + tables/matrices resident across chunks and remove repeated broadcasts. + + AVX2, SSSE3, and NEON may have smaller opportunities, but register pressure + must be respected. + +3. Minor recovery micro-optimizations + + Only pursue small invariant-hoisting, load/broadcast reduction, or scheduling + improvements supported by benchmarks. + +4. More than 6 parity levels + + Future functional feature, not a current 1..6 performance problem. + + +End state +--------- + +For the current 1..6 parity RAID library: + + Generation: effectively closed + GFNI generation: effectively closed + Store policy: closed, keep MOVNT + Memory layout: tuned; do not casually change + REC1 common P path: effectively closed + REC1 generic: local tuning only + REC2 generic: substantially locally optimized + rec2of2 SIMD P+Q: open TODO, benchmark now available + RECX delta reuse: implemented + RECX address setup: implemented with T[] / C[] + RECX broadcasts: main remaining local optimization area + >6 parity: future feature + +The most important workload fact to remember is that normal recovery is a +single failed data disk with P available. That path goes through +raid_rec1of1() and is already a direct XOR-style recovery with minimal extra +work.