RAID Library ============ This directory contains the Galois Field GF(2^8) erasure coding engine used by SnapRAID. It provides multi-parity generation, missing data and parity block recovery, error candidate validation, and brute-force block failure scanning. The library supports up to 251 data blocks and up to 6 parity blocks using an Extended Cauchy Matrix construction that guarantees MDS (Maximum Distance Separable) properties for all supported combinations of disk failures. Public API ---------- Header: #include "raid.h" Before invoking any generation or recovery functions, call raid_init() once to detect CPU capabilities and initialize mode dispatch. void raid_init(void) Initializes algorithm tables and detects hardware SIMD instruction sets (SSE2, SSSE3, AVX2, AVX-512, GFNI, NEON). Sets default operating mode to RAID_MODE_CAUCHY_RAID. Must be called before any other API function. int raid_mode(int mode) Sets or queries the active operating mode. - Pass RAID_MODE_GET (-1) to query current active mode. - Pass RAID_MODE_CAUCHY_RAID, RAID_MODE_VANDERMONDE_RAID, or RAID_MODE_CAUCHY_AES to set mode. Returns the mode active prior to this call. void raid_zero(void *zero) Registers a read-only buffer filled with zeros of block size 'size'. Required prior to calling raid_rec() or raid_data() for delta parity computation. void raid_gen(int nd, int np, size_t size, void **v) Computes 'np' parity blocks from 'nd' data blocks. Vector 'v' contains 'nd + np' block pointers (data blocks v[0..nd-1], parity blocks v[nd..nd+np-1]). Data blocks are read-only; parity blocks are written. void raid_rec(int nr, int *ir, int nd, int np, size_t size, void **v) Recovers 'nr' failed blocks (data or parity) listed in sorted index array 'ir'. Indexes start at 0 (data 0..nd-1, parity nd..nd+np-1). Pointers in 'v' for good blocks are read; pointers for bad blocks are target destinations. Requires nr <= np. void raid_data(int nr, int *id, int *ip, int nd, size_t size, void **v) Recovers 'nr' failed data blocks listed in sorted array 'id', using parities listed in sorted array 'ip'. Parity blocks in 'v' are unmodified. Requires nr <= nd and nr <= RAID_PARITY_MAX. int raid_check(int nr, int *ir, int nd, int np, size_t size, void **v) Validates whether candidate failed block set 'ir' matches available parity redundancy. Returns 0 if candidate set matches, or -1 if invalid. Requires nr < np. Does not modify data or parity blocks. int raid_scan(int *ir, int nd, int np, size_t size, void **v) Searches for up to 'np - 1' failed blocks using available redundancy. Fills 'ir' with failed block indexes and returns the number of failures found (0 if no error), or -1 if failed blocks cannot be uniquely located. int raid_selftest(void) Runs startup verification tests for parity generation, recovery, and scanning across parity levels 1..6. Returns 0 on success, < 0 on failure. Memory and Alignment Requirements --------------------------------- - Vector Layout: In vector 'v', data blocks MUST precede parity blocks: v[0 ... nd-1] -> Data blocks v[nd ... nd+np-1] -> Parity blocks (PAR1, PAR2, ..., PAR6) - Block Size: Parameter 'size' MUST be a multiple of 64 bytes (size % 64 == 0). - Memory Alignment: Blocks should ideally be aligned on 16/32/64-byte boundaries to optimize SIMD vector loads and stores. - Sorted Index Arrays: Index arrays ('ir', 'id', 'ip') MUST be strictly sorted in ascending order (e.g. ir[0] < ir[1] < ... < ir[nr-1]). Operating Modes and Polynomials ------------------------------- RAID_MODE_CAUCHY_RAID (Default mode = 0) Uses Cauchy matrix generator with standard RAID polynomial 0x1d (x^8 + x^4 + x^3 + x^2 + 1, decimal 285). Supports parities 1 to 6. Matches standard Linux RAID-6 (Anvin method) for 2-parity arrays and ensures on-disk compatibility. RAID_MODE_VANDERMONDE_RAID (Mode = 1) Legacy mode supporting up to 3 parities using standard polynomial 0x1d. Uses Cauchy for PAR1/PAR2 and Vandermonde matrix coefficients for PAR3 (Z parity). Optimized for low-end CPUs without vector instructions. RAID_MODE_CAUCHY_AES (Mode = 2) Uses Cauchy matrix with AES polynomial 0x1b (x^8 + x^4 + x^3 + x + 1). Enables hardware acceleration via Intel GFNI instructions (vgf2p8mulb) on supported CPUs (Ice Lake or newer). Parity data is incompatible with standard 0x1d polynomial arrays. Implementation Structure ------------------------ raid.h Public API declarations, macros, and mode constants. raid.c Cauchy matrix solver, Gauss-Jordan inversion, delta generation. module.c Algorithm registration, CPU detection, and dynamic dispatch. int.c Portable C fallback implementation (8/32/64-bit word loops). tables.c Precomputed GF(2^8) log/exp, multiplication, Cauchy tables. mktables.c Table generator script for GF(2^8) field matrices. internal.h Private header, assertion macros (BUG_ON), alignment helpers. Platform-Specific Acceleration: sse2.c x86 SSE2 (128-bit SIMD vectorization) ssse3.c x86 SSSE3 (128-bit pshufb nibble-split multiplication) avx2.c x86 AVX2 (256-bit vectorization) avx512.c x86 AVX-512BW (512-bit vectorization) gfni.c x86 GFNI hardware Galois field vector multiplication neon.c ARM NEON (128-bit SIMD vectorization) Testing and Benchmarks ---------------------- Standalone test applications located in raid/test/ provide verification and performance evaluation: raid/test/selftest Quick startup self-test verifying basic correctness. raid/test/fulltest Exhaustive test suite testing all combinations of data disks (1..251) and single/multi-disk failure recovery. raid/test/invtest Matrix inversion test verifying non-singularity for all square submatrices of the Cauchy matrix. raid/test/speedtest Benchmark measuring single-thread throughput (MiB/s) for parity generation and recovery across all SIMD implementations.