rt/snapraid
1
0
Fork 0
mirror of https://github.com/amadvance/snapraid.git synced 2026-10-26 10:51:54 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

Documentation and comment changes.

Browse source
This commit is contained in:
Andrea Mazzoleni 2013-11-02 15:55:57 +01:00
parent 877d781204
commit d3086339aa
10 changed files with 106 additions and 107 deletions
Download patch file Download diff file Expand all files Collapse all files

7
HISTORY
View file

@ -3,10 +3,9 @@ SnapRAID HISTORY
5.0 2013/11 (BETA)
==================
* Added support for RAIDTP (Triple Parity), RAIDQP (Quad Parity),
RAIDPP (Penta Parity) and RAIDHP (Hexa Parity).
Take care that to get good performance for RAIDTP and beyond you
need a CPU with SSSE3 support.
* Added support for up to six level of parity. Take care that to get
good performance for triple parity and beyond you need a CPU with
SSSE3 support.
* Faster RAID5 and RAID6 computation for ARM 64 bit CPUs.
* In sync, if a silent error is found, directly mark the block as bad
like in scrub, and continue with the sync process.

5
mktables.c
View file

@ -60,7 +60,7 @@ unsigned char gfinv[256];
#define DISK (257-PARITY)
/**
* Setup the Normalized Extended Cauchy matrix used to generate the parity.
* Setup the Cauchy matrix used to generate the parity.
*/
void set_cauchy(unsigned char* matrix)
{
@ -108,7 +108,6 @@ void set_cauchy(unsigned char* matrix)
*/
y = 2;
for(j=0;j<PARITY-2;++j) {
inv_x = 1;
for(i=0;i<DISK;++i) {
unsigned char x = gfinv[inv_x];
@ -120,7 +119,7 @@ void set_cauchy(unsigned char* matrix)
}
/*
* Normalizes the matrix multipling each row for
* Adjusts the matrix multipling each row for
* the inverse of the first element in the row.
*
* This operation doesn't invalidate the property that all the square

8
mktest.c
View file

@ -149,7 +149,13 @@ void generate(int disk, int size)
count = size;
while (count) {
fputc(rnd(256), f);
/* We don't write zero bytes because we want to test */
/* the recovering of new files, after an aborted sync */
/* If the files contains full blocks at zero */
/* this is an impossible condition to recover */
/* because we cannot differentiate between an unused block */
/* and a file filled with 0 */
fputc(rndnotzero(256), f);
--count;
}

76
raid.c
View file

@ -16,18 +16,24 @@
*/
/*
* The RAID5 and RAID6 support were originally derived from the libraid6
* library by H. Peter Anvin released with license "GPL2 or any later
* version", used in the Linux Kernel 2.6.38.
* This support was later completely rewritten (many times), but the
* H. Peter Anvin's Copyright may still apply.
* Copyright notes:
*
* The RAID5 and RAID6 support was originally derived from the
* H. Peter Anvin paper "The mathematics of RAID-6" [1] and the
* libraid6 library, also by H. Peter Anvin, released with license
* "GPLv2 or any later version" inside the Linux Kernel 2.6.38.
*
* This support was later completely rewritten (many times), but
* the H. Peter Anvin's Copyright may still apply.
*
* The RAIDTP (Triple Parity), RAIDQP (Quad parity), RAIDPP (Penta Parity)
* and RAIDHP (Hexa parity) support is original work implemented from
* scratch.
*
* For RAID5 and RAID6 it works like the Linux Kernel RAID and it's based
* on the Anvin's paper "The mathematics of RAID-6" [1].
* and RAIDHP (Hexa parity) support, and the recovering based on matrix
* inversion is original work implemented from scratch.
*/
/*
* The RAID5 and RAID6 support works like the Linux Kernel RAID
* following the Anvin's paper "The mathematics of RAID-6" [1].
*
* We compute the parity in the Galois Field GF(2^8) with the primitive
* polynomial x^8 + x^4 + x^3 + x^2 + 1 (285 decimal), starting from a set
@ -55,36 +61,26 @@
* R = sum(4^i * Di)
* S = sum(8^i * Di) with 0<=i<N
*
* we don't have a system of independent linear equations, and in some
* cases the equations are not solvable.
* we don't have a system of always solvable equations.
*
* This approach is expected to fail at some point, because the Vandermonde
* matrix used to compute the parity has no guarantee to have all
* submatrices not singular [2, Chap 11, Problem 7] and this is a requirement
* to have a MDS (Maximum Distance Separable) code [2, Chap 11, Theorem 8].
*
* If it surprises that even using a Vandermonde matrix we don't have a
* MDS code, consider that the matrix A we use to compute the parity, is
* not the MDS generator matrix G, but only a submatrix of it.
* The generator matrix G is the concatenation of I and A, as G = [I | A],
* where I is the identity matrix.
* Setting the matrix G as Vandermonde matrix would have guaranteed to
* have a MDS code, but setting only A as a Vandermonde matrix doesn't
* give this guarantee.
* This approach was expected to fail at some point, because the Vandermonde
* matrix of coefficents used to compute the parity has no guarantee to have
* all submatrices not singular [2, Chap 11, Problem 7] and this is a
* requirement to have a MDS (Maximum Distance Separable) code [2, Chap 11,
* Theorem 8].
*
* Using the primitive polynomial 285, RAIDQP works for up to 21 data disks
* with parity coefficients "1,2,4,8". Changing polynomial to one of
* 391/451/463/487,
* it works for up to 27 disks with the same parity coefficients.
* 391/451/463/487, it works for up to 27 disks with the same parity
* coefficients.
* Using different parity coefficients, like "5,13,27,35", it's possible
* to make it working for up to 33 disks. But no more.
*
* To support more disks it's possible to use GF(2^16) with primitive
* polynomial 100087 or 122563 that supports up to Hexa (6) Parity
* with parity coefficients 1,2,4,8,16,32 for up to 89 disks.
* with parity coefficients 1,2,4,8,16,32, but only for up to 89 disks.
*
* To overcome these limitations we instead use an Extended Cauchy Matrix
* [3][4] to compute the parity.
* To overcome these limitations we instead use a Cauchy Matrix [3][4]
* to compute the parity.
* Such matrix has the mathematical property to have all the square
* submatrices not singular, resulting in always solvable equations, for
* any number of parities and for any number of disks.
@ -97,16 +93,16 @@
* using SSSE3 instructions [1][5]. Method already competitive with the
* computation of RAIDTP parity using power coefficients.
*
* Another important property of the Extended Cauchy matrix is that we can
* setup the first two rows with power coeffients equal at the RAID5 and
* RAID6 approach described in Anvin's paper [1], resulting in a compatible
* extension, and requiring SSSE3 instructions only if RAIDTP is used.
* Another important property of the Cauchy matrix is that we can setup
* the first two rows with coeffients equal at the RAID5 and RAID6 approach
* described in Anvin's paper [1], resulting in a compatible extension,
* and requiring SSSE3 instructions only if RAIDTP is used.
*
* We also "normalize" the matrix, multipling each row for a constant
* factor to make the first column with all 1.
* We also adjust the matrix, multipling each row for a constant factor
* to make the first column with all 1, to optimize the computation
* for the first disk.
*
* This results in the "normalized" Extended Cauchy matrix A[row,col]
* defined as:
* This results in the matrix A[row,col] defined as:
*
* 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01...
* 01 02 04 08 10 20 40 80 1d 3a 74 e8 cd 87 13 26 4c 98 2d 5a b4 75...
@ -194,7 +190,7 @@
* [1] Anvin, "The mathematics of RAID-6", 2004
* [2] MacWilliams, Sloane, "The Theory of Error-Correcting Codes", 1977
* [3] Blömer, "An XOR-Based Erasure-Resilient Coding Scheme", 1995
* [4] Vinocha, Bhullar, Brar, "On Generator Cauchy Matrices of GDRS/GTRS Codes", 2012
* [4] Roth, "Introduction to Coding Theory", 2006
* [5] Plank, "Screaming Fast Galois Field Arithmetic Using Intel SIMD Instructions", 2013
*/

28
snapraid.1
View file

@ -151,7 +151,7 @@ For example, suppose that you are interested only at one parity level
of protection, and that your disks are present in:
.PP
.RS 4
/mnt/diskpar <\(hy selected disk for parity
/mnt/diskp <\(hy selected disk for parity
.PD 0
.PP
.PD
@ -159,7 +159,7 @@ of protection, and that your disks are present in:
.PD 0
.PP
.PD
/mnt/disk2 <\(hy second disk to backup
/mnt/disk2 <\(hy second disk to backup
.PD 0
.PP
.PD
@ -173,7 +173,7 @@ you have to create the configuration file /etc/snapraid.conf with
the following options:
.PP
.RS 4
parity /mnt/diskpar/parity
parity /mnt/diskp/parity
.PD 0
.PP
.PD
@ -207,7 +207,7 @@ If you are in Windows, you should use drive letters and backslashes
instead of slashes, and if you like, also file extensions.
.PP
.RS 4
parity E:\(rspar\(rsparity.par
parity E:\(rsparity.par
.PD 0
.PP
.PD
@ -789,15 +789,15 @@ For each parity file specified, one additional level of protection
is enabled:
.PD 0
.IP \(bu
q\(hyparity enables RAID6 Double Parity
q\(hyparity enables RAID6 double parity
.IP \(bu
r\(hyparity enables RAIDTP Triple Parity
r\(hyparity enables triple parity
.IP \(bu
s\(hyparity enables RAIDQP Quad Parity
s\(hyparity enables quad parity
.IP \(bu
t\(hyparity enables RAIDPP Penta Parity
t\(hyparity enables penta (five) parity
.IP \(bu
u\(hyparity enables RAIDHP Hexa Parity
u\(hyparity enables hexa (six) parity
.PD
.PP
Each parity level requires also all the files of the previous levels.
@ -897,11 +897,11 @@ The directory must already exist.
An example of a typical configuration for Unix is:
.PP
.RS 4
parity /mnt/diskpar/parity
parity /mnt/diskp/parity
.PD 0
.PP
.PD
content /mnt/diskpar/content
content /mnt/diskp/content
.PD 0
.PP
.PD
@ -934,15 +934,15 @@ exclude /tmp/
An example of a typical configuration for Windows is:
.PP
.RS 4
parity E:\(rspar\(rsparity
parity E:\(rsparity.par
.PD 0
.PP
.PD
content E:\(rspar\(rscontent
content E:\(rscontent.lst
.PD 0
.PP
.PD
content C:\(rssnapraid\(rscontent
content C:\(rssnapraid\(rscontent.lst
.PD 0
.PP
.PD

22
snapraid.conf.example
View file

@ -1,22 +1,22 @@
# Example configuration for snapraid
# Defines the file to use as Parity storage
# Defines the file to use as parity storage
# It must NOT be in a data disk
# Format: "parity FILE_PATH"
parity /mnt/diskpar/parity
parity /mnt/diskp/parity
# Defines the files to use as additional Parity storage.
# If specified, they enable the multiple failures protection with RAID6
# (Double), RAIDTP (Triple), RAIDQP (Quad), RAIDPP (Penta) and RAIDHP (Hexa).
# To enable, uncomment one parity file for each extra level of extra
# Defines the files to use as additional parity storage.
# If specified, they enable the multiple failures protection
# from two to six level of parity.
# To enable, uncomment one parity file for each level of extra
# protection required. Start from q-parity, and follow in order.
# It must NOT be in a data disk
# Format: "X-parity FILE_PATH"
#q-parity /mnt/diskqar/q-parity
#r-parity /mnt/diskqar/r-parity
#s-parity /mnt/diskqar/s-parity
#t-parity /mnt/diskqar/t-parity
#u-parity /mnt/diskqar/u-parity
#q-parity /mnt/diskq/q-parity
#r-parity /mnt/diskr/r-parity
#s-parity /mnt/disks/s-parity
#t-parity /mnt/diskt/t-parity
#u-parity /mnt/disku/u-parity
# Defines the files to use as content list
# You can use multiple specification to store more copies

10
snapraid.conf.example.windows
View file

@ -1,14 +1,14 @@
# Example configuration for snapraid for Windows
# Defines the file to use as Parity storage
# Defines the file to use as parity storage
# It must NOT be in a data disk
# Format: "parity FILE_PATH"
parity E:\parity.par
# Defines the files to use as additional Parity storage.
# If specified, they enable the multiple failures protection with RAID6
# (Double), RAIDTP (Triple), RAIDQP (Quad), RAIDPP (Penta) and RAIDHP (Hexa).
# To enable, uncomment one parity file for each extra level of extra
# Defines the files to use as additional parity storage.
# If specified, they enable the multiple failures protection
# from two to six level of parity.
# To enable, uncomment one parity file for each level of extra
# protection required. Start from q-parity, and follow in order.
# It must NOT be in a data disk
# Format: "X-parity FILE_PATH"

28
snapraid.d
View file

@ -101,15 +101,15 @@ Getting Started
For example, suppose that you are interested only at one parity level
of protection, and that your disks are present in:
:/mnt/diskpar <- selected disk for parity
:/mnt/diskp <- selected disk for parity
:/mnt/disk1 <- first disk to backup
:/mnt/disk2 <- second disk to backup
:/mnt/disk2 <- second disk to backup
:/mnt/disk3 <- third disk to backup
you have to create the configuration file /etc/snapraid.conf with
the following options:
:parity /mnt/diskpar/parity
:parity /mnt/diskp/parity
:content /var/snapraid/content
:content /mnt/disk1/content
:content /mnt/disk2/content
@ -120,7 +120,7 @@ Getting Started
If you are in Windows, you should use drive letters and backslashes
instead of slashes, and if you like, also file extensions.
:parity E:\par\parity.par
:parity E:\parity.par
:content C:\snapraid\content.lst
:content F:\array\content.lst
:content G:\array\content.lst
@ -586,11 +586,11 @@ Configuration
For each parity file specified, one additional level of protection
is enabled:
* q-parity enables RAID6 Double Parity
* r-parity enables RAIDTP Triple Parity
* s-parity enables RAIDQP Quad Parity
* t-parity enables RAIDPP Penta Parity
* u-parity enables RAIDHP Hexa Parity
* q-parity enables RAID6 double parity
* r-parity enables triple parity
* s-parity enables quad parity
* t-parity enables penta (five) parity
* u-parity enables hexa (six) parity
Each parity level requires also all the files of the previous levels.
@ -693,8 +693,8 @@ Configuration
Examples
An example of a typical configuration for Unix is:
:parity /mnt/diskpar/parity
:content /mnt/diskpar/content
:parity /mnt/diskp/parity
:content /mnt/diskp/content
:content /var/snapraid/content
:disk d1 /mnt/disk1/
:disk d2 /mnt/disk2/
@ -704,9 +704,9 @@ Configuration
An example of a typical configuration for Windows is:
:parity E:\par\parity
:content E:\par\content
:content C:\snapraid\content
:parity E:\parity.par
:content E:\content.lst
:content C:\snapraid\content.lst
:disk d1 G:\array\
:disk d2 H:\array\
:disk d3 I:\array\

28
snapraid.txt
View file

@ -114,15 +114,15 @@ disk failures at least one copy is available.
For example, suppose that you are interested only at one parity level
of protection, and that your disks are present in:
/mnt/diskpar <- selected disk for parity
/mnt/diskp <- selected disk for parity
/mnt/disk1 <- first disk to backup
/mnt/disk2 <- second disk to backup
/mnt/disk2 <- second disk to backup
/mnt/disk3 <- third disk to backup
you have to create the configuration file /etc/snapraid.conf with
the following options:
parity /mnt/diskpar/parity
parity /mnt/diskp/parity
content /var/snapraid/content
content /mnt/disk1/content
content /mnt/disk2/content
@ -133,7 +133,7 @@ the following options:
If you are in Windows, you should use drive letters and backslashes
instead of slashes, and if you like, also file extensions.
parity E:\par\parity.par
parity E:\parity.par
content C:\snapraid\content.lst
content F:\array\content.lst
content G:\array\content.lst
@ -640,11 +640,11 @@ Defines the files to use to store extra parity information.
For each parity file specified, one additional level of protection
is enabled:
* q-parity enables RAID6 Double Parity
* r-parity enables RAIDTP Triple Parity
* s-parity enables RAIDQP Quad Parity
* t-parity enables RAIDPP Penta Parity
* u-parity enables RAIDHP Hexa Parity
* q-parity enables RAID6 double parity
* r-parity enables triple parity
* s-parity enables quad parity
* t-parity enables penta (five) parity
* u-parity enables hexa (six) parity
Each parity level requires also all the files of the previous levels.
@ -759,8 +759,8 @@ The directory must already exist.
An example of a typical configuration for Unix is:
parity /mnt/diskpar/parity
content /mnt/diskpar/content
parity /mnt/diskp/parity
content /mnt/diskp/content
content /var/snapraid/content
disk d1 /mnt/disk1/
disk d2 /mnt/disk2/
@ -770,9 +770,9 @@ An example of a typical configuration for Unix is:
An example of a typical configuration for Windows is:
parity E:\par\parity
content E:\par\content
content C:\snapraid\content
parity E:\parity.par
content E:\content.lst
content C:\snapraid\content.lst
disk d1 G:\array\
disk d2 H:\array\
disk d3 I:\array\

1
util.c
View file

@ -1574,7 +1574,6 @@ int lock_unlock(int f)
* Removing it just introduces race course with other process
* that could have already opened it.
*/
if (close(f) == -1) {
return -1;
}