mirror of
https://github.com/amadvance/snapraid.git
synced 2026-10-26 10:51:54 -04:00
Reformats the documentation.
This commit is contained in:
Andrea Mazzoleni
parent
1ed3d59c02
commit
476697e68a
3 changed files with 398 additions and 324 deletions
120
snapraid.1
120
snapraid.1
|
|
@ -75,7 +75,7 @@ consider before using it.
|
|||
The main one is that if a disk fails, and you haven\(cqt recently synced,
|
||||
you may not able to do a complete recover.
|
||||
More specifically, you may be unable to recover up to the size of the
|
||||
amount of the changed or deleted files, from the last sync operation.
|
||||
amount of the changed or deleted files from the last sync operation.
|
||||
This happens even if the files changed or deleted are not in the
|
||||
failed disk.
|
||||
New added files don\(cqt prevent the recovering of the already existing
|
||||
|
|
@ -112,7 +112,7 @@ to recover from a single disk failure, like RAID5.
|
|||
If you want to recover from two disk failures, like RAID6, you must
|
||||
reserve another disk for the \(a"q\(hyparity\(a" information.
|
||||
.PP
|
||||
For the parity disks, you have to pick the biggest disks in the array,
|
||||
As parity disks, you have to pick the biggest disks in the array,
|
||||
as the redundancy information may grow in size as the biggest data
|
||||
disk in the array.
|
||||
.PP
|
||||
|
|
@ -170,8 +170,8 @@ disk d3 /mnt/disk3/
|
|||
.PD
|
||||
.RE
|
||||
.PP
|
||||
If you are in Windows, you should use drive letters and backslash
|
||||
instead of slash:
|
||||
If you are in Windows, you should use drive letters and backslashes
|
||||
instead of slashes:
|
||||
.PP
|
||||
.RS 4
|
||||
parity E:\(rspar\(rsparity
|
||||
|
|
@ -266,37 +266,40 @@ Restore the last backup/snapshot \(hy> \(a"fix\(a".
|
|||
.PD
|
||||
.PP
|
||||
Take care that the commands have be written in lower case.
|
||||
.TP
|
||||
.B sync
|
||||
.SS sync
|
||||
Updates the redundancy information. All the modified files
|
||||
in the disk array are read, and the redundancy data is
|
||||
recomputed.
|
||||
.PP
|
||||
Files are identified by inode and checked by time and size,
|
||||
meaning that you can move them on the disk without triggering
|
||||
any redundancy recomputation.
|
||||
.PP
|
||||
You can stop this process at any time pressing Ctrl+C,
|
||||
without losing the work already done.
|
||||
.PP
|
||||
The \(a"content\(a", \(a"parity\(a" and \(a"q\(hyparity\(a" files are modified if necessary.
|
||||
The files in the array are NOT modified.
|
||||
.TP
|
||||
.B diff
|
||||
.SS diff
|
||||
Lists all the files modified from the last \(a"sync\(a" command that
|
||||
needs to recompute their redundancy data.
|
||||
have to recompute their redundancy data.
|
||||
.PP
|
||||
Nothing is modified.
|
||||
.TP
|
||||
.B check
|
||||
.SS check
|
||||
Checks all the files and the redundancy data.
|
||||
All the files are hashed and compared with the snapshot saved
|
||||
in the previous \(a"sync\(a" command.
|
||||
.PP
|
||||
Files are identified by path, and checked by content.
|
||||
Nothing is modified.
|
||||
.TP
|
||||
.B fix
|
||||
.SS fix
|
||||
Checks and fix all the files. It\(cqs like \(a"check\(a" but it
|
||||
also tries to fix problems reverting the state of the
|
||||
disk array at the previous \(a"sync\(a" command.
|
||||
.PP
|
||||
After a successful \(a"fix\(a", you should also run a \(a"sync\(a"
|
||||
command to update the new state of the files.
|
||||
.PP
|
||||
The \(a"content\(a" file is NOT modified.
|
||||
The \(a"parity\(a" and \(a"q\(hyparity\(a" files are modified if necessary.
|
||||
The files in the array are modified if necessary.
|
||||
|
|
@ -304,9 +307,9 @@ The files in the array are modified if necessary.
|
|||
SnapRAID provides the following options:
|
||||
.TP
|
||||
.B \(hyc, \(hy\(hyconf CONFIG
|
||||
Selects the configuration file. If not specified is assumed
|
||||
the file \(cq/etc/snapraid.conf\(cq in Unix, and \(cqsnapraid.conf\(cq
|
||||
in Windows.
|
||||
Selects the configuration file. If not specified it\(cqs assumed
|
||||
the file \(cq/etc/snapraid.conf\(cq in Unix, and \(cqsnapraid.conf\(cq in
|
||||
the current directory in Windows.
|
||||
.TP
|
||||
.B \(hyf, \(hy\(hyfilter PATTERN
|
||||
Filters the files to operate on with the \(a"check\(a" and \(a"fix\(a"
|
||||
|
|
@ -318,17 +321,19 @@ In Unix, ensure to quote globbing chars if used.
|
|||
.TP
|
||||
.B \(hyZ, \(hy\(hyforce\(hyzero
|
||||
Forces the insecure operation of syncing a file with zero
|
||||
size that before was not empty.
|
||||
size that before was not.
|
||||
If SnapRAID detects such condition, it stops proceeding
|
||||
unless you specify this option.
|
||||
This allows to easily detect when after a system crash,
|
||||
some accessed files were zeroed.
|
||||
This is a possible condition in Linux with the ext3/ext4
|
||||
filesystems.
|
||||
.TP
|
||||
.B \(hyE, \(hy\(hyforce\(hyempty
|
||||
Forces the insecure operation of syncing a disk with all
|
||||
the original files missing.
|
||||
If SnapRAID detects that all the files originally present
|
||||
int the disk are missing or overwritten, it stops proceeding
|
||||
in the disk are missing or rewritten, it stops proceeding
|
||||
unless you specify this option.
|
||||
This allows to easily detect when a data file\(hysystem is not
|
||||
mounted.
|
||||
|
|
@ -358,98 +363,99 @@ This configuration file is located in /etc/snapraid.conf in Unix or
|
|||
in the execution directory in Windows.
|
||||
.PP
|
||||
It should contain the following options (case sensitive):
|
||||
.TP
|
||||
.B parity FILE
|
||||
.SS parity FILE
|
||||
Defines the file to use to store the parity information.
|
||||
The parity enables the protection from a single disk
|
||||
failure, like RAID5.
|
||||
.PP
|
||||
It must be placed in a disk dedicated for this purpose with
|
||||
as much free space as the biggest disk in the array.
|
||||
Leaving the parity disk reserved for only this file ensures that
|
||||
it doesn\(cqt get fragmented, improving the performance.
|
||||
.PP
|
||||
This option is mandatory and it can be used only one time.
|
||||
.TP
|
||||
.B q\(hyparity FILE
|
||||
.SS q\(hyparity FILE
|
||||
Defines the file to use to store the q\(hyparity information.
|
||||
If present, the q\(hyparity enables the protection from two disk
|
||||
failures, like RAID6.
|
||||
.PP
|
||||
It must be placed in a disk dedicated for this purpose with
|
||||
as much free space as the biggest disk in the array.
|
||||
Leaving the q\(hyparity disk reserved for only this file ensures that
|
||||
it doesn\(cqt get fragmented, improving the performance.
|
||||
.PP
|
||||
This option is optional and it can be used only one time.
|
||||
.TP
|
||||
.B content FILE
|
||||
.SS content FILE
|
||||
Defines the file to use to store the list and checksums of all the
|
||||
files present in your disk array.
|
||||
.PP
|
||||
It can be placed in the same disk of the parity and q\(hyparity files,
|
||||
but NOT in a data disk of the array.
|
||||
.PP
|
||||
This option is mandatory and it can be used more time to save
|
||||
more copies of the same files.
|
||||
.PP
|
||||
It\(cqs suggested to store at least one copy for each parity disk.
|
||||
One more doesn\(cqt hurt, just in case you lose all the parity disks,
|
||||
and you want to be still able to check the data integrity.
|
||||
.TP
|
||||
.B disk NAME DIR
|
||||
.SS disk NAME DIR
|
||||
Defines the name and the mount point of the disks of the array.
|
||||
NAME is used to identify the disk, and it must be unique.
|
||||
DIR is the mount point of the disk in the filesystem.
|
||||
.PP
|
||||
You can change the mount point as you like, as long you
|
||||
keep the NAME fixed.
|
||||
.PP
|
||||
The specification order is also important, if you change it,
|
||||
you will invalidate the q\(hyparity file.
|
||||
.PP
|
||||
You should use one option for each disk of the array.
|
||||
.TP
|
||||
.B exclude PATTERN
|
||||
.TP
|
||||
.B include PATTERN
|
||||
.SS exclude/include PATTERN
|
||||
Defines the file or directory patterns to exclude and include
|
||||
in the sync process.
|
||||
All the patterns are processed in the specified order.
|
||||
.PP
|
||||
If the first pattern that matches is an \(a"exclude\(a" one, the file
|
||||
is excluded. If it\(cqs an \(a"include\(a" one the file is included.
|
||||
If no pattern matches, the file is excluded if the last pattern
|
||||
is an \(a"include\(a", or included if the last pattern is an \(a"exclude\(a".
|
||||
See the PATTERN section for more details in the
|
||||
pattern specifications.
|
||||
.PP
|
||||
See the PATTERN section for more details in the pattern
|
||||
specifications.
|
||||
.PP
|
||||
This option can be used many times.
|
||||
.TP
|
||||
.B block_size SIZE_IN_KIBIBYTES
|
||||
.SS block_size SIZE_IN_KIBIBYTES
|
||||
Defines the basic block size in kibi bytes of the redundancy
|
||||
blocks. Where one kibi bytes is 1024 bytes.
|
||||
The default is 256 and it should work for most conditions.
|
||||
You could increase this value if you do not have enough RAM
|
||||
memory to run SnapRAID.
|
||||
SnapRAID requires about TS*24/BS bytes of RAM memory.
|
||||
Where TS is the total size in bytes of your disk array,
|
||||
and BS is the block size in bytes.
|
||||
.PP
|
||||
.RS 4
|
||||
As a rule of thumb, with 4GB or more memory use the default 256,
|
||||
with 2GB use 512, and with 1GB use 1024.
|
||||
.PP
|
||||
In more details SnapRAID requires about TS*24/BS bytes
|
||||
of RAM memory to run. Where TS is the total size in bytes of
|
||||
your disk array, and BS is the block size in bytes.
|
||||
.PP
|
||||
For example with 6 disk of 2 TiB and a block size of 256 KiB
|
||||
(1 KiB = 1024 Bytes) you have:
|
||||
.RE
|
||||
.PP
|
||||
.RS 4
|
||||
RAM = (6 * 2 * 2^40) * 24 / (256 * 2^10) = 1.1 GiB
|
||||
.PD 0
|
||||
.PP
|
||||
.PD
|
||||
.RE
|
||||
.PP
|
||||
.RS 4
|
||||
You could instead decrease this value if you have a lot of
|
||||
small files in the disk array. For each file, even if of few
|
||||
bytes, a whole block is always allocated, so you may have a lot
|
||||
of unused space.
|
||||
As approximation, you can assume that half of the block size is
|
||||
wasted for each file.
|
||||
.RE
|
||||
.PP
|
||||
.RS 4
|
||||
For example, with 10000 files and a 256 KiB block size, you are
|
||||
going to waste 1.2 GiB.
|
||||
.RE
|
||||
.PP
|
||||
.SS Examples
|
||||
An example of a typical configuration for Unix is:
|
||||
.PP
|
||||
.RS 4
|
||||
|
|
@ -691,35 +697,37 @@ for documentation.
|
|||
.PP
|
||||
This file is read and written by the \(a"sync\(a" command, and only read by
|
||||
\(a"fix\(a" and \(a"check\(a".
|
||||
.TP
|
||||
.B blk_size SIZE
|
||||
.SS blk_size SIZE
|
||||
Defines the size of the block in bytes. It must match the size
|
||||
defined in the configuration file.
|
||||
.TP
|
||||
.B checksum CHECKSUM
|
||||
.SS checksum CHECKSUM
|
||||
Defines the checksum kind used. It can be \(a"md5\(a" or \(a"murmur3\(a".
|
||||
.TP
|
||||
.B file DISK SIZE TIME INODE PATH
|
||||
.SS file DISK SIZE TIME INODE PATH
|
||||
Defines a file in the specified DISK.
|
||||
.PP
|
||||
The INODE number is used to identify the file in the \(a"sync\(a"
|
||||
command, allowing to rename or move the file in disk without
|
||||
the need to recompute the parity for it.
|
||||
.PP
|
||||
The SIZE and TIME information are used to identify if the file
|
||||
changed from the last \(a"sync\(a" command, and if there is the need
|
||||
to recompute the parity.
|
||||
.PP
|
||||
The PATH information is used in the \(a"check\(a" and \(a"fix\(a" commands
|
||||
to identify the file.
|
||||
.TP
|
||||
.B blk BLOCK HASH
|
||||
.SS blk BLOCK HASH
|
||||
Defines the ordered parity block list used by the last defined file.
|
||||
.PP
|
||||
BLOCK is the block position in the \(a"parity\(a" file.
|
||||
0 for the first block, 1 for the second one and so on.
|
||||
.PP
|
||||
HASH is the hash of the block. In the last block of the file,
|
||||
the HASH is the hash of only the used part of the block.
|
||||
.TP
|
||||
.B inv BLOCK [HASH]
|
||||
.SS inv BLOCK [HASH]
|
||||
Like \(a"blk\(a", but inform that the parity of this block is invalid.
|
||||
.PP
|
||||
The HASH may be missing if not yet computed.
|
||||
.PP
|
||||
This field is used only when you interrupt manually the \(a"sync\(a"
|
||||
command.
|
||||
.SH PARITY
|
||||
|
|
|
|||
301
snapraid.d
301
snapraid.d
|
|
@ -46,7 +46,7 @@ Limitations
|
|||
The main one is that if a disk fails, and you haven't recently synced,
|
||||
you may not able to do a complete recover.
|
||||
More specifically, you may be unable to recover up to the size of the
|
||||
amount of the changed or deleted files, from the last sync operation.
|
||||
amount of the changed or deleted files from the last sync operation.
|
||||
This happens even if the files changed or deleted are not in the
|
||||
failed disk.
|
||||
New added files don't prevent the recovering of the already existing
|
||||
|
|
@ -78,7 +78,7 @@ Getting Started
|
|||
If you want to recover from two disk failures, like RAID6, you must
|
||||
reserve another disk for the "q-parity" information.
|
||||
|
||||
For the parity disks, you have to pick the biggest disks in the array,
|
||||
As parity disks, you have to pick the biggest disks in the array,
|
||||
as the redundancy information may grow in size as the biggest data
|
||||
disk in the array.
|
||||
|
||||
|
|
@ -105,8 +105,8 @@ Getting Started
|
|||
:disk d2 /mnt/disk2/
|
||||
:disk d3 /mnt/disk3/
|
||||
|
||||
If you are in Windows, you should use drive letters and backslash
|
||||
instead of slash:
|
||||
If you are in Windows, you should use drive letters and backslashes
|
||||
instead of slashes:
|
||||
|
||||
:parity E:\par\parity
|
||||
:content E:\par\content
|
||||
|
|
@ -161,47 +161,54 @@ Commands
|
|||
|
||||
Take care that the commands have be written in lower case.
|
||||
|
||||
=sync
|
||||
Updates the redundancy information. All the modified files
|
||||
in the disk array are read, and the redundancy data is
|
||||
recomputed.
|
||||
Files are identified by inode and checked by time and size,
|
||||
meaning that you can move them on the disk without triggering
|
||||
any redundancy recomputation.
|
||||
You can stop this process at any time pressing Ctrl+C,
|
||||
without losing the work already done.
|
||||
The "content", "parity" and "q-parity" files are modified if necessary.
|
||||
The files in the array are NOT modified.
|
||||
sync
|
||||
Updates the redundancy information. All the modified files
|
||||
in the disk array are read, and the redundancy data is
|
||||
recomputed.
|
||||
|
||||
=diff
|
||||
Lists all the files modified from the last "sync" command that
|
||||
needs to recompute their redundancy data.
|
||||
Nothing is modified.
|
||||
Files are identified by inode and checked by time and size,
|
||||
meaning that you can move them on the disk without triggering
|
||||
any redundancy recomputation.
|
||||
|
||||
=check
|
||||
Checks all the files and the redundancy data.
|
||||
All the files are hashed and compared with the snapshot saved
|
||||
in the previous "sync" command.
|
||||
Files are identified by path, and checked by content.
|
||||
Nothing is modified.
|
||||
You can stop this process at any time pressing Ctrl+C,
|
||||
without losing the work already done.
|
||||
|
||||
=fix
|
||||
Checks and fix all the files. It's like "check" but it
|
||||
also tries to fix problems reverting the state of the
|
||||
disk array at the previous "sync" command.
|
||||
After a successful "fix", you should also run a "sync"
|
||||
command to update the new state of the files.
|
||||
The "content" file is NOT modified.
|
||||
The "parity" and "q-parity" files are modified if necessary.
|
||||
The files in the array are modified if necessary.
|
||||
The "content", "parity" and "q-parity" files are modified if necessary.
|
||||
The files in the array are NOT modified.
|
||||
|
||||
diff
|
||||
Lists all the files modified from the last "sync" command that
|
||||
have to recompute their redundancy data.
|
||||
|
||||
Nothing is modified.
|
||||
|
||||
check
|
||||
Checks all the files and the redundancy data.
|
||||
All the files are hashed and compared with the snapshot saved
|
||||
in the previous "sync" command.
|
||||
|
||||
Files are identified by path, and checked by content.
|
||||
Nothing is modified.
|
||||
|
||||
fix
|
||||
Checks and fix all the files. It's like "check" but it
|
||||
also tries to fix problems reverting the state of the
|
||||
disk array at the previous "sync" command.
|
||||
|
||||
After a successful "fix", you should also run a "sync"
|
||||
command to update the new state of the files.
|
||||
|
||||
The "content" file is NOT modified.
|
||||
The "parity" and "q-parity" files are modified if necessary.
|
||||
The files in the array are modified if necessary.
|
||||
|
||||
Options
|
||||
SnapRAID provides the following options:
|
||||
|
||||
-c, --conf CONFIG
|
||||
Selects the configuration file. If not specified is assumed
|
||||
the file '/etc/snapraid.conf' in Unix, and 'snapraid.conf'
|
||||
in Windows.
|
||||
Selects the configuration file. If not specified it's assumed
|
||||
the file '/etc/snapraid.conf' in Unix, and 'snapraid.conf' in
|
||||
the current directory in Windows.
|
||||
|
||||
-f, --filter PATTERN
|
||||
Filters the files to operate on with the "check" and "fix"
|
||||
|
|
@ -213,17 +220,19 @@ Options
|
|||
|
||||
-Z, --force-zero
|
||||
Forces the insecure operation of syncing a file with zero
|
||||
size that before was not empty.
|
||||
size that before was not.
|
||||
If SnapRAID detects such condition, it stops proceeding
|
||||
unless you specify this option.
|
||||
This allows to easily detect when after a system crash,
|
||||
some accessed files were zeroed.
|
||||
This is a possible condition in Linux with the ext3/ext4
|
||||
filesystems.
|
||||
|
||||
-E, --force-empty
|
||||
Forces the insecure operation of syncing a disk with all
|
||||
the original files missing.
|
||||
If SnapRAID detects that all the files originally present
|
||||
int the disk are missing or overwritten, it stops proceeding
|
||||
in the disk are missing or rewritten, it stops proceeding
|
||||
unless you specify this option.
|
||||
This allows to easily detect when a data file-system is not
|
||||
mounted.
|
||||
|
|
@ -255,85 +264,102 @@ Configuration
|
|||
|
||||
It should contain the following options (case sensitive):
|
||||
|
||||
=parity FILE
|
||||
Defines the file to use to store the parity information.
|
||||
The parity enables the protection from a single disk
|
||||
failure, like RAID5.
|
||||
It must be placed in a disk dedicated for this purpose with
|
||||
as much free space as the biggest disk in the array.
|
||||
Leaving the parity disk reserved for only this file ensures that
|
||||
it doesn't get fragmented, improving the performance.
|
||||
This option is mandatory and it can be used only one time.
|
||||
parity FILE
|
||||
Defines the file to use to store the parity information.
|
||||
The parity enables the protection from a single disk
|
||||
failure, like RAID5.
|
||||
|
||||
It must be placed in a disk dedicated for this purpose with
|
||||
as much free space as the biggest disk in the array.
|
||||
Leaving the parity disk reserved for only this file ensures that
|
||||
it doesn't get fragmented, improving the performance.
|
||||
|
||||
=q-parity FILE
|
||||
Defines the file to use to store the q-parity information.
|
||||
If present, the q-parity enables the protection from two disk
|
||||
failures, like RAID6.
|
||||
It must be placed in a disk dedicated for this purpose with
|
||||
as much free space as the biggest disk in the array.
|
||||
Leaving the q-parity disk reserved for only this file ensures that
|
||||
it doesn't get fragmented, improving the performance.
|
||||
This option is optional and it can be used only one time.
|
||||
This option is mandatory and it can be used only one time.
|
||||
|
||||
=content FILE
|
||||
Defines the file to use to store the list and checksums of all the
|
||||
files present in your disk array.
|
||||
It can be placed in the same disk of the parity and q-parity files,
|
||||
but NOT in a data disk of the array.
|
||||
This option is mandatory and it can be used more time to save
|
||||
more copies of the same files.
|
||||
It's suggested to store at least one copy for each parity disk.
|
||||
One more doesn't hurt, just in case you lose all the parity disks,
|
||||
and you want to be still able to check the data integrity.
|
||||
q-parity FILE
|
||||
Defines the file to use to store the q-parity information.
|
||||
If present, the q-parity enables the protection from two disk
|
||||
failures, like RAID6.
|
||||
|
||||
=disk NAME DIR
|
||||
Defines the name and the mount point of the disks of the array.
|
||||
NAME is used to identify the disk, and it must be unique.
|
||||
DIR is the mount point of the disk in the filesystem.
|
||||
You can change the mount point as you like, as long you
|
||||
keep the NAME fixed.
|
||||
The specification order is also important, if you change it,
|
||||
you will invalidate the q-parity file.
|
||||
You should use one option for each disk of the array.
|
||||
It must be placed in a disk dedicated for this purpose with
|
||||
as much free space as the biggest disk in the array.
|
||||
Leaving the q-parity disk reserved for only this file ensures that
|
||||
it doesn't get fragmented, improving the performance.
|
||||
|
||||
=exclude PATTERN
|
||||
=include PATTERN
|
||||
Defines the file or directory patterns to exclude and include
|
||||
in the sync process.
|
||||
All the patterns are processed in the specified order.
|
||||
If the first pattern that matches is an "exclude" one, the file
|
||||
is excluded. If it's an "include" one the file is included.
|
||||
If no pattern matches, the file is excluded if the last pattern
|
||||
is an "include", or included if the last pattern is an "exclude".
|
||||
See the PATTERN section for more details in the
|
||||
pattern specifications.
|
||||
This option can be used many times.
|
||||
This option is optional and it can be used only one time.
|
||||
|
||||
=block_size SIZE_IN_KIBIBYTES
|
||||
Defines the basic block size in kibi bytes of the redundancy
|
||||
blocks. Where one kibi bytes is 1024 bytes.
|
||||
The default is 256 and it should work for most conditions.
|
||||
You could increase this value if you do not have enough RAM
|
||||
memory to run SnapRAID.
|
||||
SnapRAID requires about TS*24/BS bytes of RAM memory.
|
||||
Where TS is the total size in bytes of your disk array,
|
||||
and BS is the block size in bytes.
|
||||
content FILE
|
||||
Defines the file to use to store the list and checksums of all the
|
||||
files present in your disk array.
|
||||
|
||||
For example with 6 disk of 2 TiB and a block size of 256 KiB
|
||||
(1 KiB = 1024 Bytes) you have:
|
||||
It can be placed in the same disk of the parity and q-parity files,
|
||||
but NOT in a data disk of the array.
|
||||
|
||||
:RAM = (6 * 2 * 2^40) * 24 / (256 * 2^10) = 1.1 GiB
|
||||
This option is mandatory and it can be used more time to save
|
||||
more copies of the same files.
|
||||
|
||||
You could instead decrease this value if you have a lot of
|
||||
small files in the disk array. For each file, even if of few
|
||||
bytes, a whole block is always allocated, so you may have a lot
|
||||
of unused space.
|
||||
As approximation, you can assume that half of the block size is
|
||||
wasted for each file.
|
||||
It's suggested to store at least one copy for each parity disk.
|
||||
One more doesn't hurt, just in case you lose all the parity disks,
|
||||
and you want to be still able to check the data integrity.
|
||||
|
||||
For example, with 10000 files and a 256 KiB block size, you are
|
||||
going to waste 1.2 GiB.
|
||||
disk NAME DIR
|
||||
Defines the name and the mount point of the disks of the array.
|
||||
NAME is used to identify the disk, and it must be unique.
|
||||
DIR is the mount point of the disk in the filesystem.
|
||||
|
||||
You can change the mount point as you like, as long you
|
||||
keep the NAME fixed.
|
||||
|
||||
The specification order is also important, if you change it,
|
||||
you will invalidate the q-parity file.
|
||||
|
||||
You should use one option for each disk of the array.
|
||||
|
||||
exclude/include PATTERN
|
||||
Defines the file or directory patterns to exclude and include
|
||||
in the sync process.
|
||||
All the patterns are processed in the specified order.
|
||||
|
||||
If the first pattern that matches is an "exclude" one, the file
|
||||
is excluded. If it's an "include" one the file is included.
|
||||
If no pattern matches, the file is excluded if the last pattern
|
||||
is an "include", or included if the last pattern is an "exclude".
|
||||
|
||||
See the PATTERN section for more details in the pattern
|
||||
specifications.
|
||||
|
||||
This option can be used many times.
|
||||
|
||||
block_size SIZE_IN_KIBIBYTES
|
||||
Defines the basic block size in kibi bytes of the redundancy
|
||||
blocks. Where one kibi bytes is 1024 bytes.
|
||||
The default is 256 and it should work for most conditions.
|
||||
You could increase this value if you do not have enough RAM
|
||||
memory to run SnapRAID.
|
||||
|
||||
As a rule of thumb, with 4GB or more memory use the default 256,
|
||||
with 2GB use 512, and with 1GB use 1024.
|
||||
|
||||
In more details SnapRAID requires about TS*24/BS bytes
|
||||
of RAM memory to run. Where TS is the total size in bytes of
|
||||
your disk array, and BS is the block size in bytes.
|
||||
|
||||
For example with 6 disk of 2 TiB and a block size of 256 KiB
|
||||
(1 KiB = 1024 Bytes) you have:
|
||||
|
||||
:RAM = (6 * 2 * 2^40) * 24 / (256 * 2^10) = 1.1 GiB
|
||||
|
||||
You could instead decrease this value if you have a lot of
|
||||
small files in the disk array. For each file, even if of few
|
||||
bytes, a whole block is always allocated, so you may have a lot
|
||||
of unused space.
|
||||
As approximation, you can assume that half of the block size is
|
||||
wasted for each file.
|
||||
|
||||
For example, with 10000 files and a 256 KiB block size, you are
|
||||
going to waste 1.2 GiB.
|
||||
|
||||
Examples
|
||||
An example of a typical configuration for Unix is:
|
||||
|
||||
:parity /mnt/diskpar/parity
|
||||
|
|
@ -446,36 +472,43 @@ Content
|
|||
This file is read and written by the "sync" command, and only read by
|
||||
"fix" and "check".
|
||||
|
||||
=blk_size SIZE
|
||||
Defines the size of the block in bytes. It must match the size
|
||||
defined in the configuration file.
|
||||
blk_size SIZE
|
||||
Defines the size of the block in bytes. It must match the size
|
||||
defined in the configuration file.
|
||||
|
||||
=checksum CHECKSUM
|
||||
Defines the checksum kind used. It can be "md5" or "murmur3".
|
||||
checksum CHECKSUM
|
||||
Defines the checksum kind used. It can be "md5" or "murmur3".
|
||||
|
||||
=file DISK SIZE TIME INODE PATH
|
||||
Defines a file in the specified DISK.
|
||||
The INODE number is used to identify the file in the "sync"
|
||||
command, allowing to rename or move the file in disk without
|
||||
the need to recompute the parity for it.
|
||||
The SIZE and TIME information are used to identify if the file
|
||||
changed from the last "sync" command, and if there is the need
|
||||
to recompute the parity.
|
||||
The PATH information is used in the "check" and "fix" commands
|
||||
to identify the file.
|
||||
file DISK SIZE TIME INODE PATH
|
||||
Defines a file in the specified DISK.
|
||||
|
||||
=blk BLOCK HASH
|
||||
Defines the ordered parity block list used by the last defined file.
|
||||
BLOCK is the block position in the "parity" file.
|
||||
0 for the first block, 1 for the second one and so on.
|
||||
HASH is the hash of the block. In the last block of the file,
|
||||
the HASH is the hash of only the used part of the block.
|
||||
The INODE number is used to identify the file in the "sync"
|
||||
command, allowing to rename or move the file in disk without
|
||||
the need to recompute the parity for it.
|
||||
|
||||
The SIZE and TIME information are used to identify if the file
|
||||
changed from the last "sync" command, and if there is the need
|
||||
to recompute the parity.
|
||||
|
||||
=inv BLOCK [HASH]
|
||||
Like "blk", but inform that the parity of this block is invalid.
|
||||
The HASH may be missing if not yet computed.
|
||||
This field is used only when you interrupt manually the "sync"
|
||||
command.
|
||||
The PATH information is used in the "check" and "fix" commands
|
||||
to identify the file.
|
||||
|
||||
blk BLOCK HASH
|
||||
Defines the ordered parity block list used by the last defined file.
|
||||
|
||||
BLOCK is the block position in the "parity" file.
|
||||
0 for the first block, 1 for the second one and so on.
|
||||
|
||||
HASH is the hash of the block. In the last block of the file,
|
||||
the HASH is the hash of only the used part of the block.
|
||||
|
||||
inv BLOCK [HASH]
|
||||
Like "blk", but inform that the parity of this block is invalid.
|
||||
|
||||
The HASH may be missing if not yet computed.
|
||||
|
||||
This field is used only when you interrupt manually the "sync"
|
||||
command.
|
||||
|
||||
Parity
|
||||
SnapRAID stores the redundancy information of your array in the parity
|
||||
|
|
|
|||
301
snapraid.txt
301
snapraid.txt
|
|
@ -56,7 +56,7 @@ consider before using it.
|
|||
The main one is that if a disk fails, and you haven't recently synced,
|
||||
you may not able to do a complete recover.
|
||||
More specifically, you may be unable to recover up to the size of the
|
||||
amount of the changed or deleted files, from the last sync operation.
|
||||
amount of the changed or deleted files from the last sync operation.
|
||||
This happens even if the files changed or deleted are not in the
|
||||
failed disk.
|
||||
New added files don't prevent the recovering of the already existing
|
||||
|
|
@ -91,7 +91,7 @@ to recover from a single disk failure, like RAID5.
|
|||
If you want to recover from two disk failures, like RAID6, you must
|
||||
reserve another disk for the "q-parity" information.
|
||||
|
||||
For the parity disks, you have to pick the biggest disks in the array,
|
||||
As parity disks, you have to pick the biggest disks in the array,
|
||||
as the redundancy information may grow in size as the biggest data
|
||||
disk in the array.
|
||||
|
||||
|
|
@ -118,8 +118,8 @@ the following options:
|
|||
disk d2 /mnt/disk2/
|
||||
disk d3 /mnt/disk3/
|
||||
|
||||
If you are in Windows, you should use drive letters and backslash
|
||||
instead of slash:
|
||||
If you are in Windows, you should use drive letters and backslashes
|
||||
instead of slashes:
|
||||
|
||||
parity E:\par\parity
|
||||
content E:\par\content
|
||||
|
|
@ -177,39 +177,46 @@ SnapRAID provides four simple commands that allow to:
|
|||
|
||||
Take care that the commands have be written in lower case.
|
||||
|
||||
sync
|
||||
Updates the redundancy information. All the modified files
|
||||
in the disk array are read, and the redundancy data is
|
||||
recomputed.
|
||||
Files are identified by inode and checked by time and size,
|
||||
meaning that you can move them on the disk without triggering
|
||||
any redundancy recomputation.
|
||||
You can stop this process at any time pressing Ctrl+C,
|
||||
without losing the work already done.
|
||||
The "content", "parity" and "q-parity" files are modified if necessary.
|
||||
The files in the array are NOT modified.
|
||||
---- 5.0.1 sync ----
|
||||
Updates the redundancy information. All the modified files
|
||||
in the disk array are read, and the redundancy data is
|
||||
recomputed.
|
||||
|
||||
diff
|
||||
Lists all the files modified from the last "sync" command that
|
||||
needs to recompute their redundancy data.
|
||||
Nothing is modified.
|
||||
Files are identified by inode and checked by time and size,
|
||||
meaning that you can move them on the disk without triggering
|
||||
any redundancy recomputation.
|
||||
|
||||
check
|
||||
Checks all the files and the redundancy data.
|
||||
All the files are hashed and compared with the snapshot saved
|
||||
in the previous "sync" command.
|
||||
Files are identified by path, and checked by content.
|
||||
Nothing is modified.
|
||||
You can stop this process at any time pressing Ctrl+C,
|
||||
without losing the work already done.
|
||||
|
||||
fix
|
||||
Checks and fix all the files. It's like "check" but it
|
||||
also tries to fix problems reverting the state of the
|
||||
disk array at the previous "sync" command.
|
||||
After a successful "fix", you should also run a "sync"
|
||||
command to update the new state of the files.
|
||||
The "content" file is NOT modified.
|
||||
The "parity" and "q-parity" files are modified if necessary.
|
||||
The files in the array are modified if necessary.
|
||||
The "content", "parity" and "q-parity" files are modified if necessary.
|
||||
The files in the array are NOT modified.
|
||||
|
||||
---- 5.0.2 diff ----
|
||||
Lists all the files modified from the last "sync" command that
|
||||
have to recompute their redundancy data.
|
||||
|
||||
Nothing is modified.
|
||||
|
||||
---- 5.0.3 check ----
|
||||
Checks all the files and the redundancy data.
|
||||
All the files are hashed and compared with the snapshot saved
|
||||
in the previous "sync" command.
|
||||
|
||||
Files are identified by path, and checked by content.
|
||||
Nothing is modified.
|
||||
|
||||
---- 5.0.4 fix ----
|
||||
Checks and fix all the files. It's like "check" but it
|
||||
also tries to fix problems reverting the state of the
|
||||
disk array at the previous "sync" command.
|
||||
|
||||
After a successful "fix", you should also run a "sync"
|
||||
command to update the new state of the files.
|
||||
|
||||
The "content" file is NOT modified.
|
||||
The "parity" and "q-parity" files are modified if necessary.
|
||||
The files in the array are modified if necessary.
|
||||
|
||||
|
||||
6 OPTIONS
|
||||
|
|
@ -218,9 +225,9 @@ Take care that the commands have be written in lower case.
|
|||
SnapRAID provides the following options:
|
||||
|
||||
-c, --conf CONFIG
|
||||
Selects the configuration file. If not specified is assumed
|
||||
the file '/etc/snapraid.conf' in Unix, and 'snapraid.conf'
|
||||
in Windows.
|
||||
Selects the configuration file. If not specified it's assumed
|
||||
the file '/etc/snapraid.conf' in Unix, and 'snapraid.conf' in
|
||||
the current directory in Windows.
|
||||
|
||||
-f, --filter PATTERN
|
||||
Filters the files to operate on with the "check" and "fix"
|
||||
|
|
@ -232,17 +239,19 @@ SnapRAID provides the following options:
|
|||
|
||||
-Z, --force-zero
|
||||
Forces the insecure operation of syncing a file with zero
|
||||
size that before was not empty.
|
||||
size that before was not.
|
||||
If SnapRAID detects such condition, it stops proceeding
|
||||
unless you specify this option.
|
||||
This allows to easily detect when after a system crash,
|
||||
some accessed files were zeroed.
|
||||
This is a possible condition in Linux with the ext3/ext4
|
||||
filesystems.
|
||||
|
||||
-E, --force-empty
|
||||
Forces the insecure operation of syncing a disk with all
|
||||
the original files missing.
|
||||
If SnapRAID detects that all the files originally present
|
||||
int the disk are missing or overwritten, it stops proceeding
|
||||
in the disk are missing or rewritten, it stops proceeding
|
||||
unless you specify this option.
|
||||
This allows to easily detect when a data file-system is not
|
||||
mounted.
|
||||
|
|
@ -277,85 +286,102 @@ in the execution directory in Windows.
|
|||
|
||||
It should contain the following options (case sensitive):
|
||||
|
||||
parity FILE
|
||||
Defines the file to use to store the parity information.
|
||||
The parity enables the protection from a single disk
|
||||
failure, like RAID5.
|
||||
It must be placed in a disk dedicated for this purpose with
|
||||
as much free space as the biggest disk in the array.
|
||||
Leaving the parity disk reserved for only this file ensures that
|
||||
it doesn't get fragmented, improving the performance.
|
||||
This option is mandatory and it can be used only one time.
|
||||
---- 7.0.1 parity FILE ----
|
||||
Defines the file to use to store the parity information.
|
||||
The parity enables the protection from a single disk
|
||||
failure, like RAID5.
|
||||
|
||||
q-parity FILE
|
||||
Defines the file to use to store the q-parity information.
|
||||
If present, the q-parity enables the protection from two disk
|
||||
failures, like RAID6.
|
||||
It must be placed in a disk dedicated for this purpose with
|
||||
as much free space as the biggest disk in the array.
|
||||
Leaving the q-parity disk reserved for only this file ensures that
|
||||
it doesn't get fragmented, improving the performance.
|
||||
This option is optional and it can be used only one time.
|
||||
It must be placed in a disk dedicated for this purpose with
|
||||
as much free space as the biggest disk in the array.
|
||||
Leaving the parity disk reserved for only this file ensures that
|
||||
it doesn't get fragmented, improving the performance.
|
||||
|
||||
content FILE
|
||||
Defines the file to use to store the list and checksums of all the
|
||||
files present in your disk array.
|
||||
It can be placed in the same disk of the parity and q-parity files,
|
||||
but NOT in a data disk of the array.
|
||||
This option is mandatory and it can be used more time to save
|
||||
more copies of the same files.
|
||||
It's suggested to store at least one copy for each parity disk.
|
||||
One more doesn't hurt, just in case you lose all the parity disks,
|
||||
and you want to be still able to check the data integrity.
|
||||
This option is mandatory and it can be used only one time.
|
||||
|
||||
disk NAME DIR
|
||||
Defines the name and the mount point of the disks of the array.
|
||||
NAME is used to identify the disk, and it must be unique.
|
||||
DIR is the mount point of the disk in the filesystem.
|
||||
You can change the mount point as you like, as long you
|
||||
keep the NAME fixed.
|
||||
The specification order is also important, if you change it,
|
||||
you will invalidate the q-parity file.
|
||||
You should use one option for each disk of the array.
|
||||
---- 7.0.2 q-parity FILE ----
|
||||
Defines the file to use to store the q-parity information.
|
||||
If present, the q-parity enables the protection from two disk
|
||||
failures, like RAID6.
|
||||
|
||||
exclude PATTERN
|
||||
include PATTERN
|
||||
Defines the file or directory patterns to exclude and include
|
||||
in the sync process.
|
||||
All the patterns are processed in the specified order.
|
||||
If the first pattern that matches is an "exclude" one, the file
|
||||
is excluded. If it's an "include" one the file is included.
|
||||
If no pattern matches, the file is excluded if the last pattern
|
||||
is an "include", or included if the last pattern is an "exclude".
|
||||
See the PATTERN section for more details in the
|
||||
pattern specifications.
|
||||
This option can be used many times.
|
||||
It must be placed in a disk dedicated for this purpose with
|
||||
as much free space as the biggest disk in the array.
|
||||
Leaving the q-parity disk reserved for only this file ensures that
|
||||
it doesn't get fragmented, improving the performance.
|
||||
|
||||
block_size SIZE_IN_KIBIBYTES
|
||||
Defines the basic block size in kibi bytes of the redundancy
|
||||
blocks. Where one kibi bytes is 1024 bytes.
|
||||
The default is 256 and it should work for most conditions.
|
||||
You could increase this value if you do not have enough RAM
|
||||
memory to run SnapRAID.
|
||||
SnapRAID requires about TS*24/BS bytes of RAM memory.
|
||||
Where TS is the total size in bytes of your disk array,
|
||||
and BS is the block size in bytes.
|
||||
This option is optional and it can be used only one time.
|
||||
|
||||
For example with 6 disk of 2 TiB and a block size of 256 KiB
|
||||
(1 KiB = 1024 Bytes) you have:
|
||||
---- 7.0.3 content FILE ----
|
||||
Defines the file to use to store the list and checksums of all the
|
||||
files present in your disk array.
|
||||
|
||||
RAM = (6 * 2 * 2^40) * 24 / (256 * 2^10) = 1.1 GiB
|
||||
It can be placed in the same disk of the parity and q-parity files,
|
||||
but NOT in a data disk of the array.
|
||||
|
||||
You could instead decrease this value if you have a lot of
|
||||
small files in the disk array. For each file, even if of few
|
||||
bytes, a whole block is always allocated, so you may have a lot
|
||||
of unused space.
|
||||
As approximation, you can assume that half of the block size is
|
||||
wasted for each file.
|
||||
This option is mandatory and it can be used more time to save
|
||||
more copies of the same files.
|
||||
|
||||
For example, with 10000 files and a 256 KiB block size, you are
|
||||
going to waste 1.2 GiB.
|
||||
It's suggested to store at least one copy for each parity disk.
|
||||
One more doesn't hurt, just in case you lose all the parity disks,
|
||||
and you want to be still able to check the data integrity.
|
||||
|
||||
---- 7.0.4 disk NAME DIR ----
|
||||
Defines the name and the mount point of the disks of the array.
|
||||
NAME is used to identify the disk, and it must be unique.
|
||||
DIR is the mount point of the disk in the filesystem.
|
||||
|
||||
You can change the mount point as you like, as long you
|
||||
keep the NAME fixed.
|
||||
|
||||
The specification order is also important, if you change it,
|
||||
you will invalidate the q-parity file.
|
||||
|
||||
You should use one option for each disk of the array.
|
||||
|
||||
---- 7.0.5 exclude/include PATTERN ----
|
||||
Defines the file or directory patterns to exclude and include
|
||||
in the sync process.
|
||||
All the patterns are processed in the specified order.
|
||||
|
||||
If the first pattern that matches is an "exclude" one, the file
|
||||
is excluded. If it's an "include" one the file is included.
|
||||
If no pattern matches, the file is excluded if the last pattern
|
||||
is an "include", or included if the last pattern is an "exclude".
|
||||
|
||||
See the PATTERN section for more details in the pattern
|
||||
specifications.
|
||||
|
||||
This option can be used many times.
|
||||
|
||||
---- 7.0.6 block_size SIZE_IN_KIBIBYTES ----
|
||||
Defines the basic block size in kibi bytes of the redundancy
|
||||
blocks. Where one kibi bytes is 1024 bytes.
|
||||
The default is 256 and it should work for most conditions.
|
||||
You could increase this value if you do not have enough RAM
|
||||
memory to run SnapRAID.
|
||||
|
||||
As a rule of thumb, with 4GB or more memory use the default 256,
|
||||
with 2GB use 512, and with 1GB use 1024.
|
||||
|
||||
In more details SnapRAID requires about TS*24/BS bytes
|
||||
of RAM memory to run. Where TS is the total size in bytes of
|
||||
your disk array, and BS is the block size in bytes.
|
||||
|
||||
For example with 6 disk of 2 TiB and a block size of 256 KiB
|
||||
(1 KiB = 1024 Bytes) you have:
|
||||
|
||||
RAM = (6 * 2 * 2^40) * 24 / (256 * 2^10) = 1.1 GiB
|
||||
|
||||
You could instead decrease this value if you have a lot of
|
||||
small files in the disk array. For each file, even if of few
|
||||
bytes, a whole block is always allocated, so you may have a lot
|
||||
of unused space.
|
||||
As approximation, you can assume that half of the block size is
|
||||
wasted for each file.
|
||||
|
||||
For example, with 10000 files and a 256 KiB block size, you are
|
||||
going to waste 1.2 GiB.
|
||||
|
||||
---- 7.0.7 Examples ----
|
||||
An example of a typical configuration for Unix is:
|
||||
|
||||
parity /mnt/diskpar/parity
|
||||
|
|
@ -474,36 +500,43 @@ for documentation.
|
|||
This file is read and written by the "sync" command, and only read by
|
||||
"fix" and "check".
|
||||
|
||||
blk_size SIZE
|
||||
Defines the size of the block in bytes. It must match the size
|
||||
defined in the configuration file.
|
||||
---- 9.0.1 blk_size SIZE ----
|
||||
Defines the size of the block in bytes. It must match the size
|
||||
defined in the configuration file.
|
||||
|
||||
checksum CHECKSUM
|
||||
Defines the checksum kind used. It can be "md5" or "murmur3".
|
||||
---- 9.0.2 checksum CHECKSUM ----
|
||||
Defines the checksum kind used. It can be "md5" or "murmur3".
|
||||
|
||||
file DISK SIZE TIME INODE PATH
|
||||
Defines a file in the specified DISK.
|
||||
The INODE number is used to identify the file in the "sync"
|
||||
command, allowing to rename or move the file in disk without
|
||||
the need to recompute the parity for it.
|
||||
The SIZE and TIME information are used to identify if the file
|
||||
changed from the last "sync" command, and if there is the need
|
||||
to recompute the parity.
|
||||
The PATH information is used in the "check" and "fix" commands
|
||||
to identify the file.
|
||||
---- 9.0.3 file DISK SIZE TIME INODE PATH ----
|
||||
Defines a file in the specified DISK.
|
||||
|
||||
blk BLOCK HASH
|
||||
Defines the ordered parity block list used by the last defined file.
|
||||
BLOCK is the block position in the "parity" file.
|
||||
0 for the first block, 1 for the second one and so on.
|
||||
HASH is the hash of the block. In the last block of the file,
|
||||
the HASH is the hash of only the used part of the block.
|
||||
The INODE number is used to identify the file in the "sync"
|
||||
command, allowing to rename or move the file in disk without
|
||||
the need to recompute the parity for it.
|
||||
|
||||
inv BLOCK [HASH]
|
||||
Like "blk", but inform that the parity of this block is invalid.
|
||||
The HASH may be missing if not yet computed.
|
||||
This field is used only when you interrupt manually the "sync"
|
||||
command.
|
||||
The SIZE and TIME information are used to identify if the file
|
||||
changed from the last "sync" command, and if there is the need
|
||||
to recompute the parity.
|
||||
|
||||
The PATH information is used in the "check" and "fix" commands
|
||||
to identify the file.
|
||||
|
||||
---- 9.0.4 blk BLOCK HASH ----
|
||||
Defines the ordered parity block list used by the last defined file.
|
||||
|
||||
BLOCK is the block position in the "parity" file.
|
||||
0 for the first block, 1 for the second one and so on.
|
||||
|
||||
HASH is the hash of the block. In the last block of the file,
|
||||
the HASH is the hash of only the used part of the block.
|
||||
|
||||
---- 9.0.5 inv BLOCK [HASH] ----
|
||||
Like "blk", but inform that the parity of this block is invalid.
|
||||
|
||||
The HASH may be missing if not yet computed.
|
||||
|
||||
This field is used only when you interrupt manually the "sync"
|
||||
command.
|
||||
|
||||
|
||||
10 PARITY
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue