snapraid/snapraid.1

.TH "SnapRAID Backup For Disk Arrays" 1
.SH NAME
snapraid \(hy SnapRAID Backup For Disk Arrays
.SH SYNOPSIS 
snapraid [\(hyc, \(hy\(hyconf CONFIG]
.PD 0
.PP
.PD
	[\(hyf, \(hy\(hyfilter PATTERN] [\(hyd, \(hy\(hyfilter\(hydisk NAME]
.PD 0
.PP
.PD
	[\(hym, \(hy\(hyfilter\(hymissing] [\(hye, \(hy\(hyfilter\(hyerror]
.PD 0
.PP
.PD
	[\(hya, \(hy\(hyaudit\(hyonly] [\(hyi, \(hy\(hyimport DIR]
.PD 0
.PP
.PD
	[\(hyp, \(hy\(hypercentage PERC] [\(hyo, \(hy\(hyolder\(hythan DAYS]
.PD 0
.PP
.PD
	[\(hyZ, \(hy\(hyforce\(hyzero] [\(hyE, \(hy\(hyforce\(hyempty]
.PD 0
.PP
.PD
	[\(hyU, \(hy\(hyforce\(hyuuid] [\(hyD, \(hy\(hyforce\(hydevice]
.PD 0
.PP
.PD
	[\(hyv, \(hy\(hyverbose] [\(hyl, \(hy\(hylog FILE]
.PD 0
.PP
.PD
	[\(hys, \(hy\(hystart BLKSTART] [\(hyt, \(hy\(hycount BLKCOUNT]
.PD 0
.PP
.PD
	sync|status|scrub|list|diff|dup|pool|check|fix|rehash
.PD 0
.PP
.PD
.PP
snapraid [\(hyV, \(hy\(hyversion] [\(hyh, \(hy\(hyhelp] [\(hyC, \(hy\(hygen\(hyconf CONTENT]
.PD 0
.PP
.PD
.SH DESCRIPTION 
SnapRAID is a backup program for disk arrays. It stores redundancy
information of your data and it\(cqs able to recover from up to six
disk failures.
.PP
SnapRAID is mainly targeted for a home media center, with a lot of
big files that rarely change.
.PP
Beside the ability to recover from disk failures, other
features of SnapRAID are:
.PD 0
.IP \(bu
All your data is hashed to ensure data integrity and to avoid
silent corruption.
.IP \(bu
If the failed disks are too many to allow a recovery,
you lose the data only on the failed disks.
All the data in the other disks is safe.
.IP \(bu
If you accidentally delete some files in a disk, you can
recover them.
.IP \(bu
You can start with already filled disks.
.IP \(bu
The disks can have different sizes.
.IP \(bu
You can add disks at any time.
.IP \(bu
It doesn\(cqt lock\(hyin your data. You can stop using SnapRAID at any
time without the need to reformat or move data.
.PD
.PP
The official site of SnapRAID is:
.PP
.RS 4
http://snapraid.sourceforge.net
.PD 0
.PP
.PD
.RE
.SH LIMITATIONS 
SnapRAID is in between a RAID and a Backup program trying to get the best
benefits of them. Although it also has some limitations that you should
consider before using it.
.PP
The main one is that if a disk fails, and you haven\(cqt recently synced,
you may not able to do a complete recover.
More specifically, you may be unable to recover up to the size of the
amount of the changed or deleted files from the last sync operation.
This happens even if the files changed or deleted are not in the
failed disk.
New added files don\(cqt prevent the recovering of the already existing
files. You may only lose the just added files, if they are on the failed
disk.
.PP
This is the reason because SnapRAID is better suited for data that
rarely change.
.PP
Other limitations are:
.PD 0
.IP \(bu
You have different file\(hysystems for each disk.
Using a RAID you have only a big file\(hysystem.
.IP \(bu
It doesn\(cqt stripe data.
With RAID you get a speed boost with striping.
.IP \(bu
It doesn\(cqt support real\(hytime recovery.
With RAID you do not have to stop working when a disk fails.
.IP \(bu
It\(cqs able to recover damages only from a limited number of disks.
With a Backup you are able to recover from a complete
failure of the whole disk array.
.IP \(bu
Only file, timestamps, symlinks and hardlinks are saved.
Permissions, ownership and extended attributes are not saved.
.PD
.SH GETTING STARTED 
To use SnapRAID you need to first select one disk of your disk array
to dedicate at the \(a"parity\(a" information. With one disk for parity you
will be able to recover from a single disk failure, like RAID5.
.PP
If you want to be able to recover from more disk failures, like RAID6,
you must reserve additional disks for parity. Any additional parity
disk allow to recover from one more disk failure.
.PP
As parity disks, you have to pick the biggest disks in the array,
as the redundancy information may grow in size as the biggest data
disk in the array.
.PP
These disks will be dedicated to store the \(a"parity\(a" files.
You should not store your data in them.
.PP
The list of files is saved in the \(a"content\(a" files, usually
stored in the data, parity or boot disks.
These files contain the details of your backup, with all the
checksums to verify its integrity.
The \(a"content\(a" file is stored in multiple copies, and each one must
be in a different disk, to ensure that in even in case of multiple
disk failures at least one copy is available.
.PP
For example, suppose that you are interested only at one parity level
of protection, and that your disks are present in:
.PP
.RS 4
/mnt/diskp <\(hy selected disk for parity
.PD 0
.PP
.PD
/mnt/disk1 <\(hy first disk to backup
.PD 0
.PP
.PD
/mnt/disk2 <\(hy second disk to backup
.PD 0
.PP
.PD
/mnt/disk3 <\(hy third disk to backup
.PD 0
.PP
.PD
.RE
.PP
you have to create the configuration file /etc/snapraid.conf with
the following options:
.PP
.RS 4
parity /mnt/diskp/parity
.PD 0
.PP
.PD
content /var/snapraid/content
.PD 0
.PP
.PD
content /mnt/disk1/content
.PD 0
.PP
.PD
content /mnt/disk2/content
.PD 0
.PP
.PD
disk d1 /mnt/disk1/
.PD 0
.PP
.PD
disk d2 /mnt/disk2/
.PD 0
.PP
.PD
disk d3 /mnt/disk3/
.PD 0
.PP
.PD
.RE
.PP
If you are in Windows, you should use drive letters and backslashes
instead of slashes, and if you like, also file extensions.
.PP
.RS 4
parity E:\(rsparity.par
.PD 0
.PP
.PD
content C:\(rssnapraid\(rscontent.lst
.PD 0
.PP
.PD
content F:\(rsarray\(rscontent.lst
.PD 0
.PP
.PD
content G:\(rsarray\(rscontent.lst
.PD 0
.PP
.PD
disk d1 F:\(rsarray\(rs
.PD 0
.PP
.PD
disk d2 G:\(rsarray\(rs
.PD 0
.PP
.PD
disk d3 H:\(rsarray\(rs
.PD 0
.PP
.PD
.RE
.PP
At this point you are ready to start the \(a"sync\(a" command to build the
redundancy information.
.PP
.RS 4
snapraid sync
.PD 0
.PP
.PD
.RE
.PP
This process may take some hours the first time, depending on the size
of the data already present in the disks. If the disks are empty
the process is immediate.
.PP
You can stop it at any time pressing Ctrl+C, and at the next run it
will start where interrupted.
.PP
When this command completes, your data is SAFE.
.PP
At this point you can start using your array as you like, and periodically
update the redundancy information running the \(a"sync\(a" command.
.SS Checking & Fixing 
To check the integrity of your data you can use the \(a"check\(a" command:
.PP
.RS 4
snapraid check
.PD 0
.PP
.PD
.RE
.PP
If will read all your data, to check if it\(cqs correct.
.PP
If an error is found, you can use the \(a"fix\(a" command to fix it.
.PP
.RS 4
snapraid fix
.PD 0
.PP
.PD
.RE
.PP
Note that the fix command will revert your data at the state of the
last \(a"sync\(a" command executed. It works like a snapshot was taken
in \(a"sync\(a".
.SS Recovering and Undeleting 
In recovering SnapRAID is more like a backup program than a RAID system,
and it can be used to restore or undelete only a single file or directory
to its previous state using the \(hyf, \(hy\(hyfilter option :
.PP
.RS 4
snapraid fix \(hyf FILE
.PD 0
.PP
.PD
.RE
.PP
or for a directory:
.PP
.RS 4
snapraid fix \(hyf DIR/
.PD 0
.PP
.PD
.RE
.PP
You can also use it to recover only accidentally deleted files inside
a directory using the \(hym, \(hy\(hyfilter\(hymissing option, that restores
only missing files, leaving untouched all the others.
.PP
.RS 4
snapraid fix \(hym \(hyf DIR/
.PD 0
.PP
.PD
.RE
.PP
Or to recover all the deleted files in all the drives with:
.PP
.RS 4
snapraid fix \(hym
.PD 0
.PP
.PD
.RE
.SS Scrubbing 
To periodically check the old data for errors, you can run the \(a"scrub\(a"
command.
.PP
.RS 4
snapraid scrub
.PD 0
.PP
.PD
.RE
.PP
This command is similar at \(a"check\(a" but verifies only the oldest data
in your array. Every run of the command checks about 12% of the data,
but nothing newer than 10 days.
You can use the \(hyp, \(hy\(hypercentage option to specify a different amount,
and the \(hyo, \(hy\(hyolder\(hythan option to specify a different age in days.
For example, to check 5% of the array older than 20 days use:
.PP
.RS 4
snapraid \(hyp 5 \(hyo 20 scrub
.PD 0
.PP
.PD
.RE
.PP
If silent errors are found, the corresponding blocks are marked as bad
in the \(a"content\(a" file, and listed in the \(a"status\(a" command.
.PP
.RS 4
snapraid status
.PD 0
.PP
.PD
.RE
.PP
To fix them, you can use the \(a"fix\(a" command filtering for files
containing bad blocks:
.PP
.RS 4
snapraid \(hye fix
.PD 0
.PP
.PD
.RE
.PP
At the next \(a"scrub\(a" the errors will disappear from the \(a"status\(a" report
if really fixed. You can use \(hyp 0 to scrub only blocks marked as bad.
.PP
.RS 4
snapraid \(hyp 0 scrub
.PD 0
.PP
.PD
.RE
.SS Pooling 
To have all the files in your array shown in the same directory tree,
you can enable \(a"pooling\(a", that consists in creating a virtual view of all
the files in your array using symbolic links.
You can configure the \(a"pooling\(a" directory in the configuration file with:
.PP
.RS 4
pool /pool
.PD 0
.PP
.PD
.RE
.PP
or, if you are in Windows, with:
.PP
.RS 4
pool C:\(rspool
.PD 0
.PP
.PD
.RE
.PP
and then run the \(a"pool\(a" command.
.PP
.RS 4
snapraid pool
.PD 0
.PP
.PD
.RE
.PP
If you are using a Unix platform and you want to configure SAMBA to
share such directory, you should add to your /etc/samba/smb.conf the
following options:
.PP
.RS 4
# In the global section of smb.conf
.PD 0
.PP
.PD
unix extensions = no
.PD 0
.PP
.PD
.RE
.PP
.RS 4
# In the share section of smb.conf
.PD 0
.PP
.PD
[pool]
.PD 0
.PP
.PD
comment = Pool
.PD 0
.PP
.PD
path = /pool
.PD 0
.PP
.PD
read only = yes
.PD 0
.PP
.PD
guest ok = yes
.PD 0
.PP
.PD
wide links = yes
.PD 0
.PP
.PD
.RE
.SH COMMANDS 
SnapRAID provides some simple commands that allow to:
.PD 0
.IP \(bu
Make a backup/snapshot \(hy> \(a"sync\(a"
.IP \(bu
Periodically checks old data \(hy> \(a"scrub\(a"
.IP \(bu
Prints a report of the status of the array \(hy> \(a"status\(a"
.IP \(bu
Check for integrity the full array \(hy> \(a"check\(a"
.IP \(bu
Restore the last backup/snapshot \(hy> \(a"fix\(a".
.PD
.PP
Take care that the commands have to be written in lower case.
.SS sync 
Updates the redundancy information. All the modified files
in the disk array are read, and the redundancy data is
recomputed.
.PP
Files are identified by path and/or inode and checked by
size and timestamp.
If the size or timestamp are different, the redundancy data is
recomputed for the whole file.
Using inode allow you can move them on the disk
without triggering any redundancy recomputation.
.PP
You can stop this process at any time pressing Ctrl+C,
without losing the work already done.
.PP
The \(a"content\(a", \(a"parity\(a" files are modified if necessary.
The files in the array are NOT modified.
.SS check 
Checks all the files and the redundancy data.
All the files are hashed and compared with the snapshot saved
in the previous \(a"sync\(a" command.
.PP
If an error if found, a recovery attempt is simulated to check
if the error is a recoverable one or not.
.PP
If you use the \(hya, \(hy\(hyaudit\(hyonly option, only the file
data is checked, and the redundandy data is ignored.
.PP
Files are identified by path, and checked by content.
.PP
Nothing is modified.
.SS fix 
Checks and fix all the files. It\(cqs like \(a"check\(a" but it also fixes
errors reverting the state of the disk array to the previous \(a"sync\(a"
command.
.PP
After a successful \(a"fix\(a", you should also run a \(a"sync\(a" command to
update the new state of the files.
.PP
All the files that cannot be fixed are renamed adding
the \(a".unrecoverable\(a" extension.
.PP
The \(a"content\(a" file is NOT modified.
The \(a"parity\(a" files are modified if necessary.
The files in the array are modified if necessary.
.SS scrub 
Scrubs the array, checking for silent errors.
.PP
For each command invocation, the 12% of the array is checked, but
nothing that it\(cqs more recent than 10 days.
This means that scrubbing once a week, every bit of data is checked
at least one time every two months.
.PP
You can use the \(hyp, \(hy\(hypercentage option to specify a different amount,
and the \(hyo, \(hy\(hyolder\(hythan option to specify a different age in days.
Note that if only one of \(hyp and \(hyo is specified the default value of
the other option is not used.
.PP
Any silent error identified is recorded in the content file,
and it\(cqs listed in the \(a"status\(a" command until it\(cqs fixed calling
\(a"fix\(a" and then \(a"scrub\(a".
.PP
The oldest blocks are scrubbed first ensuring an optimal check.
Blocks already marked as bad are always checked, and if found
correct, they are automatically unmarked.
.PP
It\(cqs recommended to run \(a"scrub\(a" on a synched array, to avoid to have
reported error caused by unsynched data. These errors are recognized
as not being silent errors, and the blocks are not marked as bad,
but such errros are reported in the output of the command.
.PP
The \(a"content\(a" file is modified to update the time of the last check
of each block.
The \(a"parity\(a" files are NOT modified.
The files in the array are NOT modified.
.SS status 
Prints a summary of the state of the disk array.
.PP
It includes information about the parity fragmentation, how old
are the blocks without checking, and all the recorded silent
errors encoutered while scrubbing.
.PP
Nothing is modified.
.SS list 
Lists all the files contained in the array at the time of the
last \(a"sync\(a" command.
.PP
Nothing is modified.
.SS diff 
Lists all the files modified from the last \(a"sync\(a" command that
have to recompute their redundancy data.
.PP
This command doesn\(cqt check the file data, but only the file timestamp
size and inode.
.PP
Nothing is modified.
.SS dup 
Lists all the duplicate files. Two files are assumed equal if their
hashes are matching. The file data is not read, but only the
precomputed hashes are used.
.PP
Nothing is modified.
.SS pool 
Creates or updates in the \(a"pooling\(a" directory a virtual view of all
the files of your disk array.
.PP
The files are not really copied here, but just linked using
symbolic links.
.PP
When updating, all the present symbolic links and empty
subdirectories are deleted and replaced with the new
view of the array. Any othe regular file is left in place.
.PP
Nothing is modified outside the pool directory.
.SS rehash 
Schedules a rehash of the whole array.
.PP
This option can be used to change the hash kind used,
typically when upgrading from a 32 bits system to a 64
bits one to switch from MurmurHash3 to the faster SpookyHash.
.PP
If you are already using the optimal hash, this command
do nothing and just inform you that nothing has to be done.
.PP
The rehash isn\(cqt done immediately, but it takes place
progressively during the \(a"sync\(a" and \(a"scrub\(a" commands.
.PP
You can get the rehash state using the \(a"status\(a" command.
.PP
During the rehash, SnapRAID maintains full functionality,
with the only expection of the \(a"dup\(a" command not able to detect
duplicated files using a different hash.
.SH OPTIONS 
SnapRAID provides the following options:
.TP
.B \(hyc, \(hy\(hyconf CONFIG
Selects the configuration file. If not specified it\(cqs assumed
the file \(a"/etc/snapraid.conf\(a" in Unix, and \(a"snapraid.conf\(a" in
the current directory in Windows.
.TP
.B \(hyf, \(hy\(hyfilter PATTERN
Filters the files to process in the \(a"check\(a" and \(a"fix\(a"
commands.
Only the files matching the entered pattern are processed.
This option can be used many times.
See the PATTERN section for more details in the
pattern specifications.
In Unix, ensure to quote globbing chars if used.
This option can be used only with the \(a"check\(a" and \(a"fix\(a" commands.
Note that it cannot be used with \(a"sync\(a", because \(a"sync\(a" always
process the whole array.
.TP
.B \(hyd, \(hy\(hyfilter\(hydisk NAME
Filters the files to process in the \(a"check\(a" and \(a"fix\(a"
commands.
Only the files present in the specified disk are processed.
You must specify a disk name as named in the configuration
file.
In \(a"check\(a", you can make it faster, specifing also \(hya, \(hy\(hyaudit\(hyonly
option, to avoid to access other disks to check parity data.
If you combine more \(hy\(hyfilter, \(hy\(hyfilter\(hydisk and \(hy\(hyfilter\(hymissing options,
only files matching all the set of filters are selected.
This option can be used many times.
This option can be used only with the \(a"check\(a" and \(a"fix\(a" commands.
Note that it cannot be used with \(a"sync\(a", because \(a"sync\(a" always
process the whole array.
.TP
.B \(hym, \(hy\(hyfilter\(hymissing
Filters the files to process in the \(a"check\(a" and \(a"fix\(a"
commands.
Only the files missing/deleted from the array are processed.
When used with \(a"fix\(a", this is a kind of \(a"undelete\(a" command.
If you combine more \(hy\(hyfilter, \(hy\(hyfilter\(hydisk and \(hy\(hyfilter\(hymissing options,
only files matching all the set of filters are selected.
This option can be used only with the \(a"check\(a" and \(a"fix\(a" commands.
Note that it cannot be used with \(a"sync\(a", because \(a"sync\(a" always
process the whole array.
.TP
.B \(hye, \(hy\(hyfilter\(hyerror
Filters the files to process in the \(a"check\(a" and \(a"fix\(a"
commands.
It process only the files containing blocks marked with silent
errors during the \(a"sync\(a" or \(a"scrub\(a" command, and listed in the
\(a"status\(a" command.
Errors found in \(a"check\(a" are not processed by this
option, because they are not marked as bad as \(a"check\(a" is a
read\(hyonly command.
This option can be used only with the \(a"check\(a" and \(a"fix\(a" commands.
.TP
.B \(hyp, \(hy\(hypercentage PERC
Selects the part of the array to process in the \(a"scrub\(a" command.
PERC is a numeric value from 0 to 100, default is 12.
When specifing 0, only the blocks marked as bad are scrubbed.
This option can be used only with the \(a"scrub\(a" command.
.TP
.B \(hyo, \(hy\(hyolder\(hythan DAYS
Selects the older the part of the array to process in the
\(a"scrub\(a" command.
DAYS is the minimum age in days for a block to be scrubbed,
default is 10.
Blocks marked as bad are always scrubbed despite this option.
This option can be used only with the \(a"scrub\(a" command.
.TP
.B \(hya, \(hy\(hyaudit\(hyonly
When checking, only verify the hash of the files, without
doing any kind of check on the redundancy data.
If you are interested in checking only the file data this
option can speedup a lot the checking process.
This option can be used only with the \(a"check\(a" command.
.TP
.B \(hyi, \(hy\(hyimport DIR
When fixing imports from the specified directory any file
that you deleted from the array after the last \(a"sync\(a"
commmand.
If you still have such files, they could be used by the \(a"fix\(a"
command to improve the recover process.
The files are read also in subdirectories and they are
identified regardless of their name.
This option can be used only with the \(a"check\(a" and \(a"fix\(a" command.
.TP
.B \(hyZ, \(hy\(hyforce\(hyzero
Forces the insecure operation of syncing a file with zero
size that before was not.
If SnapRAID detects a such condition, it stops proceeding
unless you specify this option.
This allows to easily detect when after a system crash,
some accessed files were zeroed.
This is a possible condition in Linux with the ext3/ext4
filesystems.
This option can be used only with the \(a"sync\(a" command.
.TP
.B \(hyE, \(hy\(hyforce\(hyempty
Forces the insecure operation of syncing a disk with all
the original files missing.
If SnapRAID detects that all the files originally present
in the disk are missing or rewritten, it stops proceeding
unless you specify this option.
This allows to easily detect when a data file\(hysystem is not
mounted.
This option can be used only with the \(a"sync\(a" command.
.TP
.B \(hyU, \(hy\(hyforce\(hyuuid
Forces the insecure operation of syncing, checking and fixing
with disks that have changed their UUID.
If SnapRAID detects that some disks have changed UUID,
it stops proceeding unless you specify this option.
This allows to detect when your disks are mounted in the
wrong mount points.
It\(cqs anyway allowed to have a single UUID change with
single parity, and more with multiple parity, because it\(cqs
the normal case of replacing disks after a recover.
This option can be used only with the \(a"sync\(a", \(a"check\(a" or
\(a"fix\(a" command.
.TP
.B \(hyD, \(hy\(hyforce\(hydevice
Forces the insecure operation of fixing with disks on the same
physical device.
If SnapRAID detects that some disks have the same device ID,
it stops proceeding, because it\(cqs not a supported configuration.
But it could happen that you want to temporarely restore a lost
disk in the free space left in an already used disk. and this
option allows you to continue anyway.
.TP
.B \(hyl, \(hy\(hylog FILE
Write a detailed log of errors found in check and fix.
This log contains the exact specification of which block of
any file is not recoverable and why.
If this option is not specified, no detailed log is printed,
and you\(cqll get only a summary at the end of the operations.
When checking and fixing this allows to keep separated
the possible huge list of errors from the human readable
output.
.TP
.B \(hys, \(hy\(hystart BLKSTART
Starts the processing from the specified
block number. It could be useful to retry to check
or fix some specific block, in case of a damaged disk.
It\(cqs present mainly for advanced manual recovering.
.TP
.B \(hyt, \(hy\(hycount BLKCOUNT
Processes only the specified number of blocks.
It\(cqs present mainly for advanced manual recovering.
.TP
.B \(hyC, \(hy\(hygen\(hyconf CONTENT_FILE
Generates a dummy configuration file from an existing
content file.
The configuration file is written in the standard output,
and it doesn\(cqt overwrite an existing one.
This configuration file also contains the information
needed to reconstruct the disk mount points, in case you
lose the entire system.
.TP
.B \(hyv, \(hy\(hyverbose
Prints more information in the processing.
.TP
.B \(hyh, \(hy\(hyhelp
Prints a short help screen.
.TP
.B \(hyV, \(hy\(hyversion
Prints the program version.
.SH CONFIGURATION 
SnapRAID requires a configuration file to know where your disk array
is located, and where storing the redundancy information.
.PP
This configuration file is located in /etc/snapraid.conf in Unix or
in the execution directory in Windows.
.PP
It should contain the following options (case sensitive):
.SS parity FILE 
Defines the file to use to store the parity information.
The parity enables the protection from a single disk
failure, like RAID5.
.PP
It must be placed in a disk dedicated for this purpose with
as much free space as the biggest disk in the array.
Leaving the parity disk reserved for only this file ensures that
it doesn\(cqt get fragmented, improving the performance.
.PP
This option is mandatory and it can be used only one time.
.SS [q,r,s,t,u]\(hyparity FILE 
Defines the files to use to store extra parity information.
For each parity file specified, one additional level of protection
is enabled:
.PD 0
.IP \(bu
q\(hyparity enables RAID6 double parity
.IP \(bu
r\(hyparity enables triple parity
.IP \(bu
s\(hyparity enables quad parity
.IP \(bu
t\(hyparity enables penta (five) parity
.IP \(bu
u\(hyparity enables hexa (six) parity
.PD
.PP
Each parity level requires also all the files of the previous levels.
.PP
Each file must be placed in a disk dedicated for this purpose with
as much free space as the biggest disk in the array.
Leaving the parity disks reserved for only these files ensures that
they doesn\(cqt get fragmented, improving the performance.
.PP
These options are optional and they can be used only one time.
.SS z\(hyparity FILE 
Defines an alternate file and format to store the triple parity,
instead of the other \(cqr\(hyparity\(cq one.
.PP
It\(cqs mainly intended for low end CPUs like ARM or AMD Phenom, Athlon
and Opteron that don\(cqt support the SSSE3 instructions set, and in
such case it may provide better performance.
.PP
This is also the parity format used by ZFS, but it doesn\(cqt work
for beyond triple parity.
.PP
If you are using the \(cqr\(hyparity\(cq you will be warned if it\(cqs
recommended to use the \(cqz\(hyparity\(cq format for a performance
increase.
.PP
It\(cqs possible to convert from one format to another, adjusting
the configuraton file with the wanted z\(hyparity or r\(hyparity file,
and using \(cqfix\(cq to recreate it.
.PP
This option is alternative to \(cqr\(hyparity\(cq.
.SS content FILE 
Defines the file to use to store the list and checksums of all the
files present in your disk array.
.PP
It can be placed in the disk used to store data, parity, or
any other disk available.
If you use a data disk, this file is automatically excluded
from the \(a"sync\(a" process.
.PP
This option is mandatory and it can be used more time to save
more copies of the same files.
.PP
You have to store at least one copy for each parity disk used
plus one. Using some more don\(cqt hurt.
.SS disk NAME DIR 
Defines the name and the mount point of the disks of the array.
NAME is used to identify the disk, and it must be unique.
DIR is the mount point of the disk in the filesystem.
.PP
You can change the mount point as you like, as long you
keep the NAME fixed.
.PP
You should use one option for each disk of the array.
.SS nohidden 
Excludes all the hidden files and directory.
In Unix hidden files are the ones starting with \(a".\(a".
In Windows they are the ones with the hidden attribute.
.SS exclude/include PATTERN 
Defines the file or directory patterns to exclude and include
in the sync process.
All the patterns are processed in the specified order.
.PP
If the first pattern that matches is an \(a"exclude\(a" one, the file
is excluded. If it\(cqs an \(a"include\(a" one, the file is included.
If no pattern matches, the file is excluded if the last pattern
specified is an \(a"include\(a", or included if the last pattern
specified is an \(a"exclude\(a".
.PP
See the PATTERN section for more details in the pattern
specifications.
.PP
This option can be used many times.
.SS block_size SIZE_IN_KIBIBYTES 
Defines the basic block size in kibi bytes of the redundancy
blocks. Where one kibi bytes is 1024 bytes.
The default is 256 and it should work for most conditions.
You could increase this value if you do not have enough RAM
memory to run SnapRAID.
.PP
As a rule of thumb, with 4 GiB or more memory use the default 256,
with 2 GiB use 512, and with 1 GiB use 1024.
.PP
In more details SnapRAID requires about TS*28/BS bytes
of RAM memory to run. Where TS is the total size in bytes of
your disk array, and BS is the block size in bytes.
.PP
For example with 4 disk of 3 TiB and a block size of 256 KiB
(1 KiB = 1024 Bytes) you have:
.PP
RAM = (4 * 3 * 2^40) * 28 / (256 * 2^10) = 1.4 GiB
.PD 0
.PP
.PD
.PP
You could instead decrease this value if you have a lot of
small files in the disk array. For each file, even if of few
bytes, a whole block is always allocated, so you may have a lot
of unused space.
As approximation, you can assume that half of the block size is
wasted for each file.
.PP
For example, with 10000 files and a 256 KiB block size, you are
going to waste 1.2 GiB.
.SS autosave SIZE_IN_GIBIBYTES 
Automatically save the state when synching after the specied amount
of GiB processed.
This option is useful to avoid to restart from scratch long \(a"sync\(a"
commands interrupted by a machine crash, or any other event that
may interrupt SnapRAID.
The SIZE argument is specified in gibibytes. Where one gibi bytes
is 1073741824 bytes.
.SS pool DIR 
Defines the pooling directory where the virtual view of the disk
array is created using the \(a"pool\(a" command.
The directory must already exist.
.SS Examples 
An example of a typical configuration for Unix is:
.PP
.RS 4
parity /mnt/diskp/parity
.PD 0
.PP
.PD
content /mnt/diskp/content
.PD 0
.PP
.PD
content /var/snapraid/content
.PD 0
.PP
.PD
disk d1 /mnt/disk1/
.PD 0
.PP
.PD
disk d2 /mnt/disk2/
.PD 0
.PP
.PD
disk d3 /mnt/disk3/
.PD 0
.PP
.PD
exclude /lost+found/
.PD 0
.PP
.PD
exclude /tmp/
.PD 0
.PP
.PD
.RE
.PP
An example of a typical configuration for Windows is:
.PP
.RS 4
parity E:\(rsparity.par
.PD 0
.PP
.PD
content E:\(rscontent.lst
.PD 0
.PP
.PD
content C:\(rssnapraid\(rscontent.lst
.PD 0
.PP
.PD
disk d1 G:\(rsarray\(rs
.PD 0
.PP
.PD
disk d2 H:\(rsarray\(rs
.PD 0
.PP
.PD
disk d3 I:\(rsarray\(rs
.PD 0
.PP
.PD
exclude Thumbs.db
.PD 0
.PP
.PD
exclude \(rs$RECYCLE.BIN
.PD 0
.PP
.PD
exclude \(rsSystem Volume Information
.PD 0
.PP
.PD
.RE
.SH PATTERN 
Patterns are used to select a subset of files to exclude or include in
the process.
.PP
There are four different types of patterns:
.TP
.B FILE
Selects any file named as FILE. You can use any globbing
character like * and ?.
This pattern is applied only to files and not to directories.
.TP
.B DIR/
Selects any directory named DIR. You can use any globbing
character like * and ?.
This pattern is applied only to directories and not to files.
.TP
.B /PATH/FILE
Selects the exact specified file path. You can use any
globbing character like * and ? but they never match a
directory slash.
This pattern is applied only to files and not to directories.
.TP
.B /PATH/DIR/
Selects the exact specified directory path. You can use any
globbing character like * and ? but they never match a
directory slash.
This pattern is applied only to directories and not to files.
.PP
In Windows you can freely use the backslash \(rs instead of the forward slash /.
.PP
Note that Windows system directories, junction to directories,
mount points, and any other Windows special directory is treated just
as a file, meaning that to exclude it you must use a file rule, and
not a directory one.
.PP
In the configuration file, you can use different strategies to filter
the files to process.
The simplest one is to use only \(a"exclude\(a" rules to remove all the
files and directories you do not want to process. For example:
.PP
.RS 4
# Excludes any file named \(a"*.unrecoverable\(a"
.PD 0
.PP
.PD
exclude *.unrecoverable
.PD 0
.PP
.PD
# Excludes the root directory \(a"/lost+found\(a"
.PD 0
.PP
.PD
exclude /lost+found/
.PD 0
.PP
.PD
# Excludes any sub\(hydirectory named \(a"tmp\(a"
.PD 0
.PP
.PD
exclude tmp/
.PD 0
.PP
.PD
.RE
.PP
The opposite way is to define only the file you want to process, using
only \(a"include\(a" rules. For example:
.PP
.RS 4
# Includes only some directories
.PD 0
.PP
.PD
include /movies/
.PD 0
.PP
.PD
include /musics/
.PD 0
.PP
.PD
include /pictures/
.PD 0
.PP
.PD
.RE
.PP
The final way, is to mix \(a"exclude\(a" and \(a"include\(a" rules. In this case take
care that the order of rules is important. Previous rules have the
precedence over the later ones.
To get things simpler you can first have all the \(a"exclude\(a" rules and then
all the \(a"include\(a" ones. For example:
.PP
.RS 4
# Excludes any file named \(a"*.unrecoverable\(a"
.PD 0
.PP
.PD
exclude *.unrecoverable
.PD 0
.PP
.PD
# Excludes any sub\(hydirectory named \(a"tmp\(a"
.PD 0
.PP
.PD
exclude tmp/
.PD 0
.PP
.PD
# Includes only some directories
.PD 0
.PP
.PD
include /movies/
.PD 0
.PP
.PD
include /musics/
.PD 0
.PP
.PD
include /pictures/
.PD 0
.PP
.PD
.RE
.PP
On the command line, using the \(hyf option, you can only use \(a"include\(a"
patterns. For example:
.PP
.RS 4
# Checks only the .mp3 files.
.PD 0
.PP
.PD
# Note the \(a"\(a" use to avoid globbing expansion by the shell in Unix.
.PD 0
.PP
.PD
snapraid \(hyf \(a"*.mp3\(a" check
.PD 0
.PP
.PD
.RE
.PP
In Unix, when using globbing chars in the command line, you have to quote them.
Otherwise the shell will try to expand them.
.SH RECOVERING 
The worst happened, and you lost a disk!
.PP
DO NOT PANIC! You will be able to recover it!
.PP
The first thing you have to do is to avoid futher changes at you disk array.
Disable any remote connection to it, any scheduled process, including any
scheduled SnapRAID nightly sync.
.PP
Then proceed with the following steps.
.SS STEP 1 \(hy> Reconfigure 
You need some space to recover, even better if you already have an additional
disk, but in case, also an external USB or remote one is enough.
.PP
Change the SnapRAID configuration file and make the \(a"disk\(a" option
of the failed disk to point to a place where you have enough empty space
to recover the files.
.PP
For example, if you have that disk \(a"d1\(a" failed, you can change:
.PP
.RS 4
disk d1 /mnt/disk1/
.PD 0
.PP
.PD
.RE
.PP
to:
.PP
.RS 4
disk d1 /mnt/new_spare_disk/
.PD 0
.PP
.PD
.RE
.SS STEP 2 \(hy> Fix 
Run the fix command, storing the log in an external file with:
.PP
.RS 4
snapraid \(hyd NAME \(hyl fix.log fix
.PD 0
.PP
.PD
.RE
.PP
Where NAME is the name of the disk, like \(a"d1\(a" as in our previous example.
.PP
This command will take a long time.
.PP
Take care that you need also few gigabytes free to store the fix.log file.
Run it from a disk with some free space.
.PP
Now you have recovered all the recoverable. If some file is partially or totally
unrecoverable, it will be renamed adding the \(a".unrecoverable\(a" extension.
.PP
You can get a detailed list of all the unrecoverable blocks in the fix.log file
checking all the lines starting with \(a"unrecoverable:\(a"
.PP
If you are not satified of the recovering, you can retry it as many time you wish.
For example, if you have moved away some files from other disks after the last \(a"sync\(a",
you can retry to put them inplace, and retry the \(a"fix\(a".
.PP
If you are satisfied of the recovering, you can now proceed further,
but take care that after synching you will no more able to retry the
\(a"fix\(a" command!
.SS STEP 3 \(hy> Check 
As paranoid but recommended check, you can now run a \(a"check\(a" command to ensure
that everything is OK on the disk.
.PP
.RS 4
snapraid \(hyd NAME \(hya check
.PD 0
.PP
.PD
.RE
.PP
Where NAME is the name of the disk, like \(a"d1\(a" as in our previous example.
.PP
The options \(hyd and \(hya tell SnapRAID to check only the specified disk,
and ignore all the redundancy data.
.PP
This command will take a long time.
.SS STEP 4 \(hy> Sync 
Run the \(a"sync\(a" command to resyncronize the array with the new disk.
.PP
.RS 4
snapraid sync
.PD 0
.PP
.PD
.RE
.PP
If everything was recovered, this command is immediate.
.SH CONTENT 
SnapRAID stores the list and checksums of your files in the content file.
.PP
It\(cqs a binary file, listing all the files present in your disk array,
with all the checksums to verify their integrity.
.PP
You do not need to understand its format to use SnapRAID.
.PP
This file is read and written by the \(a"sync\(a" and \(a"scrub\(a" commands, and
only read by \(a"fix\(a", \(a"check\(a" and \(a"status\(a".
.SH PARITY 
SnapRAID stores the redundancy information of your array in the parity
files.
.PP
They are binary files, containing the computed redundancy of all the
blocks defined in the \(a"content\(a" file.
.PP
These files are read and written by the \(a"sync\(a" and \(a"fix\(a" commands, and
only read by \(a"scrub\(a" and \(a"check\(a".
.SH ENCODING 
SnapRAID in Unix ignores any encoding. It simply reads and stores the
file names with the same encoding used by the filesystem.
.PP
In Windows all the names read from the filesystem are converted and
processed in the UTF\(hy8 format.
.PP
To have the file names printed correctly you have to set the Windows
console in the UTF\(hy8 mode, with the command \(a"chcp 65001\(a", and use
a TrueType font like \(a"Lucida Console\(a" as console font.
Note that it has effect only on the printed file names, if you
redirect the console output to a file, the resulting file is always
in the UTF\(hy8 format.
.SH COPYRIGHT 
This file is Copyright (C) 2011 Andrea Mazzoleni
.SH SEE ALSO 
rsync(1)