Frequently Asked Questions for SnapRAID.

General

Setup

Configuration

Use

Recover

Tech

Performance

Know Problems

General

What's SnapRAID?

SnapRAID is an application able to make a partial backup of your disk array. If some of the disks of your array fail, even if they are completely broken, you will be able to recover their content. It's only a partial backup, because it doesn't allow to recover from a failure of the whole array, but only if the number of failed disks are under a predefined limit.

How is it different than RAID?

RAIDs are mirroring solutions that propagate changes immediately to the parity data. In this sense, RAIDs are not backups, because they don't allow to restore an old state of the array.

SnapRAID is instead more similar at a backup solution, and you can restore the state of the last backup. For example, with SnapRAID, if you delete by accident a file, you can easily restore it.

Others advantages over traditional RAID5/6 solutions are:

Obviously RAID has other advantages, like having a better speed due data striping. What is best for you, depends on your specific use case.

How is it different than ZFS/Btrfs snapshots and NTFS Shadow Copy?

SnapRAID creates a filesystem snapshot conceptually similar at other snapshot solutions. The difference is that the purpose of SnapRAID is to be able to recover a disk to the saved state after a complete disk failure.

SnapRAID is more similar at the RAID-Z/RAID functionality of ZFS/Btrfs. Compared to them, SnapRAID has the following advantages:

Obviously ZFS/Btrfs have other advantages, like being real-time solutions. What is best for you, depends on your specific use case.

How is it different than other backup solutions?

With a normal backup solution you always need as much space as the full disk array for the backup. With SnapRAID you need only one additional parity disk to be able to recover the most common case of a single disk failure. Adding more parity disks, you can recover from more disk failures.

Note that only a full backup allows to recover from a complete failure of the disk array and it's surely the preferred solution if you can afford it. SnapRAID is a good option if a full backup is not a possible solution.

What are the advantages of SnapRAID compared to similar solutions like unRAID/FlexRAID/Windows Storage Spaces?

You can see the Compare page to see it compared to other similar solutions.

A strong advantage of SnapRAID is the integrity and silent error management that is at the same level (if not better) of the ZFS and Btrfs filesystems. Instead, unRAID and Storage Space have no integrity check at all! FlexRAID is at least able to report silent errors, but not to fix them.

How much does it cost?

Nothing. It's a FreeSoftware application released with the GPLv3 license.

How to get support?

Go to the Forum.

Has SnapRAID a GUI?

No. SnapRAID is a command line application. But there is the Elucidate external GUI, and a plugin for OpenMediaVault.

Does SnapRAID provide encryption, virtual views/storage pooling, SMART monitoring and Power control?

SnapRAID provides a basic virtual views/storage pooling solution, but it's also compatible with any other pooling solutions.

For encryption, SMART monitoring, Power control and Data Recovery you can use other tools. Here some suggestions:

LinuxWindowsMac OS X
Encryption dmcrypt/LUKS
(tutorial)
FreeOTFE
Virtual views/storage pooling mergerfs (tutorial), mhddfs (tutorial), aufs, Greyhole Liquesce (free, BETA), DrivePool ($$), PoolHD ($$), DriveBender ($$)
SMART monitor smartmontools, GSmartControl smartmontools, GSmartControl, CrystalDiskInfo smartmontools, GSmartControl, SMARTReporter
Power Control hdparm hdparm
Data recovering ddrescue, safecopy, dd_rescue

Is SnapRAID suitable to backup the home/My Documents directory?

No. If you have a limited amount of data that can be easily saved in an external USB HD, or in online backup services, it doesn't make sense to use SnapRAID. SnapRAID should be used when you have to backup a very large amount of data, many Tera bytes, and a full backup copy of a such amount of data is not possible in any other way.

Is SnapRAID suitable to backup the boot OS disk?

No. For the disk used for booting your machine and containing the OS files, it's better to use another solution, like RAID1 or a mirror copy. SnapRAID needs an OS to run, and then it cannot recover the OS if the OS is missing.

Can SnapRAID hash files wihout parity computation?

No. It's not possible to use SnapRAID to only hash files. The hashing used by SnapRAID is meant to be used in combination of parity computation, and it's not the best implementation for a hash only solution.

You can find a list of file verification tools at: Wikipedia: Comparison_of_file_verification_software.

Setup

For a setup example in Ubuntu 14.04 you can check the SnapRAID Zack's tutorial.

Do I need one or more parity disks?

As a rule of thumb you can stay with one parity disk (RAID5) with up to four data disks, and then using one parity disk for each group of seven data disks, like in the table:

ParitiesData disks
1/Single Parity/RAID5 2 - 4
2/Double Parity/RAID6 5 - 14
3/Triple Parity 15 - 21
4/Quad Parity 22 - 28
5/Penta Parity 29 - 35
6/Hexa Parity 36 - 42

This table takes into account that more parity levels also help to recover from unsynced array, and that likely you are running all the disks in the same box and environment, meaning that their failures could be correlated.

Take care that multiple failures happen. At now the worst case reported in the SnapRAID history, is a four disks failure, succesfully recovered using four parities.

Some interesting articles about number of parity levels and failure rate of disks are: Why RAID 5 stops working in 2009, Why RAID 6 stops working in 2019, Does RAID 6 stop working in 2019?, Triple-Parity RAID and Beyond, How long do disk drives last?.

Which filesystem is recommended for SnapRAID ?

For the data disks, you can use whatever filesystem you like. For the parity disks, follow these suggestions:

FileSystemResult
ext4 OK
ext3 No. It doesn't support the fallocate() command needed to allocated the parity files. You can anyway use it if you limit the use of the parity disk to contain only the parity file.
XFS OK
JFS No. It doesn't support the fallocate() command needed to allocated the parity files.
It also doesn't seem to implement at the best the posix_fadvise() command.
ReiserFS Unsure. Where are user reports reporting speed increase when switching from ReiserFS to other filesystem.
NTFS OK in Windows. No in Linux, as it doesn't support fallocate() command needed to allocated the parity files. You can anyway use it if you limit the use of the parity disk to contain only the parity file.
FAT No. You cannot store files bigger than 4 GB. It's also not journaled. Use NTFS instead.
HFS+ OK

In Linux, to get more space for the parity, it's recommended to format the parity filesystem with the -m 0 -T largefile4 options. Like:

mkfs.ext4 -m 0 -T largefile4 DEVICE

Which hardware configuration is recommended?

For best performance it's recommended to have all the disks connected with SATA and not with USB.

Check Raj's Prototype Builds for some examples of hardware setups.

For choosing harddisk, you can check Backblaze's What Hard Drive Should I Buy?, and Backblaze's Enterprise Drives: Fact or Fiction?.

Is a multi-core CPU recommended?

No. SnapRAID is usually not CPU bound. The limiting factor is the read performance from disks. So, also a single core CPU should be enough.

For example, in my system I get a combined read speed of 400 MiB/s, but SnapRAID is able to hash at 12000 MiB/s in a single core.

Is a 64 bit operating system recommended?

Yes. SnapRAID works better in a 64 bit operating system. It can access more than 4 GiB of memory, and use faster hashing and parity algorithms. Anyway, if you have less than 30 TB of data, it performs very well also using a 32 bit operating system.

How much memory SnapRAID requires to run?

Approximatively, the 32 bits version of SnapRAID requires 1 GiB of memory for 9 TB of data. The 64 bits version requires 1 GiB of memory for 7 TB of data.

If you have less memory than required, the recommended option is to install more memory in the machine.

Otherwise you have to increase the block_size option in the configuration file. The memory occupation is inversely proportional at the select block_size. For example, using "block_size 512", instead of the default 256, halves the memory occupation. But take care that using a bigger block size will also increase the amount of wasted space in the parity files.

Note that running with memory swapped to disk is not recommended because it will be a huge slowdown.

Do I have to check the SMART attributes of my disks?

Yes! 20% of the disks die before 4 years of life, as reported by Backblaze's How long do disk drives last?. Monitoring SMART attributes you can anticipate the failures and avoid to lose data!

If you see one of the SMART attributes Reallocated_Sector_Ct (5), Reported_Uncorrect (187), Command_Timeout (188), Current_Pending_Sector (197), and Offline_Uncorrectable (198) different than zero, replace the disk as recommended by Backblaze's Hard Drive SMART Stats.

Check also Google's Failure trends in a large disk drive population where is found that:

after the first reallocation, drives are
over 14 times more likely to fail within
60 days than drives without reallocation
counts.
after the first probational event, drives
are 16 times more likely to fail within
60 days than drives with zero probational
counts.
after the first offline reallocation, drives have over
21 times higher chances of failure within 60 days than
drives without offline reallocations.

that also recommens to replace the disk after the first error in the SMART attributes Reallocated_Sector_Ct (5), Current_Pending_Sector (197) and Offline_Uncorrectable (198).

Is it possible to use SnapRAID in a dual boot configuration, like Windows and Linux?

Yes. It's possible to use SnapRAID in a dual boot configuration. Doesn't matter the OS combination used.

In such case it's recommended to run SnapRAID only from one OS. In theory, it should also work if run in both OS. But it's a scenario not really tested, and there is always the risk to mess-up things, like using different SnapRAID version, configuration, content files, and so on.

Configuration

What's the SnapRAID 'content' file?

It's the file used by SnapRAID to save the list of all files present in your array with all the checksum, timestamp and any other information needed.

These file will be of about few GiB depending of how big is your array. Approximatively for 10 TB of data, you'll need 500 MiB of content file.

What are the SnapRAID 'parity' and 'N-parity' files?

They are the files used by SnapRAID to store the parity data used in recovering.

These files will grow in size as the biggest amount of data stored in a single disk of the array.

How to configure SAMBA to share the Pool directory

To configure SAMBA to share the pool directory, for example /pool, you should add to your /etc/samba/smb.conf the following options:

# In the global section of smb.conf
unix extensions = no

# In the share section of smb.conf
[pool]
comment = Pool
path = /pool
read only = yes
guest ok = yes
wide links = yes

How to use more disks than drive letters in Windows?

You can use Volume Mount points. You can mount a disk in a directory without using a drive letter. Just like in Linux.

Use

What are the most important things to do to prevent damages?

If you want to minimize the probability to lose data, do this:

How can I add an additional data disk to an existing array?

To add a new data disk at the array, add the new "disk" option in the configuration file, and then run a "sync" command.

snapraid sync

If the disk is empty, the "sync" command will be immediate.

How can I remove a data disk from an existing array?

To remove a data disk from the array do:

Your array is now without any reference to the removed disk.

How can I add an additional parity disk to an existing array?

To add a new parity level, add the proper "N-parity" option in the configuration file, and then run the "sync" command, using the "-F, --force-full" option:

snapraid -F sync

The "-F" option tells at SnapRAID to recompute the full parity.

Note that in the process, you will be always protected, because the existing parity is not modified.

If you wish to remove a parity, you can simply remove the highest "N-parity" option from the configuration and then delete the parity file.

Take care that after removing a parity file, you cannot reuse it anymore, because it gets outdated after the next "sync" command.

How can I replace a data disk?

If you lost the disk, see the recovering section in the manual. Otherwise, copy all the files in the new disk, maintaining the same directory structure and names.

In Linux use the command:

cp -av /from_dir/. /to_dir

WARNING! Do not use the 'rsync' command because it's not able to copy the sub-second timestamps.

In Windows use the command:

robocopy F:\from_dir T:\to_dir /e /copyall

Then change the SnapRAID configuration to point the disk at the new mount point and run a "diff" command to ensure that everything is placed correctly.

snapraid diff

If "diff" reports some "added" or "removed" files likely you have copied files with a different directory structure. In such case you have to fix it, until "diff" reports only "equal", "restored" or "copied" files.

Now you can check that the files were really copied correctly, you can run a "check" audit command limited to the replaced disk to check the checksum of all the files.

snapraid check -a -d DISK_NAME

Finally you can run "sync" to update the SnapRAID state. This command will be almost immediate.

snapraid sync

How can I replace a parity disk?

If the disk is still accessible, first ensure to have a synched array running a sync command:

snapraid sync

Then you can copy the big parity file to the new disk, and configure the new parity location in the configuration file.

In Linux a good copy command is ddrescue. You can use:

ddrescue /OLD_DISK/parity /NEW_DISK/parity /tmp/copy.log

This will copy the parity file from the old to new location, and in case of errors, it will continue copying what can be copied. To retry to copy the problematic sections, you can just retry the same copy command.

Having a partial parity file, is still beneficial, as it will still allow to recover data as long it uses the valid part. To restore it to full functionality, just run a "fix" command.

If instead you completely lost the parity file, you can run the "fix" command to recreate it, but ensure to use the -d option to fix only the interested parity and not the other disks.

snapraid fix -d PARITY_NAME

How can I safely move files from one disk to another?

In case you need to move files from one data disk of the array to another disk also in the array, the 100% safe way to proceed is: