rt/snapraid
1
0
Fork 0
mirror of https://github.com/amadvance/snapraid.git synced 2026-10-26 10:51:54 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update the manual for the new hashsize option

Browse source
This commit is contained in:
Andrea Mazzoleni 2016-04-10 19:41:56 +02:00
parent 5bf030c793
commit 0c624dd431
3 changed files with 93 additions and 87 deletions
Download patch file Download diff file Expand all files Collapse all files

68
snapraid.1
View file

@ -1325,34 +1325,16 @@ specifications.
This option can be used many times.
.SS blocksize SIZE_IN_KIBIBYTES
Defines the basic block size in kibi bytes of the parity.
One kibi bytes is 1024 bytes. The default blocksize is 256
and it should work for most cases.
One kibi bytes is 1024 bytes.
.PP
A reason to use a different blocksize is if your system has less
than 4 GiB of memory. As a rule of thumb, with 4 GiB or more memory
use the default 256, with 2 GiB use 512, and with 1 GiB use 1024.
The default blocksize is 256 and it should work for most cases.
.PP
In more details SnapRAID requires about TS*17/BS bytes of RAM memory.
Where TS is the total size in bytes of your disk array, and BS is the
block size in bytes.
WARNING! This option is for experts only, and it\'s highly
recommended to not change it. To change again this value in
future you\'ll have to recreate the whole parity.
.PP
For example with 8 disks of 4 TB and a block size of 256 KiB
(1 KiB = 1024 bytes) you have:
.PP
RAM = (8 * 4 * 10^12) * 17 / (256 * 2^10) = 1.93 GiB
.PD 0
.PP
.PD
.PP
Switching to a block size of 512 KiB, you have:
.PP
RAM = (8 * 4 * 10^12) * 17 / (512 * 2^10) = 0.97 GiB
.PD 0
.PP
.PD
.PP
Another reason to use a different blocksize is if you have a lot of
small files. In the order of many millions.
A reason to use a different hashsize is if you have a lot of small
files. In the order of many millions.
.PP
For each file, even of few bytes, a whole block of parity is allocated,
and with many files this may result in a lot of unused parity space.
@ -1392,32 +1374,52 @@ This results in about 1.5% of extra space. Meaning about 60 GB for
a 4 TB disk, that allows about 460000 files in each data disk without
any wasted space.
.SS hashsize SIZE_IN_BYTES
Defines the basic hash size in bytes of the saved blocks.
The default hashsize is 16 bytes \-> 128 bits, and it should work
Defines the hash size in bytes of the saved blocks.
.PP
The default hashsize is 16 bytes (128 bits), and it should work
for most cases.
.PP
A reason to use a different hashsize is if your system has less
than 1 GiB of memory. As a rule of thumb, with 1 GiB or more memory
use the default 16, with 512 MiB use 8, and with 256 GiB use 4.
WARNING! This option is for experts only, and it\'s highly
recommended to not change it. To change again this value in
future you\'ll have to recreate the whole parity.
.PP
In more details SnapRAID requires about TS*(1+HS)/BS bytes of RAM memory.
A reason to use a different hashsize is if your system has
small memory. As a rule of thumb SnapRAID usually requires
1 GiB of RAM memory for each 16 TB of data in the array.
.PP
Specifically, to store the hashes of the data, SnapRAID requires
about TS*(1+HS)/BS bytes of RAM memory.
Where TS is the total size in bytes of your disk array, BS is the
block size in bytes, and HS is the hash size in bytes.
.PP
For example with 8 disks of 4 TB and a block size of 256 KiB
(1 KiB = 1024 bytes), and an hash size of 16, you have:
(1 KiB = 1024 bytes), and an hash size of 16, you get:
.PP
RAM = (8 * 4 * 10^12) * (1+16) / (256 * 2^10) = 1.93 GiB
.PD 0
.PP
.PD
.PP
Switching to a hash size of 8, you have:
Switching to a hash size of 8, you get:
.PP
RAM = (8 * 4 * 10^12) * (1+8) / (256 * 2^10) = 1.02 GiB
.PD 0
.PP
.PD
.PP
Switching to a block size of 512, you get:
.PP
RAM = (8 * 4 * 10^12) * (1+16) / (512 * 2^10) = 0.96 GiB
.PD 0
.PP
.PD
.PP
Switching to both a hash size of 8, and a block size of 512 you get:
.PP
RAM = (8 * 4 * 10^12) * (1+8) / (512 * 2^10) = 0.51 GiB
.PD 0
.PP
.PD
.SS autosave SIZE_IN_GIGABYTES
Automatically save the state when syncing or scrubbing after the specified amount
of GB processed.

56
snapraid.d
View file

@ -1029,28 +1029,16 @@ Configuration
blocksize SIZE_IN_KIBIBYTES
Defines the basic block size in kibi bytes of the parity.
One kibi bytes is 1024 bytes. The default blocksize is 256
and it should work for most cases.
One kibi bytes is 1024 bytes.
A reason to use a different blocksize is if your system has less
than 4 GiB of memory. As a rule of thumb, with 4 GiB or more memory
use the default 256, with 2 GiB use 512, and with 1 GiB use 1024.
The default blocksize is 256 and it should work for most cases.
In more details SnapRAID requires about TS*17/BS bytes of RAM memory.
Where TS is the total size in bytes of your disk array, and BS is the
block size in bytes.
WARNING! This option is for experts only, and it's highly
recommended to not change it. To change again this value in
future you'll have to recreate the whole parity.
For example with 8 disks of 4 TB and a block size of 256 KiB
(1 KiB = 1024 bytes) you have:
:RAM = (8 * 4 * 10^12) * 17 / (256 * 2^10) = 1.93 GiB
Switching to a block size of 512 KiB, you have:
:RAM = (8 * 4 * 10^12) * 17 / (512 * 2^10) = 0.97 GiB
Another reason to use a different blocksize is if you have a lot of
small files. In the order of many millions.
A reason to use a different hashsize is if you have a lot of small
files. In the order of many millions.
For each file, even of few bytes, a whole block of parity is allocated,
and with many files this may result in a lot of unused parity space.
@ -1086,27 +1074,41 @@ Configuration
any wasted space.
hashsize SIZE_IN_BYTES
Defines the basic hash size in bytes of the saved blocks.
The default hashsize is 16 bytes -> 128 bits, and it should work
Defines the hash size in bytes of the saved blocks.
The default hashsize is 16 bytes (128 bits), and it should work
for most cases.
A reason to use a different hashsize is if your system has less
than 1 GiB of memory. As a rule of thumb, with 1 GiB or more memory
use the default 16, with 512 MiB use 8, and with 256 GiB use 4.
WARNING! This option is for experts only, and it's highly
recommended to not change it. To change again this value in
future you'll have to recreate the whole parity.
In more details SnapRAID requires about TS*(1+HS)/BS bytes of RAM memory.
A reason to use a different hashsize is if your system has
small memory. As a rule of thumb SnapRAID usually requires
1 GiB of RAM memory for each 16 TB of data in the array.
Specifically, to store the hashes of the data, SnapRAID requires
about TS*(1+HS)/BS bytes of RAM memory.
Where TS is the total size in bytes of your disk array, BS is the
block size in bytes, and HS is the hash size in bytes.
For example with 8 disks of 4 TB and a block size of 256 KiB
(1 KiB = 1024 bytes), and an hash size of 16, you have:
(1 KiB = 1024 bytes), and an hash size of 16, you get:
:RAM = (8 * 4 * 10^12) * (1+16) / (256 * 2^10) = 1.93 GiB
Switching to a hash size of 8, you have:
Switching to a hash size of 8, you get:
:RAM = (8 * 4 * 10^12) * (1+8) / (256 * 2^10) = 1.02 GiB
Switching to a block size of 512, you get:
:RAM = (8 * 4 * 10^12) * (1+16) / (512 * 2^10) = 0.96 GiB
Switching to both a hash size of 8, and a block size of 512 you get:
:RAM = (8 * 4 * 10^12) * (1+8) / (512 * 2^10) = 0.51 GiB
autosave SIZE_IN_GIGABYTES
Automatically save the state when syncing or scrubbing after the specified amount
of GB processed.

56
snapraid.txt
View file

@ -1109,28 +1109,16 @@ This option can be used many times.
-------------------------------
Defines the basic block size in kibi bytes of the parity.
One kibi bytes is 1024 bytes. The default blocksize is 256
and it should work for most cases.
One kibi bytes is 1024 bytes.
A reason to use a different blocksize is if your system has less
than 4 GiB of memory. As a rule of thumb, with 4 GiB or more memory
use the default 256, with 2 GiB use 512, and with 1 GiB use 1024.
The default blocksize is 256 and it should work for most cases.
In more details SnapRAID requires about TS*17/BS bytes of RAM memory.
Where TS is the total size in bytes of your disk array, and BS is the
block size in bytes.
WARNING! This option is for experts only, and it's highly
recommended to not change it. To change again this value in
future you'll have to recreate the whole parity.
For example with 8 disks of 4 TB and a block size of 256 KiB
(1 KiB = 1024 bytes) you have:
RAM = (8 * 4 * 10^12) * 17 / (256 * 2^10) = 1.93 GiB
Switching to a block size of 512 KiB, you have:
RAM = (8 * 4 * 10^12) * 17 / (512 * 2^10) = 0.97 GiB
Another reason to use a different blocksize is if you have a lot of
small files. In the order of many millions.
A reason to use a different hashsize is if you have a lot of small
files. In the order of many millions.
For each file, even of few bytes, a whole block of parity is allocated,
and with many files this may result in a lot of unused parity space.
@ -1168,27 +1156,41 @@ any wasted space.
7.9 hashsize SIZE_IN_BYTES
--------------------------
Defines the basic hash size in bytes of the saved blocks.
The default hashsize is 16 bytes -> 128 bits, and it should work
Defines the hash size in bytes of the saved blocks.
The default hashsize is 16 bytes (128 bits), and it should work
for most cases.
A reason to use a different hashsize is if your system has less
than 1 GiB of memory. As a rule of thumb, with 1 GiB or more memory
use the default 16, with 512 MiB use 8, and with 256 GiB use 4.
WARNING! This option is for experts only, and it's highly
recommended to not change it. To change again this value in
future you'll have to recreate the whole parity.
In more details SnapRAID requires about TS*(1+HS)/BS bytes of RAM memory.
A reason to use a different hashsize is if your system has
small memory. As a rule of thumb SnapRAID usually requires
1 GiB of RAM memory for each 16 TB of data in the array.
Specifically, to store the hashes of the data, SnapRAID requires
about TS*(1+HS)/BS bytes of RAM memory.
Where TS is the total size in bytes of your disk array, BS is the
block size in bytes, and HS is the hash size in bytes.
For example with 8 disks of 4 TB and a block size of 256 KiB
(1 KiB = 1024 bytes), and an hash size of 16, you have:
(1 KiB = 1024 bytes), and an hash size of 16, you get:
RAM = (8 * 4 * 10^12) * (1+16) / (256 * 2^10) = 1.93 GiB
Switching to a hash size of 8, you have:
Switching to a hash size of 8, you get:
RAM = (8 * 4 * 10^12) * (1+8) / (256 * 2^10) = 1.02 GiB
Switching to a block size of 512, you get:
RAM = (8 * 4 * 10^12) * (1+16) / (512 * 2^10) = 0.96 GiB
Switching to both a hash size of 8, and a block size of 512 you get:
RAM = (8 * 4 * 10^12) * (1+8) / (512 * 2^10) = 0.51 GiB
7.10 autosave SIZE_IN_GIGABYTES
-------------------------------