mirror of
https://github.com/amadvance/snapraid.git
synced 2026-10-26 02:49:02 -04:00
Remove the requirement to snapshot content files
The new content file format now includes metadata for previous versions of files. This allows the fix command to access historical file data from snapshots as needed, ensuring recovery even if the live filesystem has changed or the file was deleted during a sync.
This commit is contained in:
parent
275063d23c
commit
2e71602267
13 changed files with 209 additions and 368 deletions
124
doc/snapraid.txt
124
doc/snapraid.txt
|
|
@ -1234,9 +1234,10 @@ filesystem. Parity disks, or data disks using other filesystems, will
|
|||
always use the live version of the filesystem.
|
||||
|
||||
When enabled, SnapRAID ensures atomic and consistent operations by
|
||||
reading from a frozen point-in-time image. This prevents errors caused
|
||||
by file modifications during execution and ensures that deleted files
|
||||
remain available for parity-based recovery of other disks.
|
||||
reading from a frozen point-in-time image. This prevents errors
|
||||
caused by file modifications during execution and ensures that
|
||||
updated and deleted files remain available for parity-based
|
||||
recovery of other disks.
|
||||
|
||||
See the SNAPSHOTS section for more details.
|
||||
|
||||
|
|
@ -1519,12 +1520,13 @@ This provides two primary benefits:
|
|||
Consistency - Files modified on the live filesystem during a long-running
|
||||
sync or scrub will not cause parity mismatches or aborted
|
||||
operations, as SnapRAID sees a frozen point-in-time image.
|
||||
Recovery - If a file is deleted from the live filesystem, it remains
|
||||
preserved in the snapshot. If a disk failure occurs before the
|
||||
next sync is run, SnapRAID uses the data preserved in the
|
||||
snapshot to reconstruct the failed disk. Without snapshots,
|
||||
a deleted file on a healthy disk results in missing data
|
||||
blocks that may be required to fix other failed disks.
|
||||
Recovery - If a file is updated or deleted from the live filesystem,
|
||||
it remains preserved in the snapshot. If a disk failure occurs
|
||||
before the next sync is run, SnapRAID uses the data preserved
|
||||
in the snapshot to reconstruct the failed disk.
|
||||
Without snapshots, an updated or deleted file on a healthy
|
||||
disk results in missing data blocks that may be required to
|
||||
fix other failed disks.
|
||||
|
||||
Snapshots are created only for data disks and only if the underlying
|
||||
filesystem supports this functionality. Parity disks always use
|
||||
|
|
@ -1539,9 +1541,9 @@ to operate directly on the live filesystem.
|
|||
|
||||
The `sync` and `scrub` commands both operate exclusively on snapshots
|
||||
to ensure that all data operations are performed against a consistent,
|
||||
frozen state. By utilizing these snapshots, SnapRAID prevents false
|
||||
positives and parity mismatches that would otherwise be triggered
|
||||
by concurrent file modifications on the live filesystem.
|
||||
frozen state. By utilizing these snapshots, SnapRAID prevents parity
|
||||
mismatches that would otherwise be triggered by concurrent file
|
||||
modifications on the live filesystem.
|
||||
During a `sync`, the command uses a new snapshot created at the
|
||||
start of the process to capture the current state for parity computation.
|
||||
In contrast, the `scrub` command utilizes the last snapshot, the one
|
||||
|
|
@ -1564,69 +1566,45 @@ in this case, `check` and `fix` will use the live filesystems exclusively.
|
|||
|
||||
All other commands operate exclusively on the live filesystem.
|
||||
|
||||
8.2 Sync Safety
|
||||
---------------
|
||||
|
||||
To ensure maximum data safety during a sync operation, it is
|
||||
recommended to place the .content files inside the same subvolume
|
||||
used for the data, ensuring they are also part of the snapshots.
|
||||
9 SNAPSHOTS LIFECYCLE
|
||||
=====================
|
||||
|
||||
When a sync starts, SnapRAID immediately updates the .content file to
|
||||
reflect the current state of the filesystem. If you have deleted files
|
||||
on a disk, the new .content file loses the metadata for those
|
||||
deleted items and their relation to the parity.
|
||||
SnapRAID manages two specific snapshots, `stable` and `pending`,
|
||||
within a hidden `.snapraid` directory at the root of each data
|
||||
subvolume.
|
||||
|
||||
If a disk fails while a `sync` is running, the parity for that
|
||||
failed disk may still depend on the deleted files.
|
||||
Even if your snapshot physically contains the deleted files,
|
||||
the new .content file no longer knows how they relate to the
|
||||
parity blocks. This missing correlation can prevent a full recovery.
|
||||
The `stable` snapshot represents the state of the last
|
||||
successfully completed `sync`, containing the exact data used to
|
||||
compute the current parity. It serves as the primary data source
|
||||
for `scrub`, `check`, and `fix` commands.
|
||||
|
||||
Note that this specific risk only exists if a disk fails during
|
||||
an active sync. If a disk fails between syncs, the snapshot
|
||||
already contains data that is perfectly matched with the
|
||||
last successfully saved content file on the live filesystem,
|
||||
ensuring a successful recovery.
|
||||
The `pending` snapshot is a temporary image created at the start
|
||||
of a `sync` to provide a frozen state for parity computation.
|
||||
Upon successful completion of the operation, the previous stable
|
||||
snapshot is deleted, and the pending snapshot is promoted to
|
||||
take its place as the new stable reference.
|
||||
|
||||
By storing the .content file inside the snapshotted subvolume, the
|
||||
snapshot preserves an older version of the content file, and
|
||||
the metadata within it, alongside the corresponding version of
|
||||
the data.
|
||||
If a `sync` is interrupted, the `pending` snapshot is preserved.
|
||||
Subsequent `scrub`, `check`, and `fix` commands will use this
|
||||
pending snapshot as it matches the filesystem state recorded in
|
||||
the .content file, even if the parity is only partially synchronized.
|
||||
|
||||
8.3 Snapshots Lifecycle
|
||||
-----------------------
|
||||
In the event of a `sync` interruption, `check` and `fix` commands
|
||||
can also read from the `stable` snapshot to retrieve data blocks
|
||||
from files that were updated or deleted during the sync.
|
||||
This is possible because the .content file retains metadata for
|
||||
all modified or deleted files throughout the sync process.
|
||||
This historical information is only cleared once a sync finishes
|
||||
successfully and the final version of the .content file is saved.
|
||||
|
||||
SnapRAID manages two specific snapshots within a hidden `.snapraid`
|
||||
directory at the root of each data subvolume:
|
||||
|
||||
stable - The reference snapshot representing the state of the last
|
||||
successfully completed `sync`. This snapshot contains the exact
|
||||
data used to compute the parity and serves as the primary data
|
||||
source for all `scrub`, `check`, and `fix` commands, provided
|
||||
that a pending snapshot is not present.
|
||||
pending - A temporary snapshot created at the start of a sync operation.
|
||||
Upon the successful completion of a sync, the previous stable
|
||||
snapshot is deleted, and the pending snapshot is promoted to
|
||||
stable.
|
||||
During this promotion, if a content file is stored on the
|
||||
same subvolume of the data, the final version of the content
|
||||
file is cloned (via reflink) into the snapshot.
|
||||
This ensures that the snapshot contains a metadata map that
|
||||
is perfectly in sync with the data blocks it preserves.
|
||||
Once promoted, the stable snapshot is set to a read-only
|
||||
state to ensure its integrity as a recovery reference.
|
||||
If the sync fails or is interrupted, the pending snapshot is
|
||||
preserved in its current state. This allows subsequent `scrub`,
|
||||
`check`, and `fix` commands to function correctly by using the
|
||||
exact filesystem state recorded in the live content file,
|
||||
even if the parity is only partially synchronized.
|
||||
When sync is restarted after the interruption, the existing
|
||||
pending snapshot is deleted and a new one is created,
|
||||
capturing the new state of the live filesystem.
|
||||
If a `sync` is restarted after an interruption, the existing
|
||||
pending snapshot is deleted and a new one is created to
|
||||
capture the current state of the live filesystem.
|
||||
|
||||
|
||||
9 PATTERN
|
||||
=========
|
||||
10 PATTERN
|
||||
==========
|
||||
|
||||
Patterns are used to select a subset of files to exclude or include in
|
||||
the process. Globbing characters can be used to match files and paths
|
||||
|
|
@ -1743,7 +1721,7 @@ In Unix, when using globbing characters on the command line, you must
|
|||
quote them to prevent the shell from expanding them.
|
||||
|
||||
|
||||
10 IGNORE FILE
|
||||
11 IGNORE FILE
|
||||
==============
|
||||
|
||||
In addition to the global rules in the configuration file, you can
|
||||
|
|
@ -1788,7 +1766,7 @@ For example, if you have a `.snapraidignore` in `/mnt/disk1/projects/`:
|
|||
*.tmp
|
||||
|
||||
|
||||
11 CONTENT
|
||||
12 CONTENT
|
||||
==========
|
||||
|
||||
SnapRAID stores the list and checksums of your files in the content file.
|
||||
|
|
@ -1800,7 +1778,7 @@ This file is read and written by the `sync` and `scrub` commands and
|
|||
read by the `fix`, `check`, and `status` commands.
|
||||
|
||||
|
||||
12 PARITY
|
||||
13 PARITY
|
||||
=========
|
||||
|
||||
SnapRAID stores the parity information of your array in the parity
|
||||
|
|
@ -1813,7 +1791,7 @@ These files are read and written by the `sync` and `fix` commands and
|
|||
only read by the `scrub` and `check` commands.
|
||||
|
||||
|
||||
13 ENCODING
|
||||
14 ENCODING
|
||||
===========
|
||||
|
||||
SnapRAID in Unix ignores any encoding. It reads and stores the
|
||||
|
|
@ -1830,7 +1808,7 @@ redirect the console output to a file, the resulting file is always
|
|||
in UTF-8 format.
|
||||
|
||||
|
||||
14 EXIT CODE
|
||||
15 EXIT CODE
|
||||
============
|
||||
|
||||
SnapRAID terminates with the following error codes:
|
||||
|
|
@ -1841,13 +1819,13 @@ SnapRAID terminates with the following error codes:
|
|||
needed.
|
||||
|
||||
|
||||
15 COPYRIGHT
|
||||
16 COPYRIGHT
|
||||
============
|
||||
|
||||
This file is Copyright (C) 2026 Andrea Mazzoleni
|
||||
|
||||
|
||||
16 SEE ALSO
|
||||
17 SEE ALSO
|
||||
===========
|
||||
|
||||
snapraid_log(1), snapraidd(1), rsync(1)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue