Remove the requirement to snapshot content files

The new content file format now includes metadata for previous versions
of files. This allows the fix command to access historical file data from
snapshots as needed, ensuring recovery even if the live filesystem has
changed or the file was deleted during a sync.
This commit is contained in:
Andrea Mazzoleni 2026-04-28 00:35:15 +02:00
parent 275063d23c
commit 2e71602267
13 changed files with 209 additions and 368 deletions

View file

@ -1234,9 +1234,10 @@ filesystem. Parity disks, or data disks using other filesystems, will
always use the live version of the filesystem.
When enabled, SnapRAID ensures atomic and consistent operations by
reading from a frozen point-in-time image. This prevents errors caused
by file modifications during execution and ensures that deleted files
remain available for parity-based recovery of other disks.
reading from a frozen point-in-time image. This prevents errors
caused by file modifications during execution and ensures that
updated and deleted files remain available for parity-based
recovery of other disks.
See the SNAPSHOTS section for more details.
@ -1519,12 +1520,13 @@ This provides two primary benefits:
Consistency - Files modified on the live filesystem during a long-running
sync or scrub will not cause parity mismatches or aborted
operations, as SnapRAID sees a frozen point-in-time image.
Recovery - If a file is deleted from the live filesystem, it remains
preserved in the snapshot. If a disk failure occurs before the
next sync is run, SnapRAID uses the data preserved in the
snapshot to reconstruct the failed disk. Without snapshots,
a deleted file on a healthy disk results in missing data
blocks that may be required to fix other failed disks.
Recovery - If a file is updated or deleted from the live filesystem,
it remains preserved in the snapshot. If a disk failure occurs
before the next sync is run, SnapRAID uses the data preserved
in the snapshot to reconstruct the failed disk.
Without snapshots, an updated or deleted file on a healthy
disk results in missing data blocks that may be required to
fix other failed disks.
Snapshots are created only for data disks and only if the underlying
filesystem supports this functionality. Parity disks always use
@ -1539,9 +1541,9 @@ to operate directly on the live filesystem.
The `sync` and `scrub` commands both operate exclusively on snapshots
to ensure that all data operations are performed against a consistent,
frozen state. By utilizing these snapshots, SnapRAID prevents false
positives and parity mismatches that would otherwise be triggered
by concurrent file modifications on the live filesystem.
frozen state. By utilizing these snapshots, SnapRAID prevents parity
mismatches that would otherwise be triggered by concurrent file
modifications on the live filesystem.
During a `sync`, the command uses a new snapshot created at the
start of the process to capture the current state for parity computation.
In contrast, the `scrub` command utilizes the last snapshot, the one
@ -1564,69 +1566,45 @@ in this case, `check` and `fix` will use the live filesystems exclusively.
All other commands operate exclusively on the live filesystem.
8.2 Sync Safety
---------------
To ensure maximum data safety during a sync operation, it is
recommended to place the .content files inside the same subvolume
used for the data, ensuring they are also part of the snapshots.
9 SNAPSHOTS LIFECYCLE
=====================
When a sync starts, SnapRAID immediately updates the .content file to
reflect the current state of the filesystem. If you have deleted files
on a disk, the new .content file loses the metadata for those
deleted items and their relation to the parity.
SnapRAID manages two specific snapshots, `stable` and `pending`,
within a hidden `.snapraid` directory at the root of each data
subvolume.
If a disk fails while a `sync` is running, the parity for that
failed disk may still depend on the deleted files.
Even if your snapshot physically contains the deleted files,
the new .content file no longer knows how they relate to the
parity blocks. This missing correlation can prevent a full recovery.
The `stable` snapshot represents the state of the last
successfully completed `sync`, containing the exact data used to
compute the current parity. It serves as the primary data source
for `scrub`, `check`, and `fix` commands.
Note that this specific risk only exists if a disk fails during
an active sync. If a disk fails between syncs, the snapshot
already contains data that is perfectly matched with the
last successfully saved content file on the live filesystem,
ensuring a successful recovery.
The `pending` snapshot is a temporary image created at the start
of a `sync` to provide a frozen state for parity computation.
Upon successful completion of the operation, the previous stable
snapshot is deleted, and the pending snapshot is promoted to
take its place as the new stable reference.
By storing the .content file inside the snapshotted subvolume, the
snapshot preserves an older version of the content file, and
the metadata within it, alongside the corresponding version of
the data.
If a `sync` is interrupted, the `pending` snapshot is preserved.
Subsequent `scrub`, `check`, and `fix` commands will use this
pending snapshot as it matches the filesystem state recorded in
the .content file, even if the parity is only partially synchronized.
8.3 Snapshots Lifecycle
-----------------------
In the event of a `sync` interruption, `check` and `fix` commands
can also read from the `stable` snapshot to retrieve data blocks
from files that were updated or deleted during the sync.
This is possible because the .content file retains metadata for
all modified or deleted files throughout the sync process.
This historical information is only cleared once a sync finishes
successfully and the final version of the .content file is saved.
SnapRAID manages two specific snapshots within a hidden `.snapraid`
directory at the root of each data subvolume:
stable - The reference snapshot representing the state of the last
successfully completed `sync`. This snapshot contains the exact
data used to compute the parity and serves as the primary data
source for all `scrub`, `check`, and `fix` commands, provided
that a pending snapshot is not present.
pending - A temporary snapshot created at the start of a sync operation.
Upon the successful completion of a sync, the previous stable
snapshot is deleted, and the pending snapshot is promoted to
stable.
During this promotion, if a content file is stored on the
same subvolume of the data, the final version of the content
file is cloned (via reflink) into the snapshot.
This ensures that the snapshot contains a metadata map that
is perfectly in sync with the data blocks it preserves.
Once promoted, the stable snapshot is set to a read-only
state to ensure its integrity as a recovery reference.
If the sync fails or is interrupted, the pending snapshot is
preserved in its current state. This allows subsequent `scrub`,
`check`, and `fix` commands to function correctly by using the
exact filesystem state recorded in the live content file,
even if the parity is only partially synchronized.
When sync is restarted after the interruption, the existing
pending snapshot is deleted and a new one is created,
capturing the new state of the live filesystem.
If a `sync` is restarted after an interruption, the existing
pending snapshot is deleted and a new one is created to
capture the current state of the live filesystem.
9 PATTERN
=========
10 PATTERN
==========
Patterns are used to select a subset of files to exclude or include in
the process. Globbing characters can be used to match files and paths
@ -1743,7 +1721,7 @@ In Unix, when using globbing characters on the command line, you must
quote them to prevent the shell from expanding them.
10 IGNORE FILE
11 IGNORE FILE
==============
In addition to the global rules in the configuration file, you can
@ -1788,7 +1766,7 @@ For example, if you have a `.snapraidignore` in `/mnt/disk1/projects/`:
*.tmp
11 CONTENT
12 CONTENT
==========
SnapRAID stores the list and checksums of your files in the content file.
@ -1800,7 +1778,7 @@ This file is read and written by the `sync` and `scrub` commands and
read by the `fix`, `check`, and `status` commands.
12 PARITY
13 PARITY
=========
SnapRAID stores the parity information of your array in the parity
@ -1813,7 +1791,7 @@ These files are read and written by the `sync` and `fix` commands and
only read by the `scrub` and `check` commands.
13 ENCODING
14 ENCODING
===========
SnapRAID in Unix ignores any encoding. It reads and stores the
@ -1830,7 +1808,7 @@ redirect the console output to a file, the resulting file is always
in UTF-8 format.
14 EXIT CODE
15 EXIT CODE
============
SnapRAID terminates with the following error codes:
@ -1841,13 +1819,13 @@ SnapRAID terminates with the following error codes:
needed.
15 COPYRIGHT
16 COPYRIGHT
============
This file is Copyright (C) 2026 Andrea Mazzoleni
16 SEE ALSO
17 SEE ALSO
===========
snapraid_log(1), snapraidd(1), rsync(1)