mirror of
https://github.com/lrnselfreliance/wrolpi.git
synced 2026-11-21 07:26:18 -05:00
* A secondary WROLPi copies its primary's library over HTTP A WROLPi with role "secondary" pulls from its primary: a manifest of every FileGroup's files (/api/sync/manifest), then each missing or changed file from the primary's Caddy /media/ (resumable, written to a hidden partial and renamed with the primary's mtime), a refresh, then the library configs, imported as the primary wrote them. wrolpi.yaml contributes only its library settings. Files deleted on the primary are kept. A secondary never saves the synced configs and never downloads. Its API answers 503 SYNCING while a sync runs, and every page warns that it is a copy. It syncs in a daily window or on "Sync now"; WROL Mode stops it. On a Pi 4 (no AES instructions) the client uses TLS 1.2 ChaCha20, which pulls ~110 MB/s from Caddy versus ~41 MB/s for rsync over ssh. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Sync has its own /admin tab; a sync's reads from the primary are bounded The sync settings and progress move from Settings (and the component gallery, which is for library components) to /admin/sync. A download never writes past the size in the primary's manifest, and configs, listings, and manifest pages are capped. A primary address may not carry a user name or password (it is logged and shown). While a sync runs, the API answers only the exact requests that show its progress or stop it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A sync keeps the secondary's replaced configs in today's config backup The first copy of the day is kept, like a save, so a config backup import undoes a sync's config changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Docker's Caddy logs every request `docker compose logs -f web` shows each request with its status, size and duration, e.g. a secondary's /media/ downloads during a sync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A sync reports its progress as it downloads, and refuses quietly The status updated only when a page of 500 FileGroups finished, so a page of large files looked stuck; it now updates every 2 seconds. Requests refused while syncing are answered by the middleware instead of raised, so the UI's polling no longer logs an error each time. The primary's version is stored without its trailing newline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A WROLPi can back up to a second drive; sync settings live in sync.yaml The Sync tab's Backup Drive copies the media directory to a second mounted drive with rsync during the sync window (hourly), or on "Back Up Now". Nothing on the backup drive is deleted. The Tags and playlists directories are left out (WROLPi rebuilds them from the configs), and the database is written as an SQLite snapshot rather than copied as files, so the drive can be mounted as the media directory and start. A backup is written only while the chosen drive (by UUID) is mounted where it was chosen, and is not the media or system drive; rsync is stopped if the drive disappears. Not in Docker, which cannot reach other drives. Role, primary, window and backup drive move from wrolpi.yaml to sync.yaml: they belong to this machine, and are never copied from a primary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * The Sync tab shows the configured backup drive getSettings copies each setting by name and did not list the backup drive, so the Sync tab showed "None" while one was configured. It also dropped `version`. A test now fails for any setting the API sends that getSettings drops. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A backup can go to a folder on its drive, and remembers its last success The Backup Drive panel takes a folder on the chosen drive. The drive is checked through the folder's mount point: the folder is written only when the drive with the saved UUID is mounted there, so a folder left on the system drive by an unplugged drive, or on another drive mounted inside the chosen one, is refused. A missing folder is created once the drive is checked. When the last backup succeeded is saved in sync.yaml, so it is known after a restart. "Stop Backup" is disabled while the database snapshot is written (it runs to the end), and a finished backup clears the request to stop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * The backup drive has its own window The sync window was shown in the Sync panel but also scheduled the backup drive. The Backup Drive panel now has its own window (sync.yaml backup_window_start/end), saved with the drive; the sync window is shown, and used, only for a secondary's sync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Sync and the backup drive can each exclude folders A secondary or a backup drive may be smaller than the primary's library. Each Sync tab panel has its own excluded folders in sync.yaml, chosen in a folder browser: a secondary browses its primary's folders (through Caddy), the backup drive browses this WROLPi's media directory. Excluded folders are never copied; what was copied before is kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * The folder browser does not offer lost+found Every ext4 drive has a root-only lost+found folder; a sync and a backup never copy it, so it is not offered for exclusion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A sync never writes into config or tags, whatever their case On a media drive that ignores case (exFAT, NTFS, macOS), a manifest path like Config/wrolpi.db is the real config folder. The protected and unlisted top-level folders are now compared case-insensitively in the manifest, in resolve_destination, in excluded-folder validation, and in both folder listings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A backup writes the database snapshot only when the database changed Every backup rewrote the whole database snapshot (2.5 GB on a busy WROLPi), even on the window's idle hourly runs. The snapshot is now skipped when the live database (its size and mtime, and those of a non-empty WAL) and the drive's copy are both unchanged since the last snapshot. The record is in memory, so the first backup after a restart writes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Remove the unused save_sync_config switch Nothing activated it; SyncConfig saves directly. The test server's log_message override is called by http.server, so it goes in the vulture baseline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
467 lines
22 KiB
Python
467 lines
22 KiB
Python
import contextlib
|
|
import os
|
|
import pathlib
|
|
import re
|
|
import shutil
|
|
import sqlite3
|
|
import subprocess
|
|
import tempfile
|
|
from datetime import datetime, timezone
|
|
|
|
import pytest
|
|
import yaml
|
|
|
|
from wrolpi.common import get_wrolpi_config
|
|
from wrolpi.sync import backup
|
|
from wrolpi.sync.config import get_sync_config
|
|
from wrolpi.sync.status import get_backup_status, update_backup_status
|
|
|
|
|
|
def _rsync_supports_progress2() -> bool:
|
|
"""rsync 3.1+ (Raspberry Pi OS, Debian); macOS ships openrsync, which has no `--info=progress2`."""
|
|
if not shutil.which('rsync'):
|
|
return False
|
|
output = subprocess.run(['rsync', '--version'], capture_output=True, text=True).stdout
|
|
match = re.search(r'rsync\s+version\s+(\d+)\.(\d+)', output)
|
|
return bool(match) and (int(match.group(1)), int(match.group(2))) >= (3, 1)
|
|
|
|
|
|
requires_rsync = pytest.mark.skipif(not _rsync_supports_progress2(), reason='rsync 3.1 or newer is not installed')
|
|
|
|
|
|
@pytest.fixture
|
|
def backup_drive(test_directory, test_wrolpi_config, monkeypatch):
|
|
"""A temporary directory that the target checks accept as the mounted backup drive `uuid-1`."""
|
|
with tempfile.TemporaryDirectory() as tmp_dir:
|
|
drive = pathlib.Path(tmp_dir).resolve()
|
|
monkeypatch.setattr(backup, 'check_backup_target', lambda directory, uuid: None)
|
|
get_sync_config()._config.update(backup_directory=str(drive), backup_device_uuid='uuid-1')
|
|
yield drive
|
|
|
|
|
|
def test_build_rsync_command(monkeypatch):
|
|
monkeypatch.setattr(backup.shutil, 'which', lambda name: f'/usr/bin/{name}')
|
|
command = backup.build_rsync_command(pathlib.Path('/media/wrolpi'), pathlib.Path('/media/backup'), 'playlists')
|
|
assert command[:5] == ['/usr/bin/ionice', '-c3', '/usr/bin/nice', '-n', '19']
|
|
assert command[-2:] == ['/media/wrolpi/', '/media/backup/'], 'trailing slashes copy the contents, not the folder'
|
|
assert not any(i.startswith('--delete') for i in command), 'a backup never deletes'
|
|
assert '-H' not in command and '--hard-links' not in command
|
|
for exclude in ('/config/wrolpi.db', '/config/wrolpi.db-*', '/tags/', '/playlists/', '.*.partial',
|
|
'.rsync-partial/', '/lost+found/'):
|
|
assert f'--exclude={exclude}' in command, exclude
|
|
|
|
# ionice and nice are optional.
|
|
monkeypatch.setattr(backup.shutil, 'which', lambda name: '/usr/bin/rsync' if name == 'rsync' else None)
|
|
command = backup.build_rsync_command(pathlib.Path('/media/wrolpi'), pathlib.Path('/media/backup'), None)
|
|
assert command[0] == '/usr/bin/rsync'
|
|
assert not any(i.startswith('--exclude=/playlists') for i in command)
|
|
|
|
|
|
def test_build_rsync_command_excluded_directories(monkeypatch):
|
|
"""Excluded folders are anchored at the media directory and matched literally."""
|
|
monkeypatch.setattr(backup.shutil, 'which', lambda name: '/usr/bin/rsync' if name == 'rsync' else None)
|
|
command = backup.build_rsync_command(pathlib.Path('/media/wrolpi'), pathlib.Path('/media/backup'), None,
|
|
['Movies', 'videos/Movies [HD]', 'a*b?'])
|
|
assert '--exclude=/Movies/' in command
|
|
assert '--exclude=/videos/Movies \\[HD]/' in command
|
|
assert '--exclude=/a\\*b\\?/' in command
|
|
assert command[-2:] == ['/media/wrolpi/', '/media/backup/'], 'excludes come before the paths'
|
|
|
|
|
|
@pytest.mark.parametrize('line, expected', [
|
|
(' 32,768 0% 0.00kB/s 0:00:00 (xfr#1, ir-chk=1018/1023)',
|
|
dict(bytes_transferred=32768, percent=0, files_transferred=1, files_to_check=1023)),
|
|
(' 1234567890 45% 10.50MB/s 0:01:23 (xfr#12, to-chk=100/2000)',
|
|
dict(bytes_transferred=1234567890, percent=45, files_transferred=12, files_to_check=2000)),
|
|
(' 1234567890 45% 10.50MB/s 0:01:23', dict(bytes_transferred=1234567890, percent=45)),
|
|
('Number of regular files transferred: 1,234', dict(files_transferred=1234)),
|
|
('sending incremental file list', None),
|
|
('', None),
|
|
])
|
|
def test_parse_progress(line, expected):
|
|
assert backup.parse_progress(line) == expected
|
|
|
|
|
|
@pytest.fixture
|
|
def fake_mounts(test_directory, monkeypatch, tmp_path):
|
|
"""Pretend `<mount root>/backup` is a mounted drive with UUID `uuid-1` (device /dev/sdb1). The mount root is a
|
|
temporary directory standing in for /media, so folders can be made on the "drive"."""
|
|
mount_root = (tmp_path / 'media').resolve()
|
|
drive = mount_root / 'backup'
|
|
drive.mkdir(parents=True)
|
|
media = test_directory.resolve()
|
|
monkeypatch.setattr(backup, 'MOUNT_ROOT', mount_root)
|
|
|
|
state = dict(mounted=True, device=2, uuid_device='/dev/sdb1', source='/dev/sdb1', nested=None)
|
|
monkeypatch.setattr(backup, 'is_mount', lambda path: (path == drive and state['mounted'])
|
|
or path == state['nested'] or path == pathlib.Path('/'))
|
|
monkeypatch.setattr(backup, 'device_id', lambda path: {drive: state['device'], media: 1}.get(
|
|
pathlib.Path(path), 3 if pathlib.Path(path) == state['nested'] else 0))
|
|
monkeypatch.setattr(backup, 'uuid_device', lambda uuid: state['uuid_device'] if uuid == 'uuid-1' else None)
|
|
monkeypatch.setattr(backup, 'mount_source', lambda path: {str(drive): state['source']}.get(
|
|
path, '/dev/sdz1' if path == str(state['nested']) else None))
|
|
state['drive'] = drive
|
|
state['root'] = mount_root
|
|
return state
|
|
|
|
|
|
def test_check_backup_target(test_directory, fake_mounts):
|
|
drive, root = str(fake_mounts['drive']), fake_mounts['root']
|
|
assert backup.check_backup_target(drive, 'uuid-1') is None
|
|
|
|
assert backup.check_backup_target(None, None) == 'No backup drive is configured'
|
|
assert backup.check_backup_target('backup', 'uuid-1') == 'backup is not a valid directory'
|
|
assert backup.check_backup_target(f'{drive}/../backup', 'uuid-1') == f'{drive}/../backup is not a valid directory'
|
|
assert backup.check_backup_target('/opt/backup', 'uuid-1') == f'/opt/backup is not under {root}'
|
|
(root / 'wrolpi_temp_onboarding').mkdir()
|
|
fake_mounts['nested'] = root / 'wrolpi_temp_onboarding'
|
|
assert backup.check_backup_target(str(root / 'wrolpi_temp_onboarding'), 'uuid-1') \
|
|
== f'{root}/wrolpi_temp_onboarding cannot be a backup drive'
|
|
fake_mounts['nested'] = None
|
|
|
|
fake_mounts['mounted'] = False
|
|
assert backup.check_backup_target(drive, 'uuid-1') == f'No drive is mounted at {drive}', \
|
|
'an unplugged drive leaves a directory on the system drive'
|
|
fake_mounts['mounted'] = True
|
|
|
|
fake_mounts['device'] = 1
|
|
assert backup.check_backup_target(drive, 'uuid-1') == 'The backup drive is the media drive'
|
|
fake_mounts['device'] = 0
|
|
assert backup.check_backup_target(drive, 'uuid-1') == 'The backup drive is the system drive'
|
|
fake_mounts['device'] = 2
|
|
|
|
fake_mounts['source'] = '/dev/sdc1'
|
|
assert backup.check_backup_target(drive, 'uuid-1') \
|
|
== f'The chosen drive is not mounted at {drive}', 'another drive mounted at the same place'
|
|
fake_mounts['source'] = '/dev/sdb1'
|
|
assert backup.check_backup_target(drive, 'uuid-2') \
|
|
== f'The chosen drive is not mounted at {drive}', 'the drive is not plugged in'
|
|
|
|
|
|
def test_check_backup_target_folder(test_directory, fake_mounts):
|
|
"""The backup may go to a folder on the chosen drive; the drive is checked through the folder's mount point."""
|
|
drive = fake_mounts['drive']
|
|
(drive / 'wrolpi').mkdir()
|
|
assert backup.check_backup_target(str(drive / 'wrolpi'), 'uuid-1') is None
|
|
assert backup.check_backup_target(str(drive / 'not' / 'made' / 'yet'), 'uuid-1') is None, \
|
|
'the backup creates the folder'
|
|
|
|
# The folder is checked through the drive: wrong drive, unplugged drive.
|
|
assert backup.check_backup_target(str(drive / 'wrolpi'), 'uuid-2') \
|
|
== f'The chosen drive is not mounted at {drive}'
|
|
fake_mounts['mounted'] = False
|
|
assert backup.check_backup_target(str(drive / 'wrolpi'), 'uuid-1') == f'No drive is mounted at {drive}/wrolpi', \
|
|
'a folder left on the system drive by an unplugged drive is not written'
|
|
fake_mounts['mounted'] = True
|
|
|
|
# Another drive mounted inside the chosen drive is not the chosen drive.
|
|
(drive / 'other').mkdir()
|
|
fake_mounts['nested'] = drive / 'other'
|
|
assert backup.check_backup_target(str(drive / 'other' / 'wrolpi'), 'uuid-1') \
|
|
== f'The chosen drive is not mounted at {drive}/other'
|
|
|
|
|
|
def test_mount_source(monkeypatch, tmp_path):
|
|
mounts = tmp_path / 'mounts'
|
|
mounts.write_text('/dev/sda1 /media/wrolpi ext4 rw 0 0\n'
|
|
'/dev/sdb1 /media/my\\040backup ext4 rw 0 0\n'
|
|
'/dev/sdc1 /media/stacked ext4 rw 0 0\n'
|
|
'/dev/sdd1 /media/stacked ext4 rw 0 0\n')
|
|
monkeypatch.setattr(backup, 'PROC_MOUNTS', mounts)
|
|
assert backup.mount_source('/media/my backup') == '/dev/sdb1'
|
|
assert backup.mount_source('/media/stacked') == '/dev/sdd1', 'the last mount is the visible one'
|
|
assert backup.mount_source('/media/missing') is None
|
|
|
|
|
|
def test_snapshot_database(test_directory, tmp_path):
|
|
"""The snapshot is a complete, consistent database, even with writes still in the live database's WAL."""
|
|
(test_directory / 'config').mkdir(exist_ok=True)
|
|
live = sqlite3.connect(test_directory / 'config/wrolpi.db')
|
|
live.execute('PRAGMA journal_mode=WAL')
|
|
live.execute('CREATE TABLE t (v TEXT)')
|
|
live.execute("INSERT INTO t VALUES ('in the WAL')")
|
|
live.commit()
|
|
assert (test_directory / 'config/wrolpi.db-wal').stat().st_size > 0
|
|
|
|
drive = tmp_path / 'drive'
|
|
(drive / 'config').mkdir(parents=True)
|
|
# A stale WAL from an older copy must not be applied to the new snapshot.
|
|
(drive / 'config/wrolpi.db-wal').write_bytes(b'stale')
|
|
try:
|
|
backup.snapshot_database(drive)
|
|
finally:
|
|
live.close()
|
|
|
|
assert not (drive / 'config/wrolpi.db-wal').exists()
|
|
assert not (drive / 'config/.wrolpi.db.backup').exists()
|
|
with sqlite3.connect(drive / 'config/wrolpi.db') as copy:
|
|
assert copy.execute('SELECT v FROM t').fetchall() == [('in the WAL',)]
|
|
|
|
|
|
def test_snapshot_database_only_when_changed(async_client, test_directory, tmp_path):
|
|
"""The 2.5 GB snapshot is written again only when the live database, or the drive's copy, has changed."""
|
|
(test_directory / 'config').mkdir(exist_ok=True)
|
|
database = test_directory / 'config/wrolpi.db'
|
|
with contextlib.closing(sqlite3.connect(database)) as live:
|
|
live.execute('PRAGMA journal_mode=WAL')
|
|
live.execute('CREATE TABLE t (v TEXT)')
|
|
live.commit()
|
|
drive = tmp_path / 'drive'
|
|
copy = drive / 'config/wrolpi.db'
|
|
|
|
assert backup.snapshot_database_if_changed(drive) is True
|
|
written = copy.stat().st_mtime_ns
|
|
assert backup.snapshot_database_if_changed(drive) is False
|
|
assert copy.stat().st_mtime_ns == written
|
|
|
|
# Reading opens an empty WAL; nothing changed.
|
|
with contextlib.closing(sqlite3.connect(database)) as reader:
|
|
reader.execute('SELECT * FROM t').fetchall()
|
|
assert backup.snapshot_database_if_changed(drive) is False
|
|
|
|
# A write still in the WAL is a change.
|
|
with contextlib.closing(sqlite3.connect(database)) as live:
|
|
live.execute("INSERT INTO t VALUES ('new')")
|
|
live.commit()
|
|
assert (test_directory / 'config/wrolpi.db-wal').stat().st_size > 0
|
|
assert backup.snapshot_database_if_changed(drive) is True
|
|
with contextlib.closing(sqlite3.connect(copy)) as db:
|
|
assert db.execute('SELECT v FROM t').fetchall() == [('new',)]
|
|
|
|
# The live database changed when the WAL was checkpointed on close.
|
|
backup.snapshot_database_if_changed(drive)
|
|
assert backup.snapshot_database_if_changed(drive) is False
|
|
|
|
# The drive's copy is missing, or replaced by something else.
|
|
copy.unlink()
|
|
assert backup.snapshot_database_if_changed(drive) is True
|
|
copy.write_bytes(b'not the snapshot')
|
|
assert backup.snapshot_database_if_changed(drive) is True
|
|
# Another drive has no snapshot yet.
|
|
assert backup.snapshot_database_if_changed(tmp_path / 'other') is True
|
|
|
|
|
|
@requires_rsync
|
|
@pytest.mark.asyncio
|
|
async def test_run_backup(async_client, test_directory, backup_drive, events_fixture):
|
|
"""Files and configs are copied with their modification times; the database is a snapshot; hard-link directories
|
|
are not copied; nothing on the backup drive is deleted; a second backup copies nothing."""
|
|
files = {
|
|
'videos/channel/video.mp4': b'video',
|
|
'videos/channel/video.info.json': b'{}',
|
|
'config/tags.yaml': b'tags: {}\n',
|
|
'tags/one/video.mp4': b'video',
|
|
'playlists/0001_video.mp4': b'video',
|
|
'videos/channel/.video.mp4.partial': b'vid',
|
|
}
|
|
for path, content in files.items():
|
|
(test_directory / path).parent.mkdir(parents=True, exist_ok=True)
|
|
(test_directory / path).write_bytes(content)
|
|
os.utime(test_directory / path, (1_700_000_000, 1_700_000_000))
|
|
with sqlite3.connect(test_directory / 'config/wrolpi.db') as db:
|
|
db.execute('CREATE TABLE t (v TEXT)')
|
|
db.execute("INSERT INTO t VALUES ('row')")
|
|
# Only on the backup drive: never deleted.
|
|
(backup_drive / 'old.txt').write_text('deleted from the media directory long ago')
|
|
|
|
status = await backup.run_backup()
|
|
|
|
assert status['state'] == 'idle', status['last_error']
|
|
assert status['last_success_at']
|
|
for path in ('videos/channel/video.mp4', 'videos/channel/video.info.json', 'config/tags.yaml'):
|
|
assert (backup_drive / path).read_bytes() == files[path]
|
|
assert (backup_drive / path).stat().st_mtime == 1_700_000_000
|
|
assert not (backup_drive / 'tags').exists()
|
|
assert not (backup_drive / 'playlists').exists()
|
|
assert not (backup_drive / 'videos/channel/.video.mp4.partial').exists()
|
|
assert (backup_drive / 'old.txt').exists()
|
|
with sqlite3.connect(backup_drive / 'config/wrolpi.db') as copy:
|
|
assert copy.execute('SELECT v FROM t').fetchall() == [('row',)]
|
|
assert status['files_transferred'] == 3
|
|
assert status['cancel_requested'] is False
|
|
events_fixture.assert_has_event('backup_completed')
|
|
|
|
status = await backup.run_backup()
|
|
assert status['state'] == 'idle', status['last_error']
|
|
assert status['files_transferred'] == 0
|
|
|
|
|
|
@requires_rsync
|
|
@pytest.mark.asyncio
|
|
async def test_run_backup_excluded_directories(async_client, test_directory, backup_drive):
|
|
"""Excluded folders are not copied, even when their names hold rsync wildcards; what the drive already has in
|
|
them is kept."""
|
|
files = ['Movies/a.mp4', 'Movies [HD]/b.mp4', 'Movies2/c.mp4', 'videos/Movies/d.mp4', 'Mo*/e.mp4']
|
|
for path in files:
|
|
(test_directory / path).parent.mkdir(parents=True, exist_ok=True)
|
|
(test_directory / path).write_bytes(b'file')
|
|
(test_directory / 'config').mkdir(exist_ok=True)
|
|
with sqlite3.connect(test_directory / 'config/wrolpi.db') as db:
|
|
db.execute('CREATE TABLE t (v TEXT)')
|
|
(backup_drive / 'Movies').mkdir()
|
|
(backup_drive / 'Movies/old.mp4').write_bytes(b'old')
|
|
get_sync_config()._config['backup_excluded_directories'] = ['Movies', 'Movies [HD]', 'Mo*']
|
|
|
|
status = await backup.run_backup()
|
|
|
|
assert status['state'] == 'idle', status['last_error']
|
|
assert not (backup_drive / 'Movies/a.mp4').exists()
|
|
assert not (backup_drive / 'Movies [HD]').exists()
|
|
assert not (backup_drive / 'Mo*').exists()
|
|
assert (backup_drive / 'Movies2/c.mp4').read_bytes() == b'file', 'only the folder itself is excluded'
|
|
assert (backup_drive / 'videos/Movies/d.mp4').read_bytes() == b'file', 'excludes are anchored'
|
|
assert (backup_drive / 'Movies/old.mp4').read_bytes() == b'old', 'nothing is deleted'
|
|
|
|
|
|
@requires_rsync
|
|
@pytest.mark.asyncio
|
|
async def test_run_backup_to_folder(async_client, test_directory, backup_drive):
|
|
"""A backup to a folder on the drive creates the folder and mirrors the media directory inside it."""
|
|
(test_directory / 'videos').mkdir()
|
|
(test_directory / 'config').mkdir(exist_ok=True)
|
|
(test_directory / 'videos/video.mp4').write_bytes(b'video')
|
|
with sqlite3.connect(test_directory / 'config/wrolpi.db') as db:
|
|
db.execute('CREATE TABLE t (v TEXT)')
|
|
get_sync_config()._config.update(backup_directory=str(backup_drive / 'wrolpi' / 'library'))
|
|
|
|
status = await backup.run_backup()
|
|
|
|
assert status['state'] == 'idle', status['last_error']
|
|
assert (backup_drive / 'wrolpi/library/videos/video.mp4').read_bytes() == b'video'
|
|
assert (backup_drive / 'wrolpi/library/config/wrolpi.db').is_file()
|
|
assert not (backup_drive / 'videos').exists()
|
|
|
|
|
|
@requires_rsync
|
|
@pytest.mark.asyncio
|
|
async def test_run_backup_stops_when_drive_disappears(async_client, test_directory, backup_drive, monkeypatch):
|
|
"""A drive that is unplugged during a backup is not written to again: rsync is stopped."""
|
|
(test_directory / 'a.txt').write_text('a')
|
|
monkeypatch.setattr(backup, 'MOUNT_CHECK_SECONDS', 0)
|
|
monkeypatch.setattr(backup, 'check_backup_target', lambda directory, uuid: 'No drive is mounted at /media/x')
|
|
monkeypatch.setattr(backup, 'backup_skip_reason', lambda: None)
|
|
|
|
status = await backup.run_backup()
|
|
|
|
assert status['state'] == 'error'
|
|
assert 'No drive is mounted' in status['last_error']
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_run_backup_skips(async_client, test_wrolpi_config):
|
|
status = await backup.run_backup()
|
|
assert status['last_skip_reason'] == 'No backup drive is configured'
|
|
assert status['started_at'] is None
|
|
|
|
get_sync_config()._config.update(backup_directory='/media/backup', backup_device_uuid='uuid-1')
|
|
get_wrolpi_config()._config.update(wrol_mode=True)
|
|
status = await backup.run_backup()
|
|
assert status['last_skip_reason'] == 'WROL Mode is enabled'
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_backup_tick(async_client, test_wrolpi_config, monkeypatch):
|
|
calls = list()
|
|
|
|
async def fake_run_backup():
|
|
calls.append(True)
|
|
|
|
monkeypatch.setattr(backup, 'run_backup', fake_run_backup)
|
|
|
|
backup.request_backup()
|
|
await backup.backup_tick()
|
|
assert calls == [], 'no backup drive'
|
|
assert get_backup_status()['last_skip_reason'] == 'No backup drive is configured'
|
|
|
|
get_sync_config()._config.update(backup_directory='/media/backup', backup_device_uuid='uuid-1')
|
|
await backup.backup_tick()
|
|
assert calls == [], 'nothing requested and no window'
|
|
|
|
backup.request_backup()
|
|
await backup.backup_tick()
|
|
assert calls == [True]
|
|
assert get_backup_status()['requested'] is False
|
|
|
|
get_wrolpi_config()._config.update(wrol_mode=True)
|
|
backup.request_backup()
|
|
await backup.backup_tick()
|
|
assert calls == [True]
|
|
assert get_backup_status()['last_skip_reason'] == 'WROL Mode is enabled'
|
|
|
|
|
|
def test_backup_is_due_hourly_in_window(test_wrolpi_config):
|
|
"""Within the backup window, a backup repeats an hour after the last one finished."""
|
|
get_sync_config()._config.update(backup_window_start='01:00', backup_window_end='05:00')
|
|
local_now = datetime(2026, 11, 5, 3, 0, tzinfo=timezone.utc)
|
|
never = dict(finished_at=None)
|
|
recent = dict(finished_at=local_now.replace(hour=2, minute=30).isoformat())
|
|
earlier = dict(finished_at=local_now.replace(hour=1, minute=55).isoformat())
|
|
assert backup.backup_is_due(never, local_now)
|
|
assert not backup.backup_is_due(recent, local_now)
|
|
assert backup.backup_is_due(earlier, local_now)
|
|
assert not backup.backup_is_due(never, local_now.replace(hour=6)), 'outside the window'
|
|
|
|
|
|
def test_backup_and_sync_windows_are_separate(test_wrolpi_config):
|
|
"""The sync window (a secondary's sync) does not start backups, and the backup window does not start syncs."""
|
|
from wrolpi.sync.lib import sync_is_due
|
|
local_now = datetime(2026, 11, 5, 3, 0, tzinfo=timezone.utc)
|
|
never = dict(finished_at=None)
|
|
|
|
get_sync_config()._config.update(window_start='01:00', window_end='05:00')
|
|
assert sync_is_due(never, local_now)
|
|
assert not backup.backup_is_due(never, local_now)
|
|
|
|
get_sync_config()._config.update(window_start=None, window_end=None,
|
|
backup_window_start='02:00', backup_window_end='04:00')
|
|
assert not sync_is_due(never, local_now)
|
|
assert backup.backup_is_due(never, local_now)
|
|
|
|
|
|
def test_request_backup_cancel(async_client):
|
|
update_backup_status(state='running')
|
|
backup.request_backup_cancel()
|
|
assert get_backup_status()['cancel_requested'] is True
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_last_success_is_kept(async_client, test_directory, test_wrolpi_config, monkeypatch, await_switches):
|
|
"""When the last backup succeeded is saved in sync.yaml, and known again after a restart (its status is in
|
|
memory)."""
|
|
async def fake_rsync(command, target, uuid):
|
|
return 0, []
|
|
|
|
monkeypatch.setattr(backup, 'backup_skip_reason', lambda: None)
|
|
monkeypatch.setattr(backup, 'check_backup_target', lambda directory, uuid: None)
|
|
monkeypatch.setattr(backup, '_run_rsync', fake_rsync)
|
|
monkeypatch.setattr(backup, 'snapshot_database', lambda target: None)
|
|
drive = test_directory / 'drive'
|
|
get_sync_config()._config.update(backup_directory=str(drive), backup_device_uuid='uuid-1')
|
|
|
|
status = await backup.run_backup()
|
|
await await_switches()
|
|
|
|
assert status['last_success_at']
|
|
saved = yaml.safe_load((test_directory / 'config/sync.yaml').read_text())
|
|
assert saved['backup_last_success_at'] == status['last_success_at']
|
|
|
|
# A restart: the status is new, the config is imported.
|
|
update_backup_status(last_success_at=None)
|
|
get_sync_config().import_config()
|
|
assert get_backup_status()['last_success_at'] == status['last_success_at']
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_failed_backup_keeps_last_success(async_client, test_directory, test_wrolpi_config, monkeypatch):
|
|
async def failing_rsync(command, target, uuid):
|
|
return 12, ['rsync: connection unexpectedly closed']
|
|
|
|
monkeypatch.setattr(backup, 'backup_skip_reason', lambda: None)
|
|
monkeypatch.setattr(backup, '_run_rsync', failing_rsync)
|
|
get_sync_config()._config.update(backup_directory=str(test_directory / 'drive'), backup_device_uuid='uuid-1',
|
|
backup_last_success_at='2026-11-01T02:00:00+00:00')
|
|
|
|
status = await backup.run_backup()
|
|
|
|
assert status['state'] == 'error'
|
|
assert get_sync_config().backup_last_success_at == '2026-11-01T02:00:00+00:00'
|