wrolpi/app/src
lrnselfreliance 6cd233dce6
Sync and backup: a secondary WROLPi and a backup drive copy the library (#957)
* A secondary WROLPi copies its primary's library over HTTP

A WROLPi with role "secondary" pulls from its primary: a manifest of every
FileGroup's files (/api/sync/manifest), then each missing or changed file
from the primary's Caddy /media/ (resumable, written to a hidden partial and
renamed with the primary's mtime), a refresh, then the library configs,
imported as the primary wrote them.  wrolpi.yaml contributes only its library
settings.  Files deleted on the primary are kept.

A secondary never saves the synced configs and never downloads.  Its API
answers 503 SYNCING while a sync runs, and every page warns that it is a
copy.  It syncs in a daily window or on "Sync now"; WROL Mode stops it.

On a Pi 4 (no AES instructions) the client uses TLS 1.2 ChaCha20, which
pulls ~110 MB/s from Caddy versus ~41 MB/s for rsync over ssh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Sync has its own /admin tab; a sync's reads from the primary are bounded

The sync settings and progress move from Settings (and the component
gallery, which is for library components) to /admin/sync.

A download never writes past the size in the primary's manifest, and configs,
listings, and manifest pages are capped.  A primary address may not carry a
user name or password (it is logged and shown).  While a sync runs, the API
answers only the exact requests that show its progress or stop it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* A sync keeps the secondary's replaced configs in today's config backup

The first copy of the day is kept, like a save, so a config backup import
undoes a sync's config changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Docker's Caddy logs every request

`docker compose logs -f web` shows each request with its status, size and
duration, e.g. a secondary's /media/ downloads during a sync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* A sync reports its progress as it downloads, and refuses quietly

The status updated only when a page of 500 FileGroups finished, so a page of
large files looked stuck; it now updates every 2 seconds.  Requests refused
while syncing are answered by the middleware instead of raised, so the UI's
polling no longer logs an error each time.  The primary's version is stored
without its trailing newline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* A WROLPi can back up to a second drive; sync settings live in sync.yaml

The Sync tab's Backup Drive copies the media directory to a second mounted
drive with rsync during the sync window (hourly), or on "Back Up Now".
Nothing on the backup drive is deleted.  The Tags and playlists directories
are left out (WROLPi rebuilds them from the configs), and the database is
written as an SQLite snapshot rather than copied as files, so the drive can be
mounted as the media directory and start.  A backup is written only while the
chosen drive (by UUID) is mounted where it was chosen, and is not the media or
system drive; rsync is stopped if the drive disappears.  Not in Docker, which
cannot reach other drives.

Role, primary, window and backup drive move from wrolpi.yaml to sync.yaml:
they belong to this machine, and are never copied from a primary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* The Sync tab shows the configured backup drive

getSettings copies each setting by name and did not list the backup drive,
so the Sync tab showed "None" while one was configured.  It also dropped
`version`.  A test now fails for any setting the API sends that getSettings
drops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* A backup can go to a folder on its drive, and remembers its last success

The Backup Drive panel takes a folder on the chosen drive.  The drive is
checked through the folder's mount point: the folder is written only when the
drive with the saved UUID is mounted there, so a folder left on the system
drive by an unplugged drive, or on another drive mounted inside the chosen
one, is refused.  A missing folder is created once the drive is checked.

When the last backup succeeded is saved in sync.yaml, so it is known after a
restart.  "Stop Backup" is disabled while the database snapshot is written (it
runs to the end), and a finished backup clears the request to stop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* The backup drive has its own window

The sync window was shown in the Sync panel but also scheduled the backup
drive.  The Backup Drive panel now has its own window (sync.yaml
backup_window_start/end), saved with the drive; the sync window is shown,
and used, only for a secondary's sync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Sync and the backup drive can each exclude folders

A secondary or a backup drive may be smaller than the primary's library.
Each Sync tab panel has its own excluded folders in sync.yaml, chosen in
a folder browser: a secondary browses its primary's folders (through
Caddy), the backup drive browses this WROLPi's media directory.  Excluded
folders are never copied; what was copied before is kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* The folder browser does not offer lost+found

Every ext4 drive has a root-only lost+found folder; a sync and a backup
never copy it, so it is not offered for exclusion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* A sync never writes into config or tags, whatever their case

On a media drive that ignores case (exFAT, NTFS, macOS), a manifest path
like Config/wrolpi.db is the real config folder. The protected and
unlisted top-level folders are now compared case-insensitively in the
manifest, in resolve_destination, in excluded-folder validation, and in
both folder listings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* A backup writes the database snapshot only when the database changed

Every backup rewrote the whole database snapshot (2.5 GB on a busy
WROLPi), even on the window's idle hourly runs. The snapshot is now
skipped when the live database (its size and mtime, and those of a
non-empty WAL) and the drive's copy are both unchanged since the last
snapshot. The record is in memory, so the first backup after a restart
writes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Remove the unused save_sync_config switch

Nothing activated it; SyncConfig saves directly.  The test server's
log_message override is called by http.server, so it goes in the
vulture baseline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:53:45 -07:00
..
api controller: VNC toggle, gated on the Desktop running 2026-07-29 18:10:05 -06:00
components Sync and backup: a secondary WROLPi and a backup drive copy the library (#957) 2026-11-11 10:53:45 -07:00
contexts app: status and settings contexts say whether their first poll landed (#734) 2026-09-26 17:25:46 -06:00
hooks Sync and backup: a secondary WROLPi and a backup drive copy the library (#957) 2026-11-11 10:53:45 -07:00
themes review: the color-scheme half of this was already there 2026-08-08 18:05:54 -06:00
types ui: the last Semantic UI file, and the stylesheet with it 2026-08-08 18:04:03 -06:00
api.errorAsEmpty.test.js app: api helpers report failure instead of returning an empty value (#736) 2026-09-26 17:47:32 -06:00
api.getSettings.test.js Sync and backup: a secondary WROLPi and a backup drive copy the library (#957) 2026-11-11 10:53:45 -07:00
api.js Sync and backup: a secondary WROLPi and a backup drive copy the library (#957) 2026-11-11 10:53:45 -07:00
App.css 3D previews load ASCII STLs, stand upright, and fit the preview modal (#952) 2026-11-07 15:33:58 -07:00
App.js Sync and backup: a secondary WROLPi and a backup drive copy the library (#957) 2026-11-11 10:53:45 -07:00
App.test.js Render unknown URLs as a 404 page inside the app layout (#805) 2026-10-09 15:00:36 -06:00
babel-pins.test.js build: put the @babel pins back, and make them fail loudly if dropped again (#653) 2026-08-08 18:54:34 -06:00
DashboardPage.js Sync and backup: a secondary WROLPi and a backup drive copy the library (#957) 2026-11-11 10:53:45 -07:00
DashboardPage.test.js A new database is refreshed on startup (#827) 2026-10-13 16:07:04 -06:00
Events.js Sync and backup: a secondary WROLPi and a backup drive copy the library (#957) 2026-11-11 10:53:45 -07:00
Events.test.js Notifications without a URL are not links (#823) 2026-10-12 08:37:26 -06:00
import-cycles.test.js fix: break the import cycle that put useCalculators in the TDZ 2026-08-08 18:05:47 -06:00
index.css ui: the last Semantic UI file, and the stylesheet with it 2026-08-08 18:04:03 -06:00
index.js Fixing changes before merge. 2023-02-08 21:18:51 -07:00
logo.svg Renaming examples to avoid grouping of pdf with ebooks. Reworking URLs for media files to avoid escaping url safe characters. 2023-03-29 13:46:39 -06:00
no-semantic-ui.test.js ui: the last of Semantic UI, in name and in dead CSS 2026-08-08 18:05:54 -06:00
reportWebVitals.js deps: cachetools 7 and the current web-vitals API (#771) 2026-10-03 11:04:16 -06:00
reportWebVitals.test.js deps: cachetools 7 and the current web-vitals API (#771) 2026-10-03 11:04:16 -06:00
setupTests.js 3D previews load ASCII STLs, stand upright, and fit the preview modal (#952) 2026-11-07 15:33:58 -07:00
Tags.cy.js test: the component suite's twelve failures, none of them about the component 2026-08-08 18:05:54 -06:00
Tags.js files: left-align the preview toolbar (#698) 2026-09-03 16:03:17 -06:00
Tags.test.js ui: the last of Semantic UI, in name and in dead CSS 2026-08-08 18:05:54 -06:00
test-fixtures.js Sync and backup: a secondary WROLPi and a backup drive copy the library (#957) 2026-11-11 10:53:45 -07:00
test-fixtures.test.js ui: the last of Semantic UI, in name and in dead CSS 2026-08-08 18:05:54 -06:00
test-utils.js style: spell it "color" everywhere 2026-08-08 18:05:50 -06:00
test-utils.test.js style: spell it "color" everywhere 2026-08-08 18:05:50 -06:00