wrolpi/controller/api/sync.py
lrnselfreliance ea7cc4dec2
Some checks are pending
Backend Tests / api-tests (core) (push) Waiting to run
Backend Tests / api-tests (files) (push) Waiting to run
Backend Tests / api-tests (media) (push) Waiting to run
Backend Tests / api-tests (rest) (push) Waiting to run
Backend Tests / api-contract-tests (push) Waiting to run
Backend Tests / api-migrations-and-start (push) Waiting to run
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Cypress Tests / cypress-component-tests (push) Waiting to run
Cypress Tests / cypress-e2e-tests (push) Waiting to run
Dead Code / vulture (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
The Controller can make a secondary a copy of its primary with rsync (#992)
* The Controller can make a secondary a copy of its primary with rsync

The second part of replacing the sync.  The Controller runs it, because the secondary's API is stopped while its
library is replaced: checks, a database snapshot on the primary, stop the API, rsync every file from the primary's
daemon (ignored directories included), fetch the snapshot and put it in the database's place, start the API, purge.

- Files the primary deleted or changed are moved to a dated folder in .sync-trash, and removed after
  delete_after_days (a new sync.yaml setting, 30 by default; none never deletes), or sooner when the drive is nearly
  full.
- A primary of another version or database migration, or with its media unmounted, is refused before anything is
  touched.  The database is replaced last, so a sync that stops earlier leaves one that works.
- On a Raspberry Pi and Debian rsync runs as the wrolpi user; in Docker in a helper container made from the api
  container.  A Controller that starts after an interrupted sync starts the API.
- When a secondary's API starts it takes the library settings of the primary's wrolpi.yaml, which the sync left.

Reached only through the Controller's /api/sync/run for now; the existing sync and its schedule are unchanged.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* The sync copies no link that leaves the library, and root follows none in the media directory

Security review of the Controller's sync:

- An exclusion that ends in a slash matches only a directory.  A file or link named config/ssl, tags or
  config/backup on the primary would have replaced this machine's directory of that name.  The exclusions now match
  whatever has the name.
- Links were copied as they are.  One that points outside the primary's library would, on the secondary, point at
  the secondary's own files, which the web server then serves.  Only links that stay inside the library are copied,
  and no device files.
- The Controller is root, in a media directory that belongs to another user.  It no longer writes, reads, replaces
  or purges through a symbolic link there.

Found on the way: the directory the Controller made for the incoming database belonged to root, so rsync, which
runs as the library's owner, could not have written into it.  It now belongs to the media directory's owner.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* A sync ends when its primary stops answering, and makes room before it copies

Review found:

- rsync waits forever by default.  A primary that lost power or its link during a copy left the secondary's API
  stopped until someone cancelled.  rsync now gives up after 30 seconds without a connection and 5 minutes without
  data, which ends the run and starts the API.
- The trash was only purged after a run that succeeded.  The trash is what fills the drive, and a full drive fails
  every run, so it never got its turn.  It is now also purged before the copy.
- A value in sync.yaml that could not be used raised after the run lock was taken and before the handlers that
  release it, leaving "A sync is already running" until the Controller restarted.
- The playlists directory left out of the copy was this machine's, not the primary's, whose hard links would have
  arrived as full copies.  The primary's wrolpi.yaml is now fetched before the copy, and both are left out.
- A primary whose media directory is elsewhere is refused: its database names every file by its full path.
- In Docker rsync is its own container, which outlives a Controller that stops.  A Controller that starts after an
  interrupted sync now removes it before it starts the API.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-18 20:21:55 -07:00

36 lines
1.1 KiB
Python

"""
Sync API endpoints for WROLPi Controller: a secondary WROLPi copying its primary (see controller.lib.sync).
The Controller answers these because the API is stopped while a sync runs.
"""
import asyncio
from fastapi import APIRouter, HTTPException
from controller.lib import sync
router = APIRouter(prefix="/api/sync", tags=["sync"])
@router.get("/status")
async def sync_status() -> dict:
"""The progress and outcome of this secondary's sync."""
return sync.get_status()
@router.post("/run", status_code=202)
async def sync_run() -> dict:
"""Copy the primary now."""
if reason := await asyncio.to_thread(sync.skip_reason):
raise HTTPException(status_code=409, detail=reason)
# Kept referenced, or the task could be collected while the sync runs.
router.sync_task = asyncio.create_task(asyncio.to_thread(sync.run_sync))
return {"success": True}
@router.post("/cancel", status_code=202)
async def sync_cancel() -> dict:
"""Stop the running sync; the API is started again."""
sync.request_cancel()
return {"success": True}