* The Controller can make a secondary a copy of its primary with rsync
The second part of replacing the sync. The Controller runs it, because the secondary's API is stopped while its
library is replaced: checks, a database snapshot on the primary, stop the API, rsync every file from the primary's
daemon (ignored directories included), fetch the snapshot and put it in the database's place, start the API, purge.
- Files the primary deleted or changed are moved to a dated folder in .sync-trash, and removed after
delete_after_days (a new sync.yaml setting, 30 by default; none never deletes), or sooner when the drive is nearly
full.
- A primary of another version or database migration, or with its media unmounted, is refused before anything is
touched. The database is replaced last, so a sync that stops earlier leaves one that works.
- On a Raspberry Pi and Debian rsync runs as the wrolpi user; in Docker in a helper container made from the api
container. A Controller that starts after an interrupted sync starts the API.
- When a secondary's API starts it takes the library settings of the primary's wrolpi.yaml, which the sync left.
Reached only through the Controller's /api/sync/run for now; the existing sync and its schedule are unchanged.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
* The sync copies no link that leaves the library, and root follows none in the media directory
Security review of the Controller's sync:
- An exclusion that ends in a slash matches only a directory. A file or link named config/ssl, tags or
config/backup on the primary would have replaced this machine's directory of that name. The exclusions now match
whatever has the name.
- Links were copied as they are. One that points outside the primary's library would, on the secondary, point at
the secondary's own files, which the web server then serves. Only links that stay inside the library are copied,
and no device files.
- The Controller is root, in a media directory that belongs to another user. It no longer writes, reads, replaces
or purges through a symbolic link there.
Found on the way: the directory the Controller made for the incoming database belonged to root, so rsync, which
runs as the library's owner, could not have written into it. It now belongs to the media directory's owner.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
* A sync ends when its primary stops answering, and makes room before it copies
Review found:
- rsync waits forever by default. A primary that lost power or its link during a copy left the secondary's API
stopped until someone cancelled. rsync now gives up after 30 seconds without a connection and 5 minutes without
data, which ends the run and starts the API.
- The trash was only purged after a run that succeeded. The trash is what fills the drive, and a full drive fails
every run, so it never got its turn. It is now also purged before the copy.
- A value in sync.yaml that could not be used raised after the run lock was taken and before the handlers that
release it, leaving "A sync is already running" until the Controller restarted.
- The playlists directory left out of the copy was this machine's, not the primary's, whose hard links would have
arrived as full copies. The primary's wrolpi.yaml is now fetched before the copy, and both are left out.
- A primary whose media directory is elsewhere is refused: its database names every file by its full path.
- In Docker rsync is its own container, which outlives a Controller that stops. A Controller that starts after an
interrupted sync now removes it before it starts the API.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
* A WROLPi serves its library over rsync, and makes a database snapshot on request
The first part of replacing the sync: a secondary will copy its primary's whole media directory with rsync and
replace its database with the primary's, instead of downloading the files of FileGroups over HTTP (which never
copies an ignored directory, such as the map's).
- An rsync daemon serves the media directory read-only on port 8873, on every WROLPi. It never serves the TLS keys,
the cookies, or the database in use. On a Raspberry Pi and Debian it is the wrolpi-rsyncd unit; in Docker the
Controller runs it in a helper container made from the api container (its image, user and volumes), so the media
directory is still configured in one place. All three read controller/rsyncd.conf.
- POST /api/sync/snapshot asks for a snapshot of the database, which the daemon serves from config/.sync. An
unchanged database is not written again.
- /api/sync/info also says the database's migration and whether the media directory is mounted.
Nothing uses these yet; the existing sync is unchanged.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
* The rsync daemon follows no links, and no snapshot is made in WROL Mode
- A client that asked the daemon to follow symbolic links could read what a link in the library points at. A
relative link to config/ssl was read this way although that directory is left out of the module. The daemon now
refuses the three options that follow links; a link is sent as a link.
- A database snapshot is a new file on the primary's drive, so WROL Mode refuses it, as it does a backup. A snapshot
made earlier is still served.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
* No rsync helper from an image without rsync, and the helper follows the api container
Review found:
- A helper made from an api image built before rsync was added exits at once, and Docker restarted it forever while
the Controller reported it created. The image is now looked at first; no helper is made from one without rsync,
an older helper is left serving, and the reason is logged once.
- The helper was only checked when the Controller started, so a recreated or rebuilt api container left it stale.
It is now checked every minute.
- A snapshot cut off by its process being killed stayed marked as running and held back every later request.
- The snapshot's half-written temporary file was served by the daemon.
- The comment on the daemon's refused options claimed more than they give: a link that is part of the requested
path is still followed.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
Nothing started the hotspot since hotspot_on_startup moved out of the
API, and NetworkManager does not autoconnect the Hotspot connection.
The Controller now starts it once per boot unless hotspot.on_startup is
false; a legacy wrolpi.yaml hotspot_on_startup: false is migrated.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Hotspot captive portal: welcome page with the WROLPi's address
Devices that join the hotspot probe a fixed plain-HTTP URL to check for
Internet access. NetworkManager's shared-mode dnsmasq now resolves those
probe hostnames (and `wrolpi`) to the hotspot address via an include the
Controller writes when the hotspot starts, the probes land on the Controller
(port 80), and are redirected to /portal: "This is your WROLPi, its address is
10.42.0.1, open WROLPi / open the Controller".
Only the probe hostnames are hijacked so the hotspot keeps forwarding
everything else when the WROLPi has an uplink. The include is corrected and
the connection re-activated once if NetworkManager picked a non-default
subnet. Writing the include is best-effort; the hotspot starts regardless.
`hotspot.captive_portal` (controller.yaml, default true) toggles the probe
redirects and the include; like the other hotspot settings it applies on the
next hotspot start, so saving while the hotspot is up reports
restart_required and both Controller UIs say so. The status endpoint gains
the hotspot address and portal URL, shown in both UIs. repair.sh only
creates the include directory so a disabled setting survives upgrades.
Native only: Docker has no hotspot.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: address review
* React: re-read the hotspot address whenever the HotspotToggle starts or
stops the hotspot, matching the fallback UI's refresh-on-status.
* A WPA3 start that reverts to WPA2 leaves the hotspot up, so it now gets
the captive portal address correction too.
* The corrective `nmcli connection up` logs a warning on a non-zero exit;
dnsmasq would otherwise keep the old address silently.
* HotspotStatusResponse.captive_portal defaults to true like everywhere else.
* Trim design narrative from the module docstring and React comments.
* The include-before-activate test now observes the include at the moment
of the `nmcli ... hotspot` call instead of inferring from call order.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: never forward any record type for probe hosts
`address=` only synthesizes A/AAAA answers. iOS also asks for the HTTPS
record type; dnsmasq forwarded that upstream, Apple's answer aliased to
captive.g.aaplimg.com, and the phone fetched the genuine "Success" from
Apple instead of the portal. Each probe host now also gets `local=`, so
dnsmasq answers every type itself, and the CDN alias is a probe host too.
Observed with dnsmasq query logging on a hotspot with an eth0 uplink.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: let acknowledged clients finish so the OS offers "Done"
A phone's sign-in sheet only shows "Done" once the OS re-probes and gets
the genuine success reply. The Controller now remembers each client (by
hotspot IP) that has loaded the portal page and answers its later probes
with the reply each OS expects (Apple "Success" HTML, Android 204, Windows
"Microsoft Connect Test", ...). The set is cleared when the hotspot
starts so a new session shows the page again.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: acknowledge on Continue, not on view
iOS fetches the portal page in the background before opening the sheet,
then re-probes after the sheet's own navigation. Acknowledging on view
made that re-probe return "Success" as the sheet's content. Phones only
re-check the network (and offer "Done") after a full-page navigation, so
the welcome page now has a Continue button leading to /portal/continue,
which is the only place a client is acknowledged. The Continue page says
to tap Done and links to WROLPi and the Controller.
Also hijack clients3.google.com (older Android probe host).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* help: bump wrolpi-help to the captive portal and Speed Test docs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: comments describe the Continue design; clear acks on stop
Three comments still described acknowledging on view, which c72a7f7b
removed. Acknowledged clients are now also cleared when the hotspot
stops, so a lease reused in a later session sees the portal page.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The RPi 5 fan connector exposes a tachometer via the pwmfan hwmon
device. Report it as fan_rpm in cpu_stats and show it in both the
Controller fallback UI and the React Status page. Hidden when no fan
is present.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Controller listed every wrolpi-* service in one flat table, which was
confusing for new users who couldn't tell always-on infrastructure from
one-off/dev/maintenance units.
Services are now split into two groups computed once in the backend and
exposed as `service.group`:
* Core Services - persistent services enabled at boot (api, app, caddy,
kiwix, ...), the normal running system.
* Optional & Maintenance - one-off/dev/maintenance tasks (bootstrap, repair,
upgrade, *-dev, discovered wrolpi-*), plus persistent
services the user has disabled at boot (e.g. Samba with
no shares configured).
Rule: core iff kind == "persistent" and enabled; otherwise optional. Group
membership only changes when the user toggles Boot, never on start/stop or
polling, so rows stay put. Within each group, failed/stopped services sort
first so problems stay visible.
Feature parity across all three render paths: the React ControllerPage, the
server-side Jinja template, and the client-side poll refresh. Docker
containers have no group field and fall back to a single ungrouped list.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
During onboarding the media drive is not mounted and ca.crt does not exist
yet (or still holds only a throwaway pre-drive state). The cert banner still
offered Download WROLPi Root CA, which 404'd. Gate the button and HTTPS
cert-setup banner on drive mounted, ca.crt present, and repair not running
so an existing CA on the drive can be served after mount/repair.
Hotspot was reported as "not supported" on x86 machines (Ubuntu/Debian PCs)
because the Controller only watched the hardcoded default wlan0 device, while
predictable interface naming produces names like wlp2s0. The Settings page
saved hotspot_device to wrolpi.yaml, which the Controller never read.
The Controller now owns hotspot settings in controller.yaml:
* New endpoints: GET/POST /api/hotspot/settings, GET /api/hotspot/devices
(WiFi interfaces discovered via nmcli).
* Legacy hotspot_* values in wrolpi.yaml are migrated into controller.yaml
once on startup, unless controller.yaml already sets them.
* The Hotspot SSID/password/device fields moved from the Settings page to the
Control page next to the Hotspot toggle; the device is now a dropdown of
the WiFi devices found on the machine.
* The Controller fallback UI gets the same hotspot settings form for parity.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Introduce an onboarding system in the Controller that guides users
through selecting and mounting their primary media drive. This is
the fallback UI flow used when the main React app is unavailable.
Key changes:
- Add onboarding API endpoints and schemas (controller/api/onboarding.py)
- Add onboarding logic to detect drives, handle already-mounted drives
(needed for RPi which auto-mounts), and save primary drive selection
to the controller config when no mounts are configured
- Extend the Controller's index.html template with the onboarding UI
- Skip root cert creation until onboarding is complete
- Update first_startup.sh to support the onboarding/repair flow
- Add comprehensive tests for onboarding logic and API