A WROLPi drive can move between hosts and across deployment styles
(Portable, RPi, Debian). /etc/fstab is host-specific and on the Portable
image it's part of a read-only squashfs, so persisting secondary mounts
there was both wrong-host and lost-on-reboot. This change makes
/media/wrolpi/config/fstab.yaml the single source of truth for WROLPi-
managed mounts and introduces a new wrolpi-mounts.service that reconciles
live state to match.
Architecture:
- The Controller writes intent (fstab.yaml) but never calls mount(8) or
umount(8) itself. Every mutation triggers `systemctl restart
wrolpi-mounts.service`; the Controller then verifies the outcome via
the same MountExecutor abstraction the service uses.
- wrolpi-mounts.service runs a thin entry point that constructs a real
Reconciler with a SubprocessMountExecutor and calls apply(). The
reconciler mounts what's in fstab.yaml but missing from the live state,
and unmounts paths it previously claimed that are no longer in
fstab.yaml. A managed-set file in /run/wrolpi-mounts/managed prevents
the reconciler from ever touching user-mounted paths it didn't own.
- One-time migration (marker-gated) lifts both /etc/fstab WROLPi entries
and any controller.yaml drives.mounts entries into fstab.yaml.
/etc/fstab is backed up to /etc/fstab.wrolpi-migration.backup;
migrated lines are commented in place with a "migrated to fstab.yaml:"
prefix so a host admin can hand-recover.
Test approach: dependency injection in place of patching. The
MountExecutor protocol has SubprocessMountExecutor (production) and
FakeMountExecutor (in-memory, records calls). The Reconciler accepts an
executor in its constructor, so the 20 reconciler tests + 13 migration
tests + 18 fstab_yaml tests use real files in tmp_path with zero
mock.patch calls. test_disks_api.py drops to mocks only for the
host-touching read helpers; the mount/unmount paths now run the real
Reconciler against the fake executor.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
persistence.conf was writing `union,source=...` entries. On the first boot
that did not matter because the bootstrap performed a real `mount --bind`
itself; from the second boot onward, however, live-boot's initramfs reads
persistence.conf and applies the entries as OverlayFS union mounts.
PostgreSQL does not support OverlayFS — switching to `bind,source=...`
makes the in-band mount and the initramfs mount match and avoids potential
data corruption.
The bootstrap script hardcoded Postgres version 15 (cluster check, create,
service-start), inconsistent with the chroot hook which already derives it
dynamically. Switched to discovery via `/usr/lib/postgresql/<ver>` so a
postgresql-16 bump no longer requires touching this script.
The tippecanoe source tarball was the only externally-fetched artifact in
the chroot hook without a SHA256 pin. Added one (computed from upstream
v2.79.0).
`RequiresMountsFor` covered only `/usr/lib/live/mount/medium`; live-boot on
newer releases mounts at `/run/live/medium`. Listed all three paths the
bootstrap script itself searches so systemd orders the unit after whichever
mount actually exists.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The chroot hook only ran `npm install`, leaving /opt/wrolpi/app/build/
missing. wrolpi-app.service runs `serve build -l 3000 ...`, so it started
fine but returned 404 for every path. RPi-OS / Debian installs handle this
via repair.sh at first boot; we bake the build into the squashfs so the
live image works offline immediately.
The Controller's `is_primary_drive_mounted()` checks for
/media/wrolpi/config/controller.yaml — without it the Controller renders
its onboarding wizard and offers the persistence partition (sdb3, mounted
at /mnt/persistence) as an unconfigured candidate, which would corrupt the
running system if selected. Touch an empty controller.yaml after creating
/media/wrolpi/config so the Controller treats the drive as configured.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LAST_END parser was assigning to a scalar (not MAX), so on iso-hybrid layouts
where the small ESP is listed after the much larger boot partition, sfdisk
got a start sector inside an existing partition. Also sfdisk -d pads values
with whitespace (`start= 64,`), which broke a naive awk tokenizer.
Switched to a sed parser that strips the padding, and computes the true
MAX(start+size).
`systemctl enable --now wrolpi-api ...` deadlocked: the runtime units have
implicit `After=wrolpi-portable-bootstrap.service` via this unit's `Before=`,
so --now waited forever for them to start while they waited for bootstrap to
finish. Split enable from start; use --no-block so systemd queues the
starts after we exit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The bootstrap script can take 30-60s on first boot (partition creation,
Postgres cluster init, Alembic migrations). Plymouth is up that whole time
since lightdm waits for bootstrap to finish. Push a status line via
'plymouth display-message --text=...' at each major milestone so the user
sees what's happening instead of staring at a frozen splash.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
wrolpi-portable-bootstrap.service runs at every boot (idempotent throughout)
and owns the persistence partition lifecycle:
- First boot: creates an ext4 partition labeled 'persistence' in free space
at the end of the boot drive (sfdisk --append --no-reread --force because
the live medium is mounted from the same disk), bind-mounts source dirs
over /var/lib/postgresql, /etc/postgresql, and /media/wrolpi, creates a
fresh Postgres cluster, and runs initialize_api_db.sh.
- Subsequent boots: live-boot's persistence machinery applies the bind
mounts in initramfs (kernel cmdline now includes 'persistence
persistence-label=persistence persistence-media=removable'); the script
mostly no-ops apart from regenerating the leaf TLS cert.
After bootstrap completes the script enables and starts wrolpi-api,
wrolpi-app, and wrolpi-kiwix — these stay disabled in the chroot hook so
they cannot start until the DB and configs are ready (avoiding the
"half-finished state with empty configs" failure mode).
Also adds parted, gdisk, dosfstools, and exfatprogs to the package list so
the live system can fully diagnose / repair / partition without internet.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The chroot hook applies chattr +i to /opt/wrolpi-blobs/*, which then
prevents rm -rf from wiping BUILD_DIR on subsequent runs. lb's
chroot_includes_after_packages rsync then fails with "Operation not
permitted" trying to overwrite the leftovers. Strip +i before rm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops straight into the XFCE session at boot. The wrolpi password is still
required for sudo and screen unlock.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- desktop.list.chroot: add user-setup (was implicitly provided by the
installer ISO's --debian-installer dep tree; the live-only build lost it
and live-config's user-creation step silently exits without it, leaving
the squashfs with no login-capable account).
- desktop.list.chroot: add plymouth-themes for non-Debian boot splash.
- 9999-wrolpi.hook.chroot: pre-create the wrolpi user (UID 1000) with
password 'wrolpi' and desktop groups; live-config sees the existing user
and skips its own user creation, preserving our chosen password.
- 9999-wrolpi.hook.chroot: set hostname to 'wrolpi', /etc/issue banner,
Plymouth theme to 'text' (no Debian logo), and a LightDM greeter banner
saying 'WROLPi Portable'.
- build.sh: add live-config.username=wrolpi, live-config.hostname=wrolpi,
live-config.user-fullname=WROLPi to bootappend-live so live-config's
other scripts (sudo, lightdm autologin) target the wrolpi user.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- build.sh: strip whitespace from VERSION before interpolating into lb config
metadata; wrap lb build in set -o pipefail so a build failure is not masked
by tee's success exit code.
- Hook: pin SHA256 for Deno and pmtiles downloads so tampered release
artifacts cannot be baked into the squashfs unnoticed.
- Hook: derive Postgres cluster version dynamically from pg_lsclusters
instead of hardcoding 15, so this stays correct when the package list is
bumped past postgresql-15.
- Hook: fix pmtiles URL (Linux_amd64 -> Linux_x86_64); the v1.30.1 release
ships the asset as x86_64 and the old URL returned 404.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New debian-live-usb/ directory produces a live ISO that boots into a full
WROLPi desktop on any x86 PC without installing to the host disk. Distinct
from debian-live-config/ (installer ISO). Phase 1: live boot only; Phase 2
will add persistence partition for the Postgres DB and media directory.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Greptile flagged that if the session is in an aborted tx (e.g. the
external commit was followed by a failed statement before our error
fired), Postgres rejects DDL on the session and the committed work
table orphans. Fall back to a fresh connection in that case. Adds
a regression test that deterministically reproduces the orphan.
compare_file_groups created a TEMP TABLE ... ON COMMIT DROP on the
session. In production each get_db_session() returns a fresh connection,
but tests share one session across every caller, so any concurrent
get_db_session(commit=True) commits this session too and drops the work
table before the batch insert runs. Use a uniquely-named regular table
with explicit DROP IF EXISTS in finally instead.
The extension is hosted in a sibling repo and
is distributed off-store. Each WROLPi release ships the prebuilt artifacts
under wrolpi/static/extensions/ and serves them at /api/extensions/<file>:
- wrolpi-chrome.zip Load-Unpacked sideload for Chrome / Brave / Edge
- wrolpi-firefox.xpi Mozilla-signed (unlisted self-distribution),
one-click install in any Firefox
- versions.json Per-browser version metadata for the install page
Backend (modules/archive/api.py, wrolpi/root_api.py):
- POST /api/archive/upload accepts an optional repeated `tagNames`
multipart form field; the upload switch handler applies tags to the
new FileGroup via FileGroup.set_tags(). Unknown tag names are auto-
created inside a per-name SAVEPOINT so concurrent inserts on the same
novel name don't abort the transaction (Tag.name has unique=True).
- GET /api/extensions returns metadata (per-file availability + size +
versions.json contents) for the install page.
- GET /api/extensions/<filename> serves the binary, whitelisted by
filename. Firefox .xpi is served as application/x-xpinstall WITHOUT a
Content-Disposition: attachment header, so Firefox triggers its
add-on install handler instead of saving the file.
Frontend (app/src/):
- New /admin/extension page (linked from AdminRoute) detects the user's
browser via navigator.userAgent and renders an install card per
Firefox / Chromium / Other. Pulls metadata from /api/extensions to
show file size and version, and gracefully reports 'Not yet
available' when binaries are missing.
- Dashboard's Getters component supports a deep link from the
extension's 'Configure Download' button:
/?downloader=<archive|video|...>&download_url=<u1>&download_url=<u2>
On mount, the matching downloader form auto-opens with URLs pre-
filled. Query params are cleared on close so a refresh doesn't
re-open the modal.
- DownloadMenu and the per-downloader forms accept an `initialUrls`
seed via the existing `download` prop pattern, joined with newlines
for the `urls` textarea (and first URL for single-URL forms).
Behavior is fully backwards compatible: pre-existing API callers can
still POST to /api/archive/upload without `tagNames`, and the
DashboardPage modal continues to open via the existing button click
when no deep-link query params are present.