Commit graph

2039 commits

Author SHA1 Message Date
Roland
f89b131ddd
Bump extension to v0.2.4
Embeds cross-origin images in SingleFile archives (extension 382d7f8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 19:23:39 -06:00
Roland
3929642167
pi-gen: clear leftover BUILD_DIR at startup (robust to killed builds)
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
A SIGTERM/kill (e.g. `systemctl stop`) skips the EXIT-trap cleanup, leaving
/var/tmp/wrolpi-build-pi-gen behind. The next build's `mv pi-gen-bookworm-arm64
wrolpi-build-pi-gen` would then nest into the stale dir and fail. Remove the
leftover (and intermediate zip/unzip dir) at startup, mirroring the debian
build, stripping immutable flags off blob files first.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 18:06:31 -06:00
Roland
33498504b0
release-watch: dedup against the Spaces origin, not the CDN
The watcher read latest.json through the CDN, which caches it ~5 min. A
catch-up timer fire right after a release published (Persistent=true) read the
stale cached tag, mismatched, and kicked off a redundant multi-hour rebuild of
the just-published version. Read the manifest from the strongly-consistent
Spaces origin for the dedup so it always reflects the latest publish.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 17:49:47 -06:00
Roland
5497e1eb61
Fixing flaky domain search test. 2026-05-31 13:09:20 -06:00
Roland
345b69b7d7
chore: address Greptile P2 review comments on PR 449
- In get_effective_video_settings: on unparseable sleep_requests, delete
  the key instead of forcing 0.0. This lets the global config default
  (already present from _extract_global_inheritable) take effect, as
  suggested by Greptile.

- In handleInputEvent: add explicit `value !== ''` guard before the
  isNaN check. Prevents silent NaN in form state for empty number
  fields (isNaN('') === false in JS).

Both changes are strict improvements with no behavior change for
valid inputs.
2026-05-31 13:02:44 -06:00
Roland
9cc5c15e32 fix: coerce sleep_requests and numeric settings from string form values
`sleep_requests` (and other numeric inheritable video download settings)
could arrive as strings ('' or '0.75') from the frontend NumberInputForm
and DownloadRequest bodies. These were persisted into Download.settings
(JSONB) and later triggered:

    if effective['sleep_requests'] > 0:
    TypeError: '>' not supported between instances of 'str' and 'int'

in VideoDownloader.execute_download (and similar sites).

Root cause: DownloadSettings.__post_init__ only filtered None/[] and did
no type coercion, even though the dataclass declared Optional[float]/int.

Changes (minimal targeted fix):
- Add _coerce_optional_number helper and apply it to all numeric fields
  in DownloadSettings.__post_init__ (the existing sanitization boundary).
- Centralize legacy string coercion for persisted data in the single
  read chokepoint: get_effective_video_settings.
- Simplify VideoDownloaderConfig.sleep_requests getter.
- Frontend: use parseFloat for type=number inputs + null defaults for
  sleep_requests (prevents '' from ever being submitted).
- Add focused regression test for string coercion path.
- Revert unrelated change to string field filtering behavior.

No NaN/Inf speculation, no duplicated try/float guards at every call site,
no behavior change for non-numeric settings.
2026-05-31 13:02:31 -06:00
Roland
cd9fb4c6de
release-watch: don't auto-retry failed builds (default MAX_RETRIES=1)
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
Builds take hours and a failure almost always needs a code fix, not a blind
retry -- retrying 3x burned ~5h overnight. Default to a single attempt: on
failure, record the marker, fire the notify hook, and wait for a human to
investigate and clear it. The MAX_RETRIES override remains for N attempts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 12:21:41 -06:00
Roland
5625f17b5b
pi-gen: set HOME before git config --global in the chroot (systemd fix)
When the release runs under the wrolpi-release systemd service, HOME is unset,
so `git config --global --add safe.directory /opt/wrolpi` in the pi-gen chroot
aborts with "fatal: $HOME not set" and the build fails at stage 3. Manual
shell builds worked only because the shell provided HOME. Export HOME=/root in
the chroot hook (self-contained), and also set Environment=HOME=/root in the
service unit template.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 12:15:32 -06:00
Roland
dcfad8b4af
debian-live-config: copy built ISO by name, not a SCRIPT_DIR glob
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
The build ended with `mv "${SCRIPT_DIR}"/*.iso "${DEST}"`. A leftover ISO from
a previous build made the glob match two files, so `mv` treated the target as
a directory and failed ("No such file or directory") -- after a successful
37-minute lb build. Take the single ISO out of the freshly-wiped BUILD_DIR via
a nullglob array and copy it to the final name; robust to leftovers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 20:27:11 -06:00
Roland
cccbac9ae9
Fetching processes from PROCFS_PATH in docker container.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-05-30 15:59:14 -06:00
Roland
03a2596799
Bumping version to 0.23.1-beta. 2026-05-30 14:32:06 -06:00
Roland
30550f4c5b
release: address Greptile review + add opaque notify hook
Greptile (PR #448):
- C1 (P1): release-watch.sh compared the GitHub tag (minus 'v') against the
  manifest's .version (from version.txt content). If a tag and its version.txt
  ever diverged, the guard never matched and it rebuilt every hour forever.
  Compare the manifest's .tag against the release tag directly -- exact and
  immune to version-string formatting. Drops the now-unused normalize().
- C2 (P2): document that .tag is the built git ref (release tag for automated
  builds, branch for manual) and that consumers use .version for display.
- C3 (P2): document that newest-by-publish-date is the intended trigger and
  that prereleases are deliberately included (every release is -beta), so we
  do not filter on .prerelease as suggested.

Also add an optional completion notify hook: release.sh invokes
$WROLPI_RELEASE_NOTIFY with a success/failure message (no-op if unset, skipped
on dry-run). The public script names no path or tool; the real notifier stays
on the host (commented Environment= placeholder in the service unit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 14:23:19 -06:00
Roland
0e3564b704 Automating build and upload of WROLPi release images. 2026-05-30 14:23:08 -06:00
Roland
f4d0468300
Force checkout in upgrade.sh so a dirty working tree can't abort the upgrade
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
scripts/upgrade.sh runs `npm install`, which rewrites app/package-lock.json
and leaves the working tree dirty. The next upgrade's `git checkout ${BRANCH}`
then aborts with "Your local changes would be overwritten by checkout" before
the following `git reset --hard` can discard them, failing the upgrade with
exit 4.

Use `git checkout -f` so the branch switch discards local modifications; the
reset --hard immediately after already forces the tree to match origin, so the
intent (blow away local changes) is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-29 13:22:56 -06:00
Roland
e7591b16ad
Self-heal wrolpi-upgrade.service unit before delegating in upgrade.sh
upgrade.sh delegates to wrolpi-upgrade.service at the very start, but that
unit is only installed by repair.sh, which runs near the end of the upgrade.
On a box that has never run a repair since the unit was introduced, the
delegation fails immediately with "Unit wrolpi-upgrade.service not found",
the service never runs (so nothing is logged to the journal), and
/opt/wrolpi/upgrade.log is never created.

Install/refresh the unit from the checked-out tree and daemon-reload right
before starting the service. The delegation block already runs as root, so
no sudoers changes are needed. This is idempotent and also self-heals a
stale unit file.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-29 13:16:07 -06:00
Roland
f245f25179
Fixing alembic migration that imported non-existent code.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-05-29 11:15:32 -06:00
Roland
bae0e1b566
Bumping version to 0.23.0-beta.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-05-28 21:44:16 -06:00
Roland
960eb81880
Documenting WROLPi Portable USB, splitting install docs into INSTALL.md.
- New INSTALL.md consolidates all install paths: WROLPi Portable (Live
  USB) as the headline option, plus Debian 12, Raspberry Pi, and Docker.
- README install sections collapsed into a single Install section that
  links to INSTALL.md.
- UPGRADE.md gains an "Upgrading a WROLPi Portable USB" section covering
  scripts/wrolpi-usb.sh upgrade, and documents that in-place conversion
  of an existing data drive is not supported (backup -> flash -> restore).
- wrolpi-usb.sh install now prints that supported procedure instead of a
  "not implemented" stub; debian-live-config/README.md documents the same.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 21:33:22 -06:00
Roland
1bc337bfb4
Addressing Greptile P2s on the Ventoy ephemeral fallback.
- Treat virtual disks (vd*/xvd*/hd*) as direct, not ephemeral.  A blank
  VM disk imaged with the ISO can safely hold a persistence partition;
  previously these fell through the wildcard to loop mode and never
  accumulated state.
- Move the ephemeral note_action inside the mountpoint guard so a manual
  wrolpi-bootstrap.service restart mid-session doesn't re-fire the "NOT
  writing to disk" first-boot dialog.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 21:13:12 -06:00
Roland
82dd7782d3 Falling back to tmpfs when booted via Ventoy or other loop-mounted ISOs.
wrolpi-bootstrap.sh now classifies the live medium by device-node
pattern.  Real partitions (/dev/sd*, /dev/nvme*, /dev/mmcblk*) continue
to get a persistence partition appended.  Loop/device-mapper sources
(/dev/loop*, /dev/dm-*, /dev/mapper/*) — the marker for Ventoy/YUMI/
Easy2Boot/etc — mount a tmpfs at /media/wrolpi instead, so we never
sfdisk-append a partition onto the user's multiboot stick.

First-boot zenity dialog branches on mode and tells ephemeral users
their data won't persist; README documents the behaviour and the
deferred persistence.dat work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 21:12:52 -06:00
Roland
342374b97f
Guard preset lookup, drop dead Media import and volume*1 no-op
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
Address Greptile review on PR #447:
- Fall back to the minimum preset when an unrecognized/stale
  water_calculator_preset is read from localStorage, so a bad persisted
  value can no longer crash the component on mount.
- Remove the unused Media import.
- Simplify the metric branch of waterWeight (volume * 1 -> volume).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 18:07:01 -06:00
Roland
d998bb93a1 Adding realistic lifecycle test. 2026-05-28 18:06:48 -06:00
Roland
d3e57ea894 Water Storage calculator: rate guardrails, persisted duration, extra demand
Addresses calculator review feedback:
- Add an "Additional daily demand" (Extra) field for pets, livestock,
  guests, or sick household members not covered by the per-person rates.
- Make the preset choice explicit buttons with a note that switching
  resets rates to the preset defaults.
- Clamp rate and extra inputs to non-negative via a shared helper that
  preserves in-progress decimal entry (so "1.5" is typeable).
- Persist the duration slider to local storage (with an out-of-range
  guard), matching the people-count/metric/preset persistence.
- Relabel "Total Weight" as "Water Weight" to avoid implying it includes
  container tare weight.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 18:06:48 -06:00
Roland
c63fece967 Add Water Storage calculator (people → duration → containers)
New offline prepper calculator under Calculators:
- Men/women/children/pregnant-nursing counts with editable per-person
  daily rates (drinking + hygiene/dish use).
- Minimum/Comfortable presets grounded in CDC/FEMA (1 gal/person/day
  floor, store 2 weeks) and National Academies adequate-intake figures;
  Pregnant/Nursing adds the CDC/Ready.gov +0.5/+1 gal. Sources cited in
  the Daily Use info popup.
- Duration slider on a clean ladder (days → weeks → months → years) so it
  never lands on awkward values like "68 days".
- Containers Needed table listing every container (IBC tote → 500 mL
  bottle) with capacity and count + spare for the total volume.
- Total water and total weight (8.345 lb/gal, 1 kg/L).
- Imperial/metric, preset, and people counts persist via local storage.

Pure functions (dailyDemand, totalWater, containersNeeded, waterWeight,
formatDuration) covered by 21 unit tests. Registered in the calculator
list and global app search.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 18:06:48 -06:00
Roland
2e950b85ad
Drop spammy console.debug from Drive Ratio render/keystroke paths
Address Greptile review: remove the per-render and per-keystroke console.debug
calls (they fired on every React render and every field edit). Keep the single
strategic solveTrain log, which fires only on a settled (debounced) solve.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 16:19:21 -06:00
Roland
4e25459d50 Add Drive Ratio calculator to global search apps
Add a SUGGESTED_APPS entry so the Drive Ratio calculator shows under "Apps"
in the global search, matching the other calculator entries.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 15:17:50 -06:00
Roland
c38f90aae8 Add Drive Ratio calculator (pulleys, gears, sprockets)
A new "Drive Ratio" calculator under /more/calculators for belt, gear, and
chain-sprocket drives, supporting a compound train of 2–6 wheels on a single
shared belt/chain (constant size·rpm across the train).

- Pure, unit-tested math: size·rpm solver, ratio, torque, surface/belt speed,
  belt length, sprocket pitch diameter, chain length, gear pitch diameter.
- Mode toggle (Pulley/Gear/Sprocket) with imperial/metric units; torque in
  lb·ft or N·m.
- Add/remove wheels (2–6), each with a colorblind-friendly color shared between
  its inputs and the schematic SVG diagram (Paul Tol bright palette).
- Dependency-free inline-SVG diagram: colored wheels threaded by one belt/chain,
  or meshing gear teeth; auto-fitting viewBox.
- Per-mode details: belt/chain length and center distance (2-wheel only, with an
  info popup), per-wheel pitch diameters, and constant train speed.
- Guards against impossible geometry (wheels overlapping) and ideal-torque note.
- Integer teeth enforced in gear/sprocket modes; pulley diameters stay continuous.

Also fix Theme.Header dropping the theme's text color when a caller passes a
`style` prop (now merges {...t.style, ...props.style}).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 15:17:50 -06:00
Roland
9edfb0de25 Addressing Greptile P1s on wrolpi-usb.sh.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
numfmt with a negative argument:
  Under `set -euo pipefail` the upgrade subcommand would die at
  `numfmt --to=iec ... $headroom_bytes` whenever the new ISO was larger
  than the headroom — numfmt treats the leading `-` as an option flag,
  exits non-zero, and the script aborts before printing the descriptive
  back-up-and-reflash error message just below.  Splitting positive
  and negative cases and feeding numfmt the absolute value fixes it;
  the eventual error message now actually shows up.

Silent umount during sanity check:
  `umount "$child" 2>/dev/null || true` was swallowing both stderr and
  the exit code.  A busy partition (open file handle from a daemon
  that auto-opened the USB) would survive the loop, and `cmd_upgrade`
  would proceed straight to `dd if=<iso> of=<dev>` — writing to a
  block device whose partition is still mounted, corrupting the
  filesystem on that partition.  Adds a post-loop assertion: re-read
  MOUNTPOINTS and die with an `lsof` hint if anything is still mounted.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
662306a458 Blocking lightdm until wrolpi-bootstrap finishes — fixes first-boot zenity race.
wrolpi-bootstrap.service is `WantedBy=multi-user.target` and `Before=`
the wrolpi-api/app/kiwix/controller services, but had no ordering
constraint against lightdm.  Systemd was free to start lightdm in
parallel with bootstrap, so XFCE autologin would land before bootstrap
had written /tmp/wrolpi-firstboot-summary.txt — and the
wrolpi-firstboot-notify autostart would fire, find no summary, and
exit silently.

On the live test this manifested as the user seeing the desktop come
up immediately but never seeing the welcome zenity dialog.  The notify
script's own log confirmed the timing race:

  ===== Mon May 25 08:01:46 PM UTC 2026 invoked ... =====
  no summary file at /tmp/wrolpi-firstboot-summary.txt — nothing to do

bootstrap was still in the middle of `pg_createcluster` at 20:01:46
and didn't write the summary file until ~20:01:48.

Adding `Before=display-manager.service lightdm.service` makes systemd
wait for bootstrap to complete before bringing up the display manager.
Tested live by logging out + back in on the running ISO (re-firing
autostart with the summary already in place) — the zenity dialog
appeared.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
677483a502 Using existing repo .desktop files instead of hand-rolled variants.
Earlier commit hand-rolled six Type=Link launchers from scratch.  The
repo already had the canonical files at etc/raspberrypios/wrolpi.desktop,
wrolpi-help.desktop, wrolpi-help-files.desktop, and controller.desktop,
used on the RPi/Debian bare-metal deployments.

Switching the chroot hook to `cp` those files into ~/Desktop/ keeps the
live ISO in sync with the other deployments — when those .desktop files
get edited (URL changes, new icon, etc.), the live ISO inherits the
change with no chroot-hook edit needed.

Drops the Kiwix Library, WROLPi Files, and Disk Management entries the
previous commit added; the user requested the four canonical entries
only.  The trust autostart from the previous commit continues to mark
all *.desktop in ~/Desktop, so the surviving icons launch on
double-click.

All four icons tested on the live ISO at 10.0.0.17:

  WROLPi App           https://127.0.0.1            ✓
  WROLPi Controller    http://127.0.0.1             ✓
  WROLPi Help          http://127.0.0.1:9086        ✓
  WROLPi Help Files    file:///opt/wrolpi-help/docs ✓

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
4f721274c6 Adding WROLPi Help Files desktop launcher.
A Type=Link to file:///opt/wrolpi-help/docs/, which dispatches via
xdg-open to Thunar.  Mirrors the wrolpi-help-files.desktop entry from
etc/raspberrypios/ used on the RPi/Debian deployments.  Lets the user
browse the raw markdown sources for the Help site directly, in addition
to the rendered HTML at https://localhost:8086/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
d88417312a Adding desktop shortcuts for WROLPi services + trust-on-login autostart.
Five Type=Link launchers drop on /home/wrolpi/Desktop/ via the chroot
hook:

  WROLPi              https://localhost/             /opt/wrolpi/icon.png
  WROLPi Controller   https://localhost/controller/  controller-logo.png
  WROLPi Help         https://localhost:8086/        help-browser (icon theme)
  Kiwix Library       https://localhost:8085/        accessories-dictionary
  WROLPi Files        file:///media/wrolpi/          folder

All HTTPS URLs land on Caddy on localhost; the file:// URL is
dispatched via xdg-open to Thunar.

XFCE Bookworm rejects any .desktop file in $HOME/Desktop without
metadata::trusted set, even for Type=Link entries (verified live).  So
this also restores the trust autostart we previously had for the now-
removed Calamares icon — generalised to walk every *.desktop in
$HOME/Desktop and stamp the trust metadata on each, every session
start.  Idempotent; gio set is a no-op when values already match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
f8071ea5f4 Fixing first-boot zenity autostart that XFCE was silently ignoring.
The original wrolpi-firstboot-notify.desktop was XDG-valid (per
`desktop-file-validate`) but xfce4-session would not pick it up: zero
log entries on first boot, no zenity dialog, no process spawned.

Bringing the file in line with the upstream autostart entries that we
saw working (nm-applet, xfce4-notifyd, etc.) fixes it.  The likely
required field is one of `TryExec=` / `Icon=` / `Categories=` —
desktop-file-validate didn't flag the absence of any of them, but
adding them as a set makes the autostart fire.

Also instruments the notify script: it now writes a one-line log entry
to /tmp/wrolpi-firstboot-notify.log on every invocation so a future
"didn't see the dialog" report has a smoking gun.

And the rm of the (root-owned) summary file now tries via sudo first.
/tmp's sticky bit prevented the unprivileged user from removing a
root-owned file — left a stale summary that would re-trigger the
dialog on a subsequent login.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
17e26fab97 Adding scripts/wrolpi-usb.sh — upgrade subcommand.
`wrolpi-usb.sh upgrade <iso> <device>` replaces the boot partition of an
existing WROLPi USB with a new ISO while preserving the persistence
partition.

Flow:
  1. Locate the persistence partition by label.
  2. Read its sector range from sfdisk -d.
  3. Confirm the new ISO fits in the headroom (i.e. its byte count <=
     persistence_start_sector * 512).  If not, bail with instructions
     telling the user to back up /media/wrolpi, re-flash from scratch,
     and restore.  This is the case for USBs flashed before the 8 GiB-
     headroom commit.
  4. Save the partition table to /tmp/ for emergency recovery.
  5. Prompt for "yes".
  6. dd the new ISO onto the device.  The persistence partition's
     sectors are past the ISO's end, so its data is untouched.
  7. sfdisk --append to re-add the persistence partition entry at the
     saved sector range.  The data on disk is still there; we are just
     fixing up the partition table the new ISO wrote.
  8. partx -u to refresh the kernel view; verify the label is reachable.

`install` subcommand is wired up to a die() for now; will land in a
follow-up commit.

The script refuses to operate on non-removable disks (foot-gun guard).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
b7ff9f7389 Reserving 8 GiB headroom between ISO and persistence partitions.
The persistence partition is created immediately after the ISO partition
ends (1-MiB aligned).  When `wrolpi-usb.sh upgrade` overwrites the ISO
partition with a future, possibly-larger ISO, growth past the current
ISO size would overwrite persistence-partition sectors.

Reserve 8 GiB between them on first boot.  Future ISO releases up to
roughly 2x the current size (~3.3 GB) can be written into that gap
without touching persistence.

The cost is ~8 GiB of persistence the user can't reclaim.  On a typical
128 GB USB this is ~6% of total capacity, which is comparable to
filesystem overhead and rounding loss.

Existing USBs flashed before this commit have no headroom — they need
a one-time backup-and-reflash to pick it up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
217ec2da70 Polishing first-boot UX: better splash placement + post-bootstrap notification.
Splash:
  Shifted the logo + text block up by half a logo-height.  GRUB's menu
  renders centred over the splash; the previous placement put the icon
  directly behind the menu entries.

First-boot summary:
  wrolpi-bootstrap.sh now accumulates a short list of notable actions
  (partition created, Postgres cluster set up, DB initialised) and
  writes them to /tmp/wrolpi-firstboot-summary.txt.  A new XDG autostart
  (wrolpi-firstboot-notify) reads that file once XFCE comes up and shows
  a zenity dialog telling the user what just happened — answering the
  "wait, what did this thing just do to my USB drive?" question.

Subsequent boots where the bootstrap is a no-op write no summary file,
so the dialog only appears the boot it actually matters.  /tmp is tmpfs,
so the file is also naturally per-boot.

Adds zenity to the package list (not pulled by --apt-recommends false).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
0dc7f60a37 Fixing rebrand binary hook (wrong cwd) and using the new transparent logo.
The rebrand hook from the previous commit failed during lb build because
it referenced files as `binary/boot/grub/grub.cfg` etc., but
live-build's binary_hooks runs each hook with cwd already set to
binary/ (it does `cd binary && ../$HOOK`).  The paths now drop the
`binary/` prefix.

Splash regenerated from the new transparent logo
(app/public/icon-transparent.png) so the wifi-pie has a clean white
outline on the dark splash background — better visual separation than
the white-stripped icon.png the previous splash used.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
e58f7cf8ce Fixing splash typography: subtitle was overlapping the title.
Pushes "Off-grid library" down twice its own height from the bottom of
"WROLPi", clearing the visual collision.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
6fe3a8311d Rebranding the boot menu: WROLPi splash, WROLPi menu entries.
The auto-generated live-build splash.png composes a yellow "build worker"
hardhat with "Debian GNU/Linux 12 (bookworm) amd64" baked in as a PNG
overlay.  Both GRUB and ISOLINUX use it as the menu background, so it's
the first thing users see at boot.

Two replacement splash images (800x600 for GRUB, 640x480 for ISOLINUX)
land via config/includes.binary/, generated once from the project's
icon.png (the wifi-pie favicon).  No build-time composition needed; the
files are checked in as binary blobs.

A *.binary hook then seds the live-build-generated menu entries so the
GRUB and ISOLINUX text reads:

  Run WROLPi (live)             ← was "Live system (amd64)"
  Run WROLPi (live, fail-safe)  ← was "Live system (amd64 fail-safe mode)"
  Install WROLPi                ← was "Start installer"
  Install WROLPi (speech)       ← was "Start installer with speech synthesis"

We can't override live-build's substitution source without patching the
distro-installed `binary_grub_cfg` script; doing it after the fact in a
.binary hook keeps the customisation in our config tree.

Note: this rebrands what GRUB / ISOLINUX render *around* the d-i menus.
Once the user selects "Install WROLPi" and d-i actually launches, the
yellow hardhat reappears inside d-i's own screens — that logo
(logo_installer.png) lives inside the d-i initrd and would require a
custom initrd rebuild to swap.  Punting on that for now.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
e691f4db1e Fixing kiwix restart cmd: missing comma silently concatenated two argv entries.
`cmd = ('sudo', '/usr/bin/systemctl', 'restart' 'wrolpi-kiwix.service')`
                                              ^ missing comma here

Python implicit string-literal concatenation joined the two strings into
'restartwrolpi-kiwix.service', so systemctl saw one argument it didn't
recognise and exited 2 (INVALIDARGUMENT).  Net effect: kiwix never
restarted after a Zim download, the user still had to manually
`sudo systemctl restart wrolpi-kiwix` to see the new Zim.

Adds a unit test that pins the argv shape so an adjacent-literal typo
here surfaces in CI.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
f54b5f0dd8 Reverting to Debian Installer (--debian-installer live) over Calamares.
The Calamares desktop-icon path doesn't launch reliably in our wrolpi
auto-login session.  We chased two stacks of fixes — gvfs + tumbler
for xfdesktop's launch path, lxpolkit for pkexec's GUI auth agent,
metadata::trusted + xfce-exe-checksum for XFCE's untrusted-launcher
check — and the icon still does nothing when double-clicked on a fresh
boot.  The upstream Debian Installer reached via GRUB has none of these
problems and is the better-tested path for installing onto disk.

build.sh adds:
  --debian-installer live          live-installer copies the squashfs
                                   onto the target instead of doing a
                                   network re-install
  --debian-installer-gui true      graphical d-i frontend available

GRUB will now show Live / Install / Graphical Install / Advanced
entries; the user picks at boot.

Drops:
- calamares + calamares-settings-debian from the package list
- /etc/calamares/branding/wrolpi/ + settings.conf + modules
- The install-wrolpi.desktop launcher + the trust autostart we shipped
  to try to make it click-able

Net ISO size will drop ~50-100 MB (Calamares + Qt5).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
8b1abd104c Polishing the combined ISO: audio, WiFi, installer launcher.
Three classes of UX gaps surfaced when test-booting the previous build:

  Audio silent.
    --apt-recommends false stripped task-xfce-desktop's recommend chain,
    which is what normally pulls the PulseAudio server.  We only had the
    panel plugin + client libs.  Adding pulseaudio + pulseaudio-utils +
    pavucontrol + alsa-utils explicitly.

  No WiFi UI.
    NetworkManager daemon was running and could see the radio, but
    network-manager-gnome (nm-applet) wasn't pulled by recommends either.
    Adding it explicitly.

  Desktop "Install WROLPi" icon does nothing.
    Two missing pieces stacked: (a) xfdesktop's launch path needs gvfs
    (for metadata::trusted) and tumbler (thumbnail dispatch raises a
    GLib critical without it).  (b) Even when xfdesktop dispatches, pkexec
    can't authenticate without a registered GUI polkit agent — task-xfce-
    desktop's recommend chain would have provided one, but it didn't, so
    pkexec falls back to a TTY agent that fails in a Thunar-spawned
    process (no /dev/tty).  Adding gvfs, lxpolkit, tumbler.  Rewrites
    the desktop launcher as Type=Link (skips the executable-trust check
    entirely) and ships a session autostart that gio-sets
    metadata::trusted + metadata::xfce-exe-checksum so xfdesktop accepts
    user-owned .desktop files.

Also fixes a Calamares FATAL on boot: branding.desc requires a slideshow
key even when there's no real slideshow content; ships a minimal show.qml
that just renders the WROLPi logo + a welcome line.

Also adds xfce4-notifyd and xfce4-power-manager to desktop.list.chroot —
not directly tied to the three issues above, but obvious omissions from
the same recommend-stripped XFCE install.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
2acb80c1ae
Branding Calamares as WROLPi; post-install cleanup of the orphan launcher.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
The installer now identifies as WROLPi rather than Debian: product name,
sidebar logo (the wifi-pie favicon from /icon.png), bootloader entry name,
and URLs all point at WROLPi.  Sidebar colors are WROLPi purple.

Three new files under etc/calamares/:
- branding/wrolpi/branding.desc  — product strings + image + color scheme
- branding/wrolpi/wrolpi-logo.png — copy of the project favicon
- settings.conf                  — override picks branding=wrolpi and adds
                                   shellprocess to the exec sequence
- modules/shellprocess.conf      — rm /home/wrolpi/Desktop/install-wrolpi.desktop
                                   on the target so the installed system
                                   doesn't ship with a broken icon

Slideshow QML is intentionally omitted: producing our own QML/asset pack
is deferred polish.  Calamares falls back to a static welcome screen.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:49:22 -06:00
Roland
2524a017e1 Adding Calamares as the graphical installer.
Pulls in `calamares` + `calamares-settings-debian` (the upstream Debian
theme/config package).  Calamares runs from the desktop via the
"Install WROLPi to disk" icon: pkexec elevates, the user types the
wrolpi account password, the standard install wizard opens.

The launcher is the upstream install-debian.desktop with Name and
Comment rebranded to "Install WROLPi to disk" / "Install WROLPi onto
this computer's internal drive".  We keep the upstream install-debian
wrapper script (it handles xhost + pkexec calamares correctly) and just
override the visible labels.

After the chroot creates the wrolpi user the hook copies the launcher
to /home/wrolpi/Desktop/install-wrolpi.desktop and marks it executable
so XFCE launches it without an "untrusted application" warning.

Behaviour:

  Live USB boot          XFCE autologs to wrolpi.  Install icon visible.
                         Click → polkit prompts → Calamares wizard.

  After install + reboot wrolpi-bootstrap.service sets up Postgres on
                         /media/wrolpi/config/postgresql/, runs DB init,
                         starts wrolpi-api / app / kiwix / controller.

Two known phase-3 polish items deferred:
- WROLPi branding for Calamares (logo, colors, slideshow); using stock
  Debian branding for now.
- An orphan install-wrolpi.desktop icon survives on the installed
  system (Calamares removes calamares-settings-debian, taking
  install-debian with it, but not the copy in the user's home).  A
  Calamares shellprocess will rm it in phase 3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:44:05 -06:00
Roland
3c168a4f2d Merging wrolpi-portable into debian-live-config; unifying Postgres on /media/wrolpi/config/postgresql.
Two parallel live-build configs (debian-live-config/ for the installer
ISO and wrolpi-portable/ for the Live USB) carry the same chroot hook
with subtly different fixes — the portable side had accumulated the
better implementation (npm build, SHA-pinned blobs, dynamic Postgres
version, partition-by-start-sector, controller.yaml sentinel, autologin,
branding).  This commit folds those improvements into debian-live-config/
and retires wrolpi-portable/.

The Postgres data + config directories now bind-mount to
/media/wrolpi/config/postgresql/{data,config} regardless of deployment
shape (Live USB, Calamares-installed system with external drive,
Calamares-installed system with no external drive).  Result: pulling
a WROLPi external drive from one host and plugging into another
brings the entire indexed library along — no re-indexing.  Previously
the DB lived on /mnt/persistence/postgresql-{data,config} on Portable
and on /var/lib/postgresql (root) everywhere else; moving a drive
required rebuilding the DB on the new host.

Single wrolpi-bootstrap.{sh,service} replaces wrolpi-portable-bootstrap.
The script detects deployment by probing for a live-boot medium mount:
present → Live (create persistence partition if missing); absent →
Installed (skip partition step; Postgres bind-mount and cluster init
still run).  --debian-installer is dropped from lb config; Calamares
will land in a follow-up commit as the graphical installer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:44:05 -06:00
Roland
40236cae97 Identify new persistence partition by start sector, not lsblk's tail.
The fallback after `partx -a` (which is wrapped in `|| true` and can
silently no-op) was `lsblk -lnpo NAME "$BOOT_DRIVE" | tail -1`.  If the
new partition node has not appeared in the kernel's partition table yet,
that fallback returns the previous last partition — typically sdb2 (the
ESP) or sdb1 (the live ISO data partition).  Both are valid block
devices, so the `[ -b ... ]` guard passes; the next line is
`mkfs.ext4 -L persistence -F`, which wipes whichever wrong partition the
fallback returned.  Narrow conditions, but catastrophic when triggered.

Switching to matching on the start sector we just assigned via sfdisk:
no existing partition can share it, so the result is unambiguous.  Retry
loop allows up to 5s for udev to register the new node.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:44:05 -06:00
Roland
719905cb77 Moving WROLPi-managed mounts off /etc/fstab onto a drive-resident fstab.yaml.
A WROLPi drive can move between hosts and across deployment styles
(Portable, RPi, Debian).  /etc/fstab is host-specific and on the Portable
image it's part of a read-only squashfs, so persisting secondary mounts
there was both wrong-host and lost-on-reboot.  This change makes
/media/wrolpi/config/fstab.yaml the single source of truth for WROLPi-
managed mounts and introduces a new wrolpi-mounts.service that reconciles
live state to match.

Architecture:

- The Controller writes intent (fstab.yaml) but never calls mount(8) or
  umount(8) itself.  Every mutation triggers `systemctl restart
  wrolpi-mounts.service`; the Controller then verifies the outcome via
  the same MountExecutor abstraction the service uses.

- wrolpi-mounts.service runs a thin entry point that constructs a real
  Reconciler with a SubprocessMountExecutor and calls apply().  The
  reconciler mounts what's in fstab.yaml but missing from the live state,
  and unmounts paths it previously claimed that are no longer in
  fstab.yaml.  A managed-set file in /run/wrolpi-mounts/managed prevents
  the reconciler from ever touching user-mounted paths it didn't own.

- One-time migration (marker-gated) lifts both /etc/fstab WROLPi entries
  and any controller.yaml drives.mounts entries into fstab.yaml.
  /etc/fstab is backed up to /etc/fstab.wrolpi-migration.backup;
  migrated lines are commented in place with a "migrated to fstab.yaml:"
  prefix so a host admin can hand-recover.

Test approach: dependency injection in place of patching.  The
MountExecutor protocol has SubprocessMountExecutor (production) and
FakeMountExecutor (in-memory, records calls).  The Reconciler accepts an
executor in its constructor, so the 20 reconciler tests + 13 migration
tests + 18 fstab_yaml tests use real files in tmp_path with zero
mock.patch calls.  test_disks_api.py drops to mocks only for the
host-touching read helpers; the mount/unmount paths now run the real
Reconciler against the fake executor.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:44:05 -06:00
Roland
94d56f313a Incorporating Greptile feedback from PR 442.
persistence.conf was writing `union,source=...` entries.  On the first boot
that did not matter because the bootstrap performed a real `mount --bind`
itself; from the second boot onward, however, live-boot's initramfs reads
persistence.conf and applies the entries as OverlayFS union mounts.
PostgreSQL does not support OverlayFS — switching to `bind,source=...`
makes the in-band mount and the initramfs mount match and avoids potential
data corruption.

The bootstrap script hardcoded Postgres version 15 (cluster check, create,
service-start), inconsistent with the chroot hook which already derives it
dynamically.  Switched to discovery via `/usr/lib/postgresql/<ver>` so a
postgresql-16 bump no longer requires touching this script.

The tippecanoe source tarball was the only externally-fetched artifact in
the chroot hook without a SHA256 pin.  Added one (computed from upstream
v2.79.0).

`RequiresMountsFor` covered only `/usr/lib/live/mount/medium`; live-boot on
newer releases mounts at `/run/live/medium`.  Listed all three paths the
bootstrap script itself searches so systemd orders the unit after whichever
mount actually exists.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:44:05 -06:00
Roland
5040349c46 Building the React app in chroot and signalling drive-ready to Controller.
The chroot hook only ran `npm install`, leaving /opt/wrolpi/app/build/
missing.  wrolpi-app.service runs `serve build -l 3000 ...`, so it started
fine but returned 404 for every path.  RPi-OS / Debian installs handle this
via repair.sh at first boot; we bake the build into the squashfs so the
live image works offline immediately.

The Controller's `is_primary_drive_mounted()` checks for
/media/wrolpi/config/controller.yaml — without it the Controller renders
its onboarding wizard and offers the persistence partition (sdb3, mounted
at /mnt/persistence) as an unconfigured candidate, which would corrupt the
running system if selected.  Touch an empty controller.yaml after creating
/media/wrolpi/config so the Controller treats the drive as configured.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:44:05 -06:00
Roland
d329c34cbc Fixing two wrolpi-portable-bootstrap.service bugs.
LAST_END parser was assigning to a scalar (not MAX), so on iso-hybrid layouts
where the small ESP is listed after the much larger boot partition, sfdisk
got a start sector inside an existing partition.  Also sfdisk -d pads values
with whitespace (`start=          64,`), which broke a naive awk tokenizer.
Switched to a sed parser that strips the padding, and computes the true
MAX(start+size).

`systemctl enable --now wrolpi-api ...` deadlocked: the runtime units have
implicit `After=wrolpi-portable-bootstrap.service` via this unit's `Before=`,
so --now waited forever for them to start while they waited for bootstrap to
finish.  Split enable from start; use --no-block so systemd queues the
starts after we exit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:44:05 -06:00