Commit graph

17 commits

Author SHA1 Message Date
lrnselfreliance
1e89cb55f6
Fresh Live USBs don't report an ownership update (#840)
Some checks are pending
Backend Tests / api-tests (core) (push) Waiting to run
Backend Tests / api-tests (files) (push) Waiting to run
Backend Tests / api-tests (media) (push) Waiting to run
Backend Tests / api-tests (rest) (push) Waiting to run
Backend Tests / api-contract-tests (push) Waiting to run
Backend Tests / api-migrations-and-start (push) Waiting to run
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Cypress Tests / cypress-component-tests (push) Waiting to run
Cypress Tests / cypress-e2e-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
The library-ownership migration compared the owner of
/mnt/persistence/wrolpi after creating it, so on a new persistence
partition the root-owned directory always looked like an old library and
the first-boot dialog said "Updated the library's ownership for this
WROLPi version".  Only re-own a library that already existed.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 19:52:26 -06:00
lrnselfreliance
baa86b4ad1
Upgraded Live USBs keep a writable library (#835)
Some checks are pending
Backend Tests / api-tests (core) (push) Waiting to run
Backend Tests / api-tests (files) (push) Waiting to run
Backend Tests / api-tests (media) (push) Waiting to run
Backend Tests / api-tests (rest) (push) Waiting to run
Backend Tests / api-contract-tests (push) Waiting to run
Backend Tests / api-migrations-and-start (push) Waiting to run
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Cypress Tests / cypress-component-tests (push) Waiting to run
Cypress Tests / cypress-e2e-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
ISOs before wrolpi became a system account wrote the Live USB's library on
the persistence partition as UID 1000.  Upgrading such a USB with a newer
ISO (wrolpi-usb.sh upgrade) keeps the persistence partition, so the library
would belong to the live session's "user" and the services could not write
to it.  Bootstrap now re-owns the library once when its owner is not this
image's wrolpi.  Live USBs only: installed systems' drives are left alone.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:23:53 -06:00
lrnselfreliance
f644a80c9a
Debian installs create the user the installer asks for (#834)
The ISO's chroot hook created wrolpi as a login user (UID 1000, password
"wrolpi", full sudo, LightDM autologin).  The Debian installer creates the
user it asks for only when no UID 1000-59999 user exists, so it silently
skipped that user: the only login on an installed system was wrolpi/wrolpi,
with SSH and sudo.

wrolpi is now a system account with no password or login sudo, as on every
other WROLPi deployment.  The installer creates the user it asks for (with
sudo, no autologin).  The live session logs in as live-config's "user".
Desktop shortcuts moved to /etc/skel so every user gets them.

check-iso.sh now fails the build when the image has a UID 1000-59999 user.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:23:40 -06:00
lrnselfreliance
dc2ab22ae0
Debian installs show the Controller drive wizard (#826)
wrolpi-bootstrap.sh created controller.yaml on installed systems, which
hid the onboarding wizard.  Installed systems now wait for the wizard,
whose repair.sh finishes setup.

"Run repair without external drive" now writes controller.yaml before
starting repair, so the wizard does not return after repair.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 15:50:20 -06:00
Roland
e01795ad7e
Debian 13 (trixie) rebase: docker, debian-live-config, and pi-gen builds.
All three build targets move from Debian 12/bookworm to Debian 13/trixie
for fresh installs; existing Debian 12 systems remain supported in place.

Docker: api/help on python:3.13-trixie, controller on 3.13-slim-trixie,
app on node:20-trixie (matches trixie's nodejs 20), samba/zim on
debian:13.  Full backend suite passes on Python 3.13 in the api container.

debian-live-config: --distribution trixie; postgres teardown for the
SQLite cutover (pg_dropcluster hook block removed, wrolpi-bootstrap.sh no
longer creates a cluster or calls the deleted initialize_api_db.sh — the
API creates wrolpi.db itself on first start).  README documents the
SQLite layout and drops Etcher recommendations.

pi-gen: upstream has no trixie branch — the arm64 branch now defaults to
RELEASE=trixie; KEYBOARD_KEYMAP=en_US is rejected by trixie's
console-setup (ckbcomp validates), use 'us'; stop stamping a Postgres
.pgpass into /etc/skel.

Package lists (both targets): postgresql-15/postgresql-doc/libpq-dev
deleted; libtiff5-dev renamed to libtiff-dev; software-properties-common
and cpufrequtils removed from trixie (Controller CPU throttle falls back
to sysfs, dead cpufreq-set sudoers rule removed); locales and gnupg added
(no longer arrive transitively under --apt-recommends false); sqlite3 CLI
added for reset_tags.sh.

help.sh: replace the Postgres service/port/socket/cluster/psql checks
(all false FAILEDs after the SQLite cutover) with SQLite checks:
wrolpi.db exists, PRAGMA quick_check, file_group table + row count, run
as the wrolpi user so root never creates the -shm sidecar.

Both build scripts take a WROLPI_BUILD_DIR override so development builds
can run beside the release pipeline's build trees; the sudo re-exec
passes it through explicitly (a plain sudo would drop it).

test_common.py: the truncate tests hard-coded sys.getsizeof results from
the 3.11 object layout (str/list shrank in 3.12); assert the contract
(fits budget, prefix, monotonic in budget) so they pass on 3.11 and 3.13.

Verified: trixie ISO (3.7G) and pi-gen desktop image (4.3G) both built
from scratch and passed their sanity checks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 15:43:08 -06:00
Roland
f500600025
Bootstrap: Postgres bind-mounts on live boots only.
wrolpi-bootstrap.service applied the /media/wrolpi/config/postgresql
bind-mounts over /var/lib/postgresql and /etc/postgresql at every boot,
including on installed systems.  The binds exist for live boots, where
the root filesystem is an ephemeral overlay; on an installed system they
put the cluster in the path of repair.sh's config chown
(pg_filenode.map permission errors) and, once the user mounts an
NTFS/exFAT primary drive over /media/wrolpi, point Postgres at a
filesystem that cannot host a data directory at all — no clusters,
connection refused, while the umbrella postgresql.service stays green.

Installed systems now keep Postgres at the normal Debian paths on the
root filesystem.  A box that still has a stale bind active from an old
boot gets a log note to reboot; on the first clean boot the existing
pg_createcluster/initialize_api_db logic builds the root-disk cluster
automatically and the database rebuilds from configs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 15:33:54 -06:00
Roland
ebbc60992c
Docs: recommend Raspberry Pi Imager instead of Etcher.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
Etcher is privacy-invasive; replace it in INSTALL.md, the USB upgrade
tool messages, and the live-boot ephemeral warning.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 18:58:54 -06:00
Roland
1bc337bfb4
Addressing Greptile P2s on the Ventoy ephemeral fallback.
- Treat virtual disks (vd*/xvd*/hd*) as direct, not ephemeral.  A blank
  VM disk imaged with the ISO can safely hold a persistence partition;
  previously these fell through the wildcard to loop mode and never
  accumulated state.
- Move the ephemeral note_action inside the mountpoint guard so a manual
  wrolpi-bootstrap.service restart mid-session doesn't re-fire the "NOT
  writing to disk" first-boot dialog.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 21:13:12 -06:00
Roland
82dd7782d3 Falling back to tmpfs when booted via Ventoy or other loop-mounted ISOs.
wrolpi-bootstrap.sh now classifies the live medium by device-node
pattern.  Real partitions (/dev/sd*, /dev/nvme*, /dev/mmcblk*) continue
to get a persistence partition appended.  Loop/device-mapper sources
(/dev/loop*, /dev/dm-*, /dev/mapper/*) — the marker for Ventoy/YUMI/
Easy2Boot/etc — mount a tmpfs at /media/wrolpi instead, so we never
sfdisk-append a partition onto the user's multiboot stick.

First-boot zenity dialog branches on mode and tells ephemeral users
their data won't persist; README documents the behaviour and the
deferred persistence.dat work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 21:12:52 -06:00
Roland
d88417312a Adding desktop shortcuts for WROLPi services + trust-on-login autostart.
Five Type=Link launchers drop on /home/wrolpi/Desktop/ via the chroot
hook:

  WROLPi              https://localhost/             /opt/wrolpi/icon.png
  WROLPi Controller   https://localhost/controller/  controller-logo.png
  WROLPi Help         https://localhost:8086/        help-browser (icon theme)
  Kiwix Library       https://localhost:8085/        accessories-dictionary
  WROLPi Files        file:///media/wrolpi/          folder

All HTTPS URLs land on Caddy on localhost; the file:// URL is
dispatched via xdg-open to Thunar.

XFCE Bookworm rejects any .desktop file in $HOME/Desktop without
metadata::trusted set, even for Type=Link entries (verified live).  So
this also restores the trust autostart we previously had for the now-
removed Calamares icon — generalised to walk every *.desktop in
$HOME/Desktop and stamp the trust metadata on each, every session
start.  Idempotent; gio set is a no-op when values already match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
f8071ea5f4 Fixing first-boot zenity autostart that XFCE was silently ignoring.
The original wrolpi-firstboot-notify.desktop was XDG-valid (per
`desktop-file-validate`) but xfce4-session would not pick it up: zero
log entries on first boot, no zenity dialog, no process spawned.

Bringing the file in line with the upstream autostart entries that we
saw working (nm-applet, xfce4-notifyd, etc.) fixes it.  The likely
required field is one of `TryExec=` / `Icon=` / `Categories=` —
desktop-file-validate didn't flag the absence of any of them, but
adding them as a set makes the autostart fire.

Also instruments the notify script: it now writes a one-line log entry
to /tmp/wrolpi-firstboot-notify.log on every invocation so a future
"didn't see the dialog" report has a smoking gun.

And the rm of the (root-owned) summary file now tries via sudo first.
/tmp's sticky bit prevented the unprivileged user from removing a
root-owned file — left a stale summary that would re-trigger the
dialog on a subsequent login.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
b7ff9f7389 Reserving 8 GiB headroom between ISO and persistence partitions.
The persistence partition is created immediately after the ISO partition
ends (1-MiB aligned).  When `wrolpi-usb.sh upgrade` overwrites the ISO
partition with a future, possibly-larger ISO, growth past the current
ISO size would overwrite persistence-partition sectors.

Reserve 8 GiB between them on first boot.  Future ISO releases up to
roughly 2x the current size (~3.3 GB) can be written into that gap
without touching persistence.

The cost is ~8 GiB of persistence the user can't reclaim.  On a typical
128 GB USB this is ~6% of total capacity, which is comparable to
filesystem overhead and rounding loss.

Existing USBs flashed before this commit have no headroom — they need
a one-time backup-and-reflash to pick it up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
217ec2da70 Polishing first-boot UX: better splash placement + post-bootstrap notification.
Splash:
  Shifted the logo + text block up by half a logo-height.  GRUB's menu
  renders centred over the splash; the previous placement put the icon
  directly behind the menu entries.

First-boot summary:
  wrolpi-bootstrap.sh now accumulates a short list of notable actions
  (partition created, Postgres cluster set up, DB initialised) and
  writes them to /tmp/wrolpi-firstboot-summary.txt.  A new XDG autostart
  (wrolpi-firstboot-notify) reads that file once XFCE comes up and shows
  a zenity dialog telling the user what just happened — answering the
  "wait, what did this thing just do to my USB drive?" question.

Subsequent boots where the bootstrap is a no-op write no summary file,
so the dialog only appears the boot it actually matters.  /tmp is tmpfs,
so the file is also naturally per-boot.

Adds zenity to the package list (not pulled by --apt-recommends false).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
f54b5f0dd8 Reverting to Debian Installer (--debian-installer live) over Calamares.
The Calamares desktop-icon path doesn't launch reliably in our wrolpi
auto-login session.  We chased two stacks of fixes — gvfs + tumbler
for xfdesktop's launch path, lxpolkit for pkexec's GUI auth agent,
metadata::trusted + xfce-exe-checksum for XFCE's untrusted-launcher
check — and the icon still does nothing when double-clicked on a fresh
boot.  The upstream Debian Installer reached via GRUB has none of these
problems and is the better-tested path for installing onto disk.

build.sh adds:
  --debian-installer live          live-installer copies the squashfs
                                   onto the target instead of doing a
                                   network re-install
  --debian-installer-gui true      graphical d-i frontend available

GRUB will now show Live / Install / Graphical Install / Advanced
entries; the user picks at boot.

Drops:
- calamares + calamares-settings-debian from the package list
- /etc/calamares/branding/wrolpi/ + settings.conf + modules
- The install-wrolpi.desktop launcher + the trust autostart we shipped
  to try to make it click-able

Net ISO size will drop ~50-100 MB (Calamares + Qt5).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
8b1abd104c Polishing the combined ISO: audio, WiFi, installer launcher.
Three classes of UX gaps surfaced when test-booting the previous build:

  Audio silent.
    --apt-recommends false stripped task-xfce-desktop's recommend chain,
    which is what normally pulls the PulseAudio server.  We only had the
    panel plugin + client libs.  Adding pulseaudio + pulseaudio-utils +
    pavucontrol + alsa-utils explicitly.

  No WiFi UI.
    NetworkManager daemon was running and could see the radio, but
    network-manager-gnome (nm-applet) wasn't pulled by recommends either.
    Adding it explicitly.

  Desktop "Install WROLPi" icon does nothing.
    Two missing pieces stacked: (a) xfdesktop's launch path needs gvfs
    (for metadata::trusted) and tumbler (thumbnail dispatch raises a
    GLib critical without it).  (b) Even when xfdesktop dispatches, pkexec
    can't authenticate without a registered GUI polkit agent — task-xfce-
    desktop's recommend chain would have provided one, but it didn't, so
    pkexec falls back to a TTY agent that fails in a Thunar-spawned
    process (no /dev/tty).  Adding gvfs, lxpolkit, tumbler.  Rewrites
    the desktop launcher as Type=Link (skips the executable-trust check
    entirely) and ships a session autostart that gio-sets
    metadata::trusted + metadata::xfce-exe-checksum so xfdesktop accepts
    user-owned .desktop files.

Also fixes a Calamares FATAL on boot: branding.desc requires a slideshow
key even when there's no real slideshow content; ships a minimal show.qml
that just renders the WROLPi logo + a welcome line.

Also adds xfce4-notifyd and xfce4-power-manager to desktop.list.chroot —
not directly tied to the three issues above, but obvious omissions from
the same recommend-stripped XFCE install.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 20:16:02 -06:00
Roland
2524a017e1 Adding Calamares as the graphical installer.
Pulls in `calamares` + `calamares-settings-debian` (the upstream Debian
theme/config package).  Calamares runs from the desktop via the
"Install WROLPi to disk" icon: pkexec elevates, the user types the
wrolpi account password, the standard install wizard opens.

The launcher is the upstream install-debian.desktop with Name and
Comment rebranded to "Install WROLPi to disk" / "Install WROLPi onto
this computer's internal drive".  We keep the upstream install-debian
wrapper script (it handles xhost + pkexec calamares correctly) and just
override the visible labels.

After the chroot creates the wrolpi user the hook copies the launcher
to /home/wrolpi/Desktop/install-wrolpi.desktop and marks it executable
so XFCE launches it without an "untrusted application" warning.

Behaviour:

  Live USB boot          XFCE autologs to wrolpi.  Install icon visible.
                         Click → polkit prompts → Calamares wizard.

  After install + reboot wrolpi-bootstrap.service sets up Postgres on
                         /media/wrolpi/config/postgresql/, runs DB init,
                         starts wrolpi-api / app / kiwix / controller.

Two known phase-3 polish items deferred:
- WROLPi branding for Calamares (logo, colors, slideshow); using stock
  Debian branding for now.
- An orphan install-wrolpi.desktop icon survives on the installed
  system (Calamares removes calamares-settings-debian, taking
  install-debian with it, but not the copy in the user's home).  A
  Calamares shellprocess will rm it in phase 3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:44:05 -06:00
Roland
3c168a4f2d Merging wrolpi-portable into debian-live-config; unifying Postgres on /media/wrolpi/config/postgresql.
Two parallel live-build configs (debian-live-config/ for the installer
ISO and wrolpi-portable/ for the Live USB) carry the same chroot hook
with subtly different fixes — the portable side had accumulated the
better implementation (npm build, SHA-pinned blobs, dynamic Postgres
version, partition-by-start-sector, controller.yaml sentinel, autologin,
branding).  This commit folds those improvements into debian-live-config/
and retires wrolpi-portable/.

The Postgres data + config directories now bind-mount to
/media/wrolpi/config/postgresql/{data,config} regardless of deployment
shape (Live USB, Calamares-installed system with external drive,
Calamares-installed system with no external drive).  Result: pulling
a WROLPi external drive from one host and plugging into another
brings the entire indexed library along — no re-indexing.  Previously
the DB lived on /mnt/persistence/postgresql-{data,config} on Portable
and on /var/lib/postgresql (root) everywhere else; moving a drive
required rebuilding the DB on the new host.

Single wrolpi-bootstrap.{sh,service} replaces wrolpi-portable-bootstrap.
The script detects deployment by probing for a live-boot medium mount:
present → Live (create persistence partition if missing); absent →
Installed (skip partition step; Postgres bind-mount and cluster init
still run).  --debian-installer is dropped from lb config; Calamares
will land in a follow-up commit as the graphical installer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:44:05 -06:00