Merging wrolpi-portable into debian-live-config; unifying Postgres on /media/wrolpi/config/postgresql.

Two parallel live-build configs (debian-live-config/ for the installer
ISO and wrolpi-portable/ for the Live USB) carry the same chroot hook
with subtly different fixes — the portable side had accumulated the
better implementation (npm build, SHA-pinned blobs, dynamic Postgres
version, partition-by-start-sector, controller.yaml sentinel, autologin,
branding).  This commit folds those improvements into debian-live-config/
and retires wrolpi-portable/.

The Postgres data + config directories now bind-mount to
/media/wrolpi/config/postgresql/{data,config} regardless of deployment
shape (Live USB, Calamares-installed system with external drive,
Calamares-installed system with no external drive).  Result: pulling
a WROLPi external drive from one host and plugging into another
brings the entire indexed library along — no re-indexing.  Previously
the DB lived on /mnt/persistence/postgresql-{data,config} on Portable
and on /var/lib/postgresql (root) everywhere else; moving a drive
required rebuilding the DB on the new host.

Single wrolpi-bootstrap.{sh,service} replaces wrolpi-portable-bootstrap.
The script detects deployment by probing for a live-boot medium mount:
present → Live (create persistence partition if missing); absent →
Installed (skip partition step; Postgres bind-mount and cluster init
still run).  --debian-installer is dropped from lb config; Calamares
will land in a follow-up commit as the graphical installer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Roland 2026-05-24 12:07:36 -06:00 • committed by lrnselfreliance
parent 40236cae97
commit 3c168a4f2d
16 changed files with 462 additions and 717 deletions

View file

@ -1,14 +1,64 @@
# Debian Live Config
This directory exists to facilitate the creation of WROLPi Debian images.
Build a bootable amd64 ISO that boots into a full WROLPi system on any x86
PC. Users can run WROLPi directly off the USB (a persistence partition is
created on the first boot so the database and media directory survive
reboots) or install to disk via the Calamares launcher on the desktop.
The chroot-phase build is identical for both modes; the resulting squashfs
is what runs in live mode AND what Calamares copies onto the target disk.
See https://live-team.pages.debian.net/live-manual/html/live-manual/index.en.html
## Generate Debian 12 ISO
## Build
1. Convert a map PBF file to your initial map DB dump:
* `docker run --rm -v /home/user/district-of-columbia-230111.osm.pbf:/data/region.osm.pbf lrnselfreliance/osm-map-dumper > debian-live-config/config/includes.chroot/opt/wrolpi-blobs/map-db-gis.dump`
2. Run the build.
* `sudo ./build.sh`
* Or, build specific branch: `sudo ./build.sh -b master`
3. The ISO is now in the `debian-live-config` directory.
```
sudo ./build.sh # build from 'release' branch
sudo ./build.sh -b master # build from a specific branch
```
Requires Debian / Ubuntu with `live-build` installed. The resulting ISO is
written to this directory as `WROLPi-v${VERSION}-amd64.iso`.
Flash with Etcher, Rufus, Raspberry Pi Imager, or:
```
sudo dd if=WROLPi-v${VERSION}-amd64.iso of=/dev/sdX bs=4M status=progress conv=fsync
```
## Layout on disk
The drive carries everything WROLPi needs. Postgres data lives alongside
the user's library so a drive moved between hosts keeps its full state:
```
/media/wrolpi/
videos/ ← user-facing media
archives/
zims/
map/
tags/
config/
wrolpi.yaml
controller.yaml
fstab.yaml
ssl/
postgresql/
data/ ← bind-mounted to /var/lib/postgresql
config/ ← bind-mounted to /etc/postgresql
```
## Implementation status
- [x] **Phase 1 — merged live + chroot config**: Single `debian-live-config/`
produces a Debian Live ISO with the full WROLPi stack, XFCE auto-login
as `wrolpi`, persistence partition created on first boot.
- [ ] **Phase 2 — Calamares installer**: Add `calamares` package and a
desktop launcher so users can install WROLPi to disk.
- [ ] **Phase 3 — branding + release**: WROLPi branding for Calamares,
CI on tag, hosting, wrolpi.org docs.
## References
- Debian Live Manual: https://live-team.pages.debian.net/live-manual/html/live-manual/index.en.html
- Persistence: https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

View file

@ -1,4 +1,12 @@
#!/bin/bash
# Build the WROLPi Debian Live + installer ISO.
#
# Output: an iso-hybrid image that boots into a full WROLPi system on any
# x86 PC. Users can run WROLPi directly off the USB (persistence partition
# created on first boot) or install to disk via the Calamares launcher on
# the desktop. Calamares wiring lands in phase 2; phase 1 ships the
# live-only flow.
#
# https://live-team.pages.debian.net/live-manual/html/live-manual/index.en.html
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
@ -23,8 +31,10 @@ while getopts ":hb:" option; do
esac
done
# Get version from the target git branch (not local checkout) so output filename matches actual content.
VERSION=$(curl -sL "https://raw.githubusercontent.com/lrnselfreliance/wrolpi/${BRANCH}/wrolpi/version.txt")
# Get version from the target git branch (not local checkout) so output
# filename matches actual content. Strip whitespace so stray newlines or
# spaces in version.txt do not corrupt the lb config metadata strings.
VERSION=$(curl -sL "https://raw.githubusercontent.com/lrnselfreliance/wrolpi/${BRANCH}/wrolpi/version.txt" | tr -d '[:space:]')
if [ -z "${VERSION}" ]; then
echo "ERROR: Could not fetch version from branch '${BRANCH}'"
exit 1
@ -37,20 +47,30 @@ if [ $EUID != 0 ]; then
exit $?
fi
# Clear out old builds.
[ -d "${BUILD_DIR}" ] && rm -rf "${BUILD_DIR}"
mkdir "${BUILD_DIR}"
# Clear out old builds. Strip immutable flag from blob files first — the
# chroot hook applies chattr +i to /opt/wrolpi-blobs/*; without removing
# that here, rm -rf leaves them behind and the next lb build's
# chroot_includes_after_packages rsync fails with "Operation not permitted"
# trying to overwrite them.
if [ -d "${BUILD_DIR}" ]; then
find "${BUILD_DIR}" -path '*/opt/wrolpi-blobs/*' -type f -exec chattr -i {} + 2>/dev/null || true
rm -rf "${BUILD_DIR}"
fi
mkdir -p "${BUILD_DIR}"
cd "${BUILD_DIR}" || (echo "Work directory must exist" && exit 1)
set -e
# Pure live config — no --debian-installer flags. The graphical installer
# (Calamares) is added in phase 2 as a package + desktop launcher rather
# than via lb's --debian-installer mechanism. noswap prevents the live
# system from activating swap partitions found on the host machine's
# internal disks.
lb config \
--binary-images iso-hybrid \
--mode debian \
--architectures amd64 \
--linux-flavours amd64 \
--distribution bookworm \
--debian-installer live \
--debian-installer-gui true \
--archive-areas "main contrib non-free non-free-firmware" \
--updates true \
--security true \
@ -63,6 +83,7 @@ lb config \
--color \
--linux-packages "linux-image linux-headers" \
--memtest memtest86+ \
--bootappend-live "boot=live components quiet splash noswap persistence persistence-storage=filesystem persistence-media=removable persistence-label=persistence live-config.username=wrolpi live-config.user-fullname=WROLPi live-config.hostname=wrolpi" \
--iso-volume "WROLPi v${VERSION}" \
--iso-application "WROLPi v${VERSION}" \
--iso-preparer WROLPi \
@ -103,13 +124,16 @@ fi
cp "${BLOB_CACHE}/map-fonts.tar.gz" "${BUILD_DIR}/config/includes.chroot/opt/wrolpi-blobs/"
cp "${BLOB_CACHE}/map-overview.pmtiles" "${BUILD_DIR}/config/includes.chroot/opt/wrolpi-blobs/"
# pipefail so an lb build failure isn't masked by tee's success exit code.
set -o pipefail
time nice -n 18 lb build 2>&1 | tee "${SCRIPT_DIR}/build.log"
set +o pipefail
grep "9999-wrolpi.hook.chroot completed" "${SCRIPT_DIR}/build.log" >/dev/null 2>&1 || (echo "build hook failed!" && exit 1)
cp "${BUILD_DIR}"/*iso "${SCRIPT_DIR}/" || (echo "Build failed. No ISOs were found!" && exit 1)
chmod 644 "${SCRIPT_DIR}"/*iso
chown 1000:1000 "${SCRIPT_DIR}"/*iso
chown "${SUDO_UID:-1000}:${SUDO_GID:-1000}" "${SCRIPT_DIR}"/*iso
DEST="${SCRIPT_DIR}/WROLPi-v${VERSION}-amd64.iso"
[ -f "${DEST}" ] && (echo "Removing conflicting ISO" && rm "${DEST}")
mv "${SCRIPT_DIR}"/*.iso "${DEST}"

View file

@ -1,5 +1,10 @@
#!/usr/bin/bash
# This script is used to generate a Debian Live ISO image.
# Chroot hook for the combined WROLPi Live + Installer ISO.
#
# Composes the full WROLPi stack into the live system during live-build's
# chroot phase. The resulting squashfs is what runs in live mode AND what
# the Calamares installer (phase 2) will copy onto the target disk, so any
# customisation here applies to both deployments.
set -e
set -x
@ -16,6 +21,28 @@ alias ll='ls -lh'
alias la='ll -a'
EOF
# Pre-create the wrolpi user. Same account is used for interactive desktop
# login (password 'wrolpi') AND as the User= for the wrolpi-* systemd
# services. live-config sees this existing user at boot and skips its own
# user creation (preserving our chosen password); the boot params set
# live-config.username=wrolpi so live-config's other scripts (sudo,
# lightdm autologin) target this account.
useradd -m -s /bin/bash -U -G sudo,audio,video,cdrom,plugdev,netdev wrolpi
echo "wrolpi:wrolpi" | chpasswd
# Rebrand from generic Debian so users don't think they have the wrong USB.
echo "wrolpi" > /etc/hostname
sed -i 's/127\.0\.1\.1.*/127.0.1.1\twrolpi/' /etc/hosts 2>/dev/null || \
echo "127.0.1.1 wrolpi" >> /etc/hosts
cat > /etc/issue <<'EOF'
WROLPi \n (\l)
EOF
cat > /etc/issue.net <<'EOF'
WROLPi %h
EOF
git --version
WROLPI_BRANCH="release"
@ -29,12 +56,21 @@ npm --version
node --version
npm install
# Build the React production bundle. Without this, /opt/wrolpi/app/build/
# does not exist and wrolpi-app.service (`serve build -l 3000 ...`) returns
# 404 for every request. Baking the build into the squashfs means the
# live image works offline immediately and the installed deployment skips
# a multi-minute first-boot step.
BROWSERSLIST_IGNORE_OLD_DATA=true npm run build
# Install Node console commands.
npm i -g serve@12.0.1 single-file-cli@2.0.73 readability-extractor@0.0.6
# Install Deno runtime.
# Install Deno runtime. Hash pinned to detect tampering of the upstream release artifact.
DENO_VERSION="v2.2.4"
DENO_SHA256="9a260a624cab6e61f7603957f94f638d68739d8c6ac8c38bc337521266603a7c"
curl -fsSL "https://github.com/denoland/deno/releases/download/${DENO_VERSION}/deno-x86_64-unknown-linux-gnu.zip" -o /tmp/deno.zip
echo "${DENO_SHA256} /tmp/deno.zip" | sha256sum -c -
unzip /tmp/deno.zip -d /usr/local/bin/
chmod +x /usr/local/bin/deno
rm /tmp/deno.zip
@ -82,6 +118,34 @@ mkdir /media/wrolpi
[[ -f /opt/wrolpi-blobs/map-overview.pmtiles && -s /opt/wrolpi-blobs/map-overview.pmtiles ]] || \
(echo "ERROR: map-overview.pmtiles missing from /opt/wrolpi-blobs!" && exit 1)
# Bake in pmtiles CLI so first boot does not need internet.
# Mirrors scripts/install_protomaps.sh; duplicated here because that script's
# migration logic relies on a running systemd, which is not present in chroot.
# Hash pinned to detect tampering of the upstream release artifact.
PMTILES_VERSION="v1.30.1"
PMTILES_SHA256="23a2a2222f658320b539ccd06ac3b9b3b803ecbdd39a6cb5249d2ce2e16e38ae"
curl -fsSL "https://github.com/protomaps/go-pmtiles/releases/download/${PMTILES_VERSION}/go-pmtiles_${PMTILES_VERSION#v}_Linux_x86_64.tar.gz" \
-o /tmp/pmtiles.tar.gz
echo "${PMTILES_SHA256} /tmp/pmtiles.tar.gz" | sha256sum -c -
tar -xz -C /usr/local/bin/ -f /tmp/pmtiles.tar.gz pmtiles
rm /tmp/pmtiles.tar.gz
chmod +x /usr/local/bin/pmtiles
pmtiles version
# Build and bake in tippecanoe (used to build map search indexes).
# Hash pinned to detect tampering of the upstream release tarball.
TIPPECANOE_VERSION="2.79.0"
TIPPECANOE_SHA256="b0fd9df49b6efc988288ea48774822c6de19eb48428017f27ee0b3b01d44f05d"
TIPPECANOE_BUILD_DIR=$(mktemp -d)
curl -fsSL "https://github.com/felt/tippecanoe/archive/refs/tags/${TIPPECANOE_VERSION}.tar.gz" \
-o /tmp/tippecanoe.tar.gz
echo "${TIPPECANOE_SHA256} /tmp/tippecanoe.tar.gz" | sha256sum -c -
tar -xz -C "${TIPPECANOE_BUILD_DIR}" --strip-components=1 -f /tmp/tippecanoe.tar.gz
rm /tmp/tippecanoe.tar.gz
(cd "${TIPPECANOE_BUILD_DIR}" && make -j"$(nproc)" && make install)
rm -rf "${TIPPECANOE_BUILD_DIR}"
tippecanoe --version
# Install WROLPi Help.
/opt/wrolpi/scripts/install_help_service.sh
@ -103,10 +167,54 @@ systemctl enable iperf3
systemctl enable wrolpi-first-startup.service
# Enable Controller service to start on boot.
systemctl enable wrolpi-controller.service
# Apply controller.yaml drives.mounts at boot (replaces /etc/fstab for
# secondary mounts so the mount table travels with the drive).
systemctl enable wrolpi-mounts.service
# Enable weekly certificate renewal check.
systemctl enable wrolpi-cert-renew.timer
# Enable weekly certificate renewal check.
systemctl enable wrolpi-cert-renew.timer
# Bootstrap service: on a Live USB it creates the persistence partition on
# first boot; on every boot (Live or Installed) it ensures a Postgres
# cluster exists on /media/wrolpi/config/postgresql and the wrolpi-* runtime
# services are enabled. Installed deployments find no live medium and
# short-circuit the partition step.
chmod +x /usr/local/sbin/wrolpi-bootstrap.sh
systemctl enable wrolpi-bootstrap.service
# Plymouth — replace Debian-branded splash with a minimal theme. live-build's
# initramfs regeneration at the end of the chroot phase will pick up this
# change so the new theme is used at boot.
plymouth-set-default-theme text || true
# LightDM greeter — add a WROLPi banner so the login screen identifies the
# system instead of looking like a stock Debian.
mkdir -p /etc/lightdm/lightdm-gtk-greeter.conf.d
cat > /etc/lightdm/lightdm-gtk-greeter.conf.d/50-wrolpi.conf <<'EOF'
[greeter]
banner-message-enable=true
banner-message-text=WROLPi
EOF
# LightDM auto-login — drop straight into the XFCE session as wrolpi without
# a password prompt. The wrolpi password ('wrolpi') is still required for
# sudo and screen unlock.
mkdir -p /etc/lightdm/lightdm.conf.d
cat > /etc/lightdm/lightdm.conf.d/50-wrolpi-autologin.conf <<'EOF'
[Seat:*]
autologin-user=wrolpi
autologin-user-timeout=0
autologin-session=xfce
EOF
# Drop the default Postgres cluster created by the postgresql-* package.
# wrolpi-bootstrap.service creates a fresh cluster on /media/wrolpi/config/postgresql
# at first boot regardless of deployment shape; the cluster files installed
# here would be masked by the bind-mount on Live and orphaned on Installed.
# Version derived from the installed binary so this stays correct if the
# package list is later bumped past postgresql-15.
PG_VERSION=$(pg_lsclusters -h | awk 'NR==1{print $1}')
[ -n "${PG_VERSION}" ] || (echo "ERROR: no Postgres cluster found to drop" && exit 1)
pg_dropcluster --stop "${PG_VERSION}" main
# Protect blob files from accidental modification or deletion.
if [ -d /opt/wrolpi-blobs ] && ls /opt/wrolpi-blobs/* >/dev/null 2>&1; then

View file

@ -0,0 +1,22 @@
[Unit]
Description=WROLPi bootstrap (persistence partition + Postgres bind-mounts + DB)
Documentation=https://github.com/lrnselfreliance/wrolpi
After=local-fs.target systemd-tmpfiles-setup.service
Before=postgresql.service wrolpi-api.service wrolpi-app.service wrolpi-kiwix.service wrolpi-controller.service wrolpi-first-startup.service wrolpi-mounts.service wrolpi.target
Wants=local-fs.target
# live-boot uses one of these mount points depending on version; on
# Installed none of them exist and RequiresMountsFor resolves to the
# nearest enclosing mount (the root filesystem). Listing all three keeps
# the unit correctly ordered on Live regardless of live-boot version.
RequiresMountsFor=/usr/lib/live/mount/medium /run/live/medium /lib/live/mount/medium /media/wrolpi
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/sbin/wrolpi-bootstrap.sh
StandardOutput=journal
StandardError=journal
TimeoutStartSec=600
[Install]
WantedBy=multi-user.target

View file

@ -0,0 +1,231 @@
#!/usr/bin/env bash
# WROLPi bootstrap — runs at every boot via wrolpi-bootstrap.service.
# Idempotent throughout. Works in all three deployment shapes:
#
# 1. Live USB, first boot. No persistence partition exists. This script
# creates one in the free space at the end of the boot drive, formats it
# as ext4 (label `persistence`), writes a persistence.conf so future
# boots pick it up automatically via live-boot, bind-mounts the
# persistence root over /media/wrolpi, creates the Postgres cluster
# under /media/wrolpi/config/postgresql/, runs initialize_api_db.sh,
# and enables the WROLPi runtime services.
#
# 2. Live USB, subsequent boot. live-boot's initramfs has already applied
# the /media/wrolpi bind mount from persistence.conf. Cluster and DB
# already exist. Most steps no-op; we always regenerate the leaf TLS
# cert and ensure services are enabled.
#
# 3. Installed (Calamares). No live medium present; partition-creation
# branch is skipped. /media/wrolpi is mounted via /etc/fstab on an
# external drive, or is a plain directory on the root filesystem if
# the user installed without one. The Postgres bind-mounts happen at
# every boot here because /etc/fstab doesn't own them — see Before=
# in the .service unit.
set -euo pipefail
log() { echo "[wrolpi-bootstrap] $*"; }
# Show progress on the Plymouth splash so the user sees what's happening
# during the 30-60s first-boot work. Plymouth is up between initramfs and
# lightdm starting; we block lightdm via systemd ordering, so the splash
# owns the screen for our entire run. Falls back to log-only if Plymouth
# isn't active (e.g. running this script manually over SSH).
progress() {
plymouth display-message --text="$1" 2>/dev/null || true
log "$1"
}
progress "Preparing WROLPi..."
# --- Detect Live vs Installed -------------------------------------------
# Live: one of the live-boot mount points is active and points at our USB.
# Installed: none of them exist; this script proceeds straight to the
# Postgres bind-mount section.
BOOT_DRIVE=""
for mp in /usr/lib/live/mount/medium /run/live/medium /lib/live/mount/medium; do
if part=$(findmnt -nr -o SOURCE "$mp" 2>/dev/null) && [ -n "$part" ]; then
BOOT_DRIVE="/dev/$(lsblk -no PKNAME "$part")"
log "live medium $part on $BOOT_DRIVE"
break
fi
done
if [ -z "$BOOT_DRIVE" ]; then
log "no live medium detected — running in Installed mode"
fi
# --- Find or create persistence partition (Live only) -------------------
if [ -n "$BOOT_DRIVE" ]; then
PERSIST_PART=$(blkid -L persistence 2>/dev/null || true)
if [ -z "$PERSIST_PART" ]; then
progress "Creating persistence partition on $BOOT_DRIVE..."
# Compute next 1-MiB-aligned start sector after the highest-ending
# existing partition. We need MAX(start+size), not LAST: iso-hybrid
# layouts list a small ESP (sdb2) AFTER the main data partition
# (sdb1) in sfdisk -d output even though sdb1 ends much later on
# disk. Parse via sed because sfdisk -d pads values with spaces
# ("start= 64,") which breaks naive awk.
LAST_END=$(sfdisk -d "$BOOT_DRIVE" \
| sed -n 's|^/dev/[^ ]* *: *start= *\([0-9]\+\), *size= *\([0-9]\+\).*|\1 \2|p' \
| awk '{e=$1+$2; if(e>m) m=e} END{print m+0}')
if [ "${LAST_END:-0}" -lt 1 ]; then
log "ERROR: failed to compute LAST_END from $BOOT_DRIVE partition table"
exit 1
fi
START=$(( ((LAST_END + 2047) / 2048) * 2048 ))
# --no-reread --force: required because the live medium is mounted from
# this same drive. Adding a new partition entry past the mounted one
# is safe; we use partx -a below to inform the running kernel without
# forcing a re-read of the whole partition table.
echo "start=$START, type=83" | sfdisk --append --no-reread --force "$BOOT_DRIVE"
partx -a "$BOOT_DRIVE" 2>/dev/null || true
# Identify the new partition by the start sector we just assigned to
# it. We deliberately avoid `lsblk | tail -1`: if partx -a or udev
# haven't registered the new node, that fallback returns the existing
# last partition (typically the ESP or the live data partition) — and
# the mkfs.ext4 -F below would wipe it. Matching on start sector is
# unambiguous: no existing partition can share it. Retry up to 5s
# for udev to settle and the partition node to appear.
PERSIST_PART=""
for _i in 1 2 3 4 5; do
udevadm settle || true
PERSIST_PART=$(lsblk -lnpo NAME,START "$BOOT_DRIVE" 2>/dev/null \
| awk -v s="$START" '$2 == s {print $1; exit}')
[ -n "$PERSIST_PART" ] && [ -b "$PERSIST_PART" ] && break
PERSIST_PART=""
sleep 1
done
if [ -z "$PERSIST_PART" ] || ! [ -b "$PERSIST_PART" ]; then
log "ERROR: failed to locate the newly created persistence partition"
exit 1
fi
progress "Formatting $PERSIST_PART as ext4..."
mkfs.ext4 -L persistence -F "$PERSIST_PART"
fi
# Mount the persistence partition + lay out the on-disk structure.
# /media/wrolpi is bound to /mnt/persistence/wrolpi by either this
# script (first boot) or live-boot's initramfs (subsequent boots). The
# Postgres bind-mounts are managed by THIS script in all cases — see
# the "Postgres bind-mounts" section below.
mkdir -p /mnt/persistence
mountpoint -q /mnt/persistence || mount "$PERSIST_PART" /mnt/persistence
mkdir -p /mnt/persistence/wrolpi
chown wrolpi:wrolpi /mnt/persistence/wrolpi
# persistence.conf: live-boot reads this on boot and applies the bind
# mount before systemd starts on subsequent boots. Only /media/wrolpi
# is here; Postgres bind-mounts are intentionally managed by this
# script (they live under /media/wrolpi/config/postgresql/ on the
# mounted /media/wrolpi, so once that bind is in place they are
# reachable).
cat > /mnt/persistence/persistence.conf <<'EOF'
/media/wrolpi bind,source=wrolpi
EOF
# Bind /media/wrolpi on first boot. live-boot will do this from the
# initramfs on subsequent boots and the call below is then a no-op.
if ! mountpoint -q /media/wrolpi; then
mkdir -p /media/wrolpi
mount --bind /mnt/persistence/wrolpi /media/wrolpi
fi
fi
# --- /media/wrolpi/config + Postgres data layout ------------------------
# Works regardless of whether /media/wrolpi is a bind (Live), a regular
# mount from /etc/fstab (Installed + external drive), or a plain
# directory on root (Installed without external drive).
if [ ! -d /media/wrolpi ]; then
mkdir -p /media/wrolpi
fi
chown wrolpi:wrolpi /media/wrolpi 2>/dev/null || true
mkdir -p /media/wrolpi/config
chown wrolpi:wrolpi /media/wrolpi/config
mkdir -p /media/wrolpi/config/postgresql/data \
/media/wrolpi/config/postgresql/config
chown postgres:postgres /media/wrolpi/config/postgresql \
/media/wrolpi/config/postgresql/data \
/media/wrolpi/config/postgresql/config
chmod 0700 /media/wrolpi/config/postgresql/data
# --- Postgres bind-mounts (idempotent) ----------------------------------
# /var/lib/postgresql holds the cluster data dir; /etc/postgresql holds
# the per-cluster configuration. Bind-mounting them onto subdirectories
# of /media/wrolpi means the entire Postgres state travels with the WROLPi
# drive.
bind_if_unmounted() {
local src=$1 dest=$2
mkdir -p "$dest"
mountpoint -q "$dest" || mount --bind "$src" "$dest"
}
bind_if_unmounted /media/wrolpi/config/postgresql/data /var/lib/postgresql
bind_if_unmounted /media/wrolpi/config/postgresql/config /etc/postgresql
# --- Postgres cluster ---------------------------------------------------
# Derive version from the installed Postgres binary so this stays correct
# if the package list is later bumped past postgresql-15.
PG_VERSION=$(ls /usr/lib/postgresql 2>/dev/null | sort -n | tail -1)
if [ -z "$PG_VERSION" ]; then
log "ERROR: no postgresql server package found under /usr/lib/postgresql"
exit 1
fi
if ! pg_lsclusters -h | awk '{print $1,$2}' | grep -q "^${PG_VERSION} main$"; then
progress "Setting up Postgres cluster..."
pg_createcluster "${PG_VERSION}" main
fi
progress "Starting Postgres..."
systemctl start "postgresql@${PG_VERSION}-main.service"
# --- WROLPi DB schema + initial config import ---------------------------
if ! sudo -iu postgres psql -lqt | cut -d \| -f 1 | grep -qw wrolpi; then
progress "Initializing WROLPi database (this can take a minute)..."
/opt/wrolpi/scripts/initialize_api_db.sh
fi
# --- Certificates -------------------------------------------------------
# CA persists in /media/wrolpi/config/ssl/ (created on first run). Leaf
# cert is regenerated every boot so SANs reflect the current hostname/IP.
progress "Generating TLS certificate..."
/opt/wrolpi/scripts/generate_certificates.sh
# --- Caddy: swap onboarding stub for the full Caddyfile if needed -------
if ! diff -q /opt/wrolpi/etc/raspberrypios/Caddyfile /etc/caddy/Caddyfile >/dev/null 2>&1; then
progress "Configuring Caddy..."
cp /opt/wrolpi/etc/raspberrypios/Caddyfile /etc/caddy/Caddyfile
systemctl restart caddy || true
fi
# Signal to the Controller that the primary drive is set up. Without this
# the Controller renders its onboarding wizard and offers the persistence
# partition as an unconfigured candidate. Empty YAML yields defaults.
if [ ! -e /media/wrolpi/config/controller.yaml ]; then
install -o wrolpi -g wrolpi -m 0644 /dev/null /media/wrolpi/config/controller.yaml
fi
# --- Enable and start the WROLPi runtime services -----------------------
# These units have implicit `After=wrolpi-bootstrap.service` (via our
# `Before=` in the unit file), so a blocking `systemctl --now` would
# deadlock: the runtime units won't start until bootstrap finishes, and
# bootstrap won't finish until systemctl returns. Enable, then start
# with --no-block so systemd queues the starts after we exit.
progress "Starting WROLPi services..."
systemctl enable wrolpi-api.service wrolpi-app.service wrolpi-kiwix.service
systemctl start --no-block wrolpi-api.service wrolpi-app.service wrolpi-kiwix.service
progress "WROLPi is ready."
log "bootstrap complete"

View file

@ -1,5 +0,0 @@
git
task-gnome-desktop
task-laptop
task-ssh-server
task-xfce-desktop

View file

@ -11,8 +11,10 @@ catdoc
chromium
cpufrequtils
curl
dosfstools
ethtool
exfat-fuse
exfatprogs
fdupes
ffmpeg
firefox-esr
@ -21,6 +23,7 @@ fonts-noto-hinted
fonts-noto-unhinted
gcc
gcc-doc
gdisk
git
git-doc
gnupg-agent
@ -43,6 +46,7 @@ libicu-dev
libpq-dev
libproj-dev
libreoffice
libsqlite3-dev
libssl-dev
libtiff5-dev
libtool
@ -53,6 +57,7 @@ nicstat
nodejs
npm
ntfs-3g
parted
pkg-config
postgresql-15
postgresql-doc
@ -78,3 +83,4 @@ vlc
wget
wpasupplicant
zim-tools
zlib1g-dev

View file

@ -1 +0,0 @@
d-i netcfg/get_hostname string wrolpi

View file

@ -4,7 +4,7 @@ Documentation=https://github.com/lrnselfreliance/wrolpi
# Read drives.mounts from controller.yaml and apply each entry that isn't
# already mounted. Replaces /etc/fstab entries for secondary drives so the
# mount table travels with the drive itself.
After=local-fs.target wrolpi-portable-bootstrap.service
After=local-fs.target wrolpi-bootstrap.service
RequiresMountsFor=/media/wrolpi
# Run before WROLPi services so secondary drives are ready when they start.
Before=wrolpi-api.service wrolpi-app.service wrolpi-kiwix.service wrolpi.target

View file

@ -1,138 +0,0 @@
#!/bin/bash
# Build WROLPi Portable — a bootable amd64 Live USB image.
#
# Output: an iso-hybrid image written to this directory. Users flash it to a
# USB drive (Etcher / Rufus / dd / Raspberry Pi Imager) and boot it on any
# x86 PC. Unlike ../debian-live-config/, this build is NOT an installer — it
# runs the full WROLPi stack live from the USB. Phase 2 will add a persistence
# partition so the Postgres DB and media directory survive reboots.
#
# https://live-team.pages.debian.net/live-manual/html/live-manual/index.en.html
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
BUILD_DIR=/var/tmp/wrolpi-build-portable
Help() {
echo "Build WROLPi Portable Live USB image."
echo
echo "Syntax: build.sh [-h] [-b BRANCH]"
echo "options:"
echo "h Print this help."
echo "b Build from this git BRANCH (default: 'release')."
echo
}
BRANCH="release"
while getopts ":hb:" option; do
case $option in
h) Help; exit ;;
b) BRANCH="${OPTARG}" ;;
*) echo "Error: Invalid option"; exit 1 ;;
esac
done
# Get version from the target git branch (not local checkout) so output filename matches actual content.
# Strip whitespace so newlines/spaces in version.txt don't corrupt the lb config metadata strings below.
VERSION=$(curl -sL "https://raw.githubusercontent.com/lrnselfreliance/wrolpi/${BRANCH}/wrolpi/version.txt" | tr -d '[:space:]')
if [ -z "${VERSION}" ]; then
echo "ERROR: Could not fetch version from branch '${BRANCH}'"
exit 1
fi
echo "Building WROLPi Portable version: ${VERSION} from branch: ${BRANCH}"
# Re-execute this script if it wasn't called with sudo.
if [ $EUID != 0 ]; then
sudo "$0" "$@"
exit $?
fi
# Clear out old builds. Strip immutable flag from blob files first — the
# chroot hook applies chattr +i to /opt/wrolpi-blobs/*; without removing
# that here, rm -rf leaves them behind and the next lb build's
# chroot_includes_after_packages rsync fails with "Operation not permitted"
# trying to overwrite them.
if [ -d "${BUILD_DIR}" ]; then
find "${BUILD_DIR}" -path '*/opt/wrolpi-blobs/*' -type f -exec chattr -i {} + 2>/dev/null || true
rm -rf "${BUILD_DIR}"
fi
mkdir -p "${BUILD_DIR}"
cd "${BUILD_DIR}" || (echo "Work directory must exist" && exit 1)
set -e
# Pure live config — no --debian-installer flags. The resulting image boots
# directly into a running WROLPi system, never offering to install to disk.
# noswap prevents the live system from activating swap partitions found on
# the host machine's internal disks (ethereal hygiene).
lb config \
--binary-images iso-hybrid \
--mode debian \
--architectures amd64 \
--linux-flavours amd64 \
--distribution bookworm \
--archive-areas "main contrib non-free non-free-firmware" \
--updates true \
--security true \
--apt-recommends false \
--firmware-binary true \
--firmware-chroot true \
--win32-loader false \
--checksums sha512 \
--clean \
--color \
--linux-packages "linux-image linux-headers" \
--bootappend-live "boot=live components quiet splash noswap persistence persistence-storage=filesystem persistence-media=removable persistence-label=persistence live-config.username=wrolpi live-config.user-fullname=WROLPi live-config.hostname=wrolpi" \
--iso-volume "WROLPi Portable v${VERSION}" \
--iso-application "WROLPi Portable v${VERSION}" \
--iso-preparer WROLPi \
--iso-publisher https://wrolpi.org
rsync -a "${SCRIPT_DIR}/config" "${BUILD_DIR}/"
# Write branch config for hook script.
mkdir -p "${BUILD_DIR}/config/includes.chroot/opt/wrolpi-blobs"
echo "${BRANCH}" > "${BUILD_DIR}/config/includes.chroot/opt/wrolpi-branch"
# Cache directory for downloaded blobs (persists across builds).
BLOB_CACHE="${SCRIPT_DIR}/files"
mkdir -p "${BLOB_CACHE}"
# Download map fonts (skip if cached).
if [ ! -f "${BLOB_CACHE}/map-fonts.tar.gz" ]; then
echo "Downloading map fonts..."
mkdir -p /tmp/map-fonts-dl
curl -fsSL https://github.com/protomaps/basemaps-assets/archive/refs/heads/main.tar.gz \
| tar -xz --strip-components=1 -C /tmp/map-fonts-dl basemaps-assets-main/fonts
tar -czf "${BLOB_CACHE}/map-fonts.tar.gz" -C /tmp/map-fonts-dl fonts
rm -rf /tmp/map-fonts-dl
fi
[[ -f "${BLOB_CACHE}/map-fonts.tar.gz" && -s "${BLOB_CACHE}/map-fonts.tar.gz" ]] || \
(echo "ERROR: Failed to download map fonts!" && exit 1)
# Download map overview (skip if cached).
if [ ! -f "${BLOB_CACHE}/map-overview.pmtiles" ]; then
echo "Downloading map overview..."
curl -fsSL https://wrolpi.nyc3.cdn.digitaloceanspaces.com/maps/map-overview.pmtiles \
-o "${BLOB_CACHE}/map-overview.pmtiles"
fi
[[ -f "${BLOB_CACHE}/map-overview.pmtiles" && -s "${BLOB_CACHE}/map-overview.pmtiles" ]] || \
(echo "ERROR: Failed to download map overview!" && exit 1)
# Copy cached blobs into the live image.
cp "${BLOB_CACHE}/map-fonts.tar.gz" "${BUILD_DIR}/config/includes.chroot/opt/wrolpi-blobs/"
cp "${BLOB_CACHE}/map-overview.pmtiles" "${BUILD_DIR}/config/includes.chroot/opt/wrolpi-blobs/"
# pipefail so an lb build failure isn't masked by tee's success exit code.
set -o pipefail
time nice -n 18 lb build 2>&1 | tee "${SCRIPT_DIR}/build.log"
set +o pipefail
grep "9999-wrolpi.hook.chroot completed" "${SCRIPT_DIR}/build.log" >/dev/null 2>&1 || (echo "build hook failed!" && exit 1)
cp "${BUILD_DIR}"/*iso "${SCRIPT_DIR}/" || (echo "Build failed. No ISOs were found!" && exit 1)
chmod 644 "${SCRIPT_DIR}"/*iso
chown 1000:1000 "${SCRIPT_DIR}"/*iso
DEST="${SCRIPT_DIR}/WROLPi-Portable-v${VERSION}-amd64.iso"
[ -f "${DEST}" ] && (echo "Removing conflicting ISO" && rm "${DEST}")
mv "${SCRIPT_DIR}"/*.iso "${DEST}"
echo "Build has completed. ISO output ${DEST}"

View file

@ -1,233 +0,0 @@
#!/usr/bin/bash
# Chroot hook for WROLPi Portable — composes the full WROLPi stack into the
# live system during live-build's chroot phase. Closely mirrors the installer
# ISO hook in ../../../debian-live-config/config/hooks/normal/, with two
# Portable-specific additions: bake in pmtiles/tippecanoe (offline-first
# first-boot) and drop the default Postgres cluster (Phase 2 firstboot will
# recreate it on the persistence partition).
set -e
set -x
# Generate and set default locale to avoid perl warnings.
# Users can change this later via: sudo dpkg-reconfigure locales
sed -i 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen
locale-gen
echo 'LANG=en_US.UTF-8' > /etc/default/locale
export LANG=en_US.UTF-8
cat >>/etc/profile <<'EOF'
alias ll='ls -lh'
alias la='ll -a'
EOF
# Pre-create the wrolpi user. Same account is used for interactive desktop
# login (password 'wrolpi') AND as the User= for the wrolpi-* systemd
# services. live-config sees this existing user at boot and skips its own
# user creation (preserving our chosen password); the boot params set
# live-config.username=wrolpi so live-config's other scripts (sudo,
# lightdm autologin) target this account.
useradd -m -s /bin/bash -U -G sudo,audio,video,cdrom,plugdev,netdev wrolpi
echo "wrolpi:wrolpi" | chpasswd
# Rebrand from generic Debian so users don't think they have the wrong USB.
echo "wrolpi" > /etc/hostname
sed -i 's/127\.0\.1\.1.*/127.0.1.1\twrolpi/' /etc/hosts 2>/dev/null || \
echo "127.0.1.1 wrolpi" >> /etc/hosts
cat > /etc/issue <<'EOF'
WROLPi Portable \n (\l)
EOF
cat > /etc/issue.net <<'EOF'
WROLPi Portable %h
EOF
git --version
WROLPI_BRANCH="release"
[ -f /opt/wrolpi-branch ] && WROLPI_BRANCH=$(cat /opt/wrolpi-branch)
git clone -b "${WROLPI_BRANCH}" https://github.com/lrnselfreliance/wrolpi.git /opt/wrolpi
rm -f /opt/wrolpi-branch
# Install webapp.
cd /opt/wrolpi/app
npm --version
node --version
npm install
# Build the React production bundle. Without this, /opt/wrolpi/app/build/
# does not exist and wrolpi-app.service (`serve build -l 3000 ...`) returns
# 404 for every request. RPi-OS / Debian-12 installs do this from repair.sh
# at first boot; we bake it into the squashfs so the live image works
# offline immediately.
BROWSERSLIST_IGNORE_OLD_DATA=true npm run build
# Install Node console commands.
npm i -g serve@12.0.1 single-file-cli@2.0.73 readability-extractor@0.0.6
# Install Deno runtime. Hash pinned to detect tampering of the upstream release artifact.
DENO_VERSION="v2.2.4"
DENO_SHA256="9a260a624cab6e61f7603957f94f638d68739d8c6ac8c38bc337521266603a7c"
curl -fsSL "https://github.com/denoland/deno/releases/download/${DENO_VERSION}/deno-x86_64-unknown-linux-gnu.zip" -o /tmp/deno.zip
echo "${DENO_SHA256} /tmp/deno.zip" | sha256sum -c -
unzip /tmp/deno.zip -d /usr/local/bin/
chmod +x /usr/local/bin/deno
rm /tmp/deno.zip
deno --version
# Install Python requirements. Try multiple times because pypi may stop responding.
python3 --version
python3 -m venv /opt/wrolpi/venv
/opt/wrolpi/venv/bin/pip3 install -r /opt/wrolpi/requirements.txt ||
/opt/wrolpi/venv/bin/pip3 install -r /opt/wrolpi/requirements.txt ||
/opt/wrolpi/venv/bin/pip3 install -r /opt/wrolpi/requirements.txt
# Install Controller with separate venv.
echo "Pre-installing WROLPi Controller..."
python3 -m venv /opt/wrolpi/controller/venv
/opt/wrolpi/controller/venv/bin/pip install --upgrade pip
/opt/wrolpi/controller/venv/bin/pip install -r /opt/wrolpi/controller/requirements.txt ||
/opt/wrolpi/controller/venv/bin/pip install -r /opt/wrolpi/controller/requirements.txt ||
/opt/wrolpi/controller/venv/bin/pip install -r /opt/wrolpi/controller/requirements.txt
echo "Controller pre-installed"
# Install Caddy from official apt repo.
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | tee /etc/apt/sources.list.d/caddy-stable.list
apt-get update
apt-get install -y caddy
# Configure Caddy.
mkdir -p /etc/caddy
cp /opt/wrolpi/etc/raspberrypios/Caddyfile /etc/caddy/Caddyfile
mkdir -p /var/www
cp /opt/wrolpi/etc/raspberrypios/50x.html /var/www/50x.html
# WROLPi needs a few privileged commands.
cp /opt/wrolpi/etc/raspberrypios/90-wrolpi /etc/sudoers.d/90-wrolpi
chmod 0440 /etc/sudoers.d/90-wrolpi
# Verify this new file is valid.
visudo -c -f /etc/sudoers.d/90-wrolpi
mkdir /media/wrolpi
# Verify blob files were copied in by includes.chroot (downloaded by build.sh).
[[ -f /opt/wrolpi-blobs/map-fonts.tar.gz && -s /opt/wrolpi-blobs/map-fonts.tar.gz ]] || \
(echo "ERROR: map-fonts.tar.gz missing from /opt/wrolpi-blobs!" && exit 1)
[[ -f /opt/wrolpi-blobs/map-overview.pmtiles && -s /opt/wrolpi-blobs/map-overview.pmtiles ]] || \
(echo "ERROR: map-overview.pmtiles missing from /opt/wrolpi-blobs!" && exit 1)
# Bake in pmtiles CLI so first boot does not need internet.
# Mirrors scripts/install_protomaps.sh; duplicated here because that script's
# migration logic relies on a running systemd, which is not present in chroot.
# Hash pinned to detect tampering of the upstream release artifact.
PMTILES_VERSION="v1.30.1"
PMTILES_SHA256="23a2a2222f658320b539ccd06ac3b9b3b803ecbdd39a6cb5249d2ce2e16e38ae"
curl -fsSL "https://github.com/protomaps/go-pmtiles/releases/download/${PMTILES_VERSION}/go-pmtiles_${PMTILES_VERSION#v}_Linux_x86_64.tar.gz" \
-o /tmp/pmtiles.tar.gz
echo "${PMTILES_SHA256} /tmp/pmtiles.tar.gz" | sha256sum -c -
tar -xz -C /usr/local/bin/ -f /tmp/pmtiles.tar.gz pmtiles
rm /tmp/pmtiles.tar.gz
chmod +x /usr/local/bin/pmtiles
pmtiles version
# Build and bake in tippecanoe (used to build map search indexes).
# Hash pinned to detect tampering of the upstream release tarball.
TIPPECANOE_VERSION="2.79.0"
TIPPECANOE_SHA256="b0fd9df49b6efc988288ea48774822c6de19eb48428017f27ee0b3b01d44f05d"
TIPPECANOE_BUILD_DIR=$(mktemp -d)
curl -fsSL "https://github.com/felt/tippecanoe/archive/refs/tags/${TIPPECANOE_VERSION}.tar.gz" \
-o /tmp/tippecanoe.tar.gz
echo "${TIPPECANOE_SHA256} /tmp/tippecanoe.tar.gz" | sha256sum -c -
tar -xz -C "${TIPPECANOE_BUILD_DIR}" --strip-components=1 -f /tmp/tippecanoe.tar.gz
rm /tmp/tippecanoe.tar.gz
(cd "${TIPPECANOE_BUILD_DIR}" && make -j"$(nproc)" && make install)
rm -rf "${TIPPECANOE_BUILD_DIR}"
tippecanoe --version
# Install WROLPi Help.
/opt/wrolpi/scripts/install_help_service.sh
cp /opt/wrolpi/etc/raspberrypios/wrolpi-*.service /etc/systemd/system/
cp /opt/wrolpi/etc/raspberrypios/wrolpi-*.timer /etc/systemd/system/
cp /opt/wrolpi/etc/raspberrypios/wrolpi.target /etc/systemd/system/
# Copy MOTD scripts, delete original.
cp /opt/wrolpi/etc/raspberrypios/motd/20-wrolpi.motd /etc/update-motd.d/20-wrolpi
cp /opt/wrolpi/etc/raspberrypios/motd/30-wrolpi-first.motd /etc/update-motd.d/30-wrolpi
chmod +x /etc/update-motd.d/*
truncate -s 0 /etc/motd
# NetworkManager for the hotspot.
systemctl enable NetworkManager
# Enable iperf3 for speed testing.
systemctl enable iperf3
# Enable first startup script to generate certificates and start Caddy.
systemctl enable wrolpi-first-startup.service
# Enable Controller service to start on boot.
systemctl enable wrolpi-controller.service
# Apply controller.yaml drives.mounts at boot (replaces /etc/fstab for
# secondary mounts so the mount table travels with the drive).
systemctl enable wrolpi-mounts.service
# Enable weekly certificate renewal check.
systemctl enable wrolpi-cert-renew.timer
# Bootstrap service: creates the persistence partition on first boot,
# Postgres cluster + DB on every boot (idempotent). The wrolpi-* runtime
# services (api/app/kiwix) are enabled BY this bootstrap, not here, so they
# cannot start until persistence + DB are ready.
chmod +x /usr/local/sbin/wrolpi-portable-bootstrap.sh
systemctl enable wrolpi-portable-bootstrap.service
# Plymouth — replace Debian-branded splash with a minimal theme. live-build's
# initramfs regeneration at the end of the chroot phase will pick up this
# change so the new theme is used at boot.
plymouth-set-default-theme text || true
# LightDM greeter — add a WROLPi banner so the login screen identifies the
# system instead of looking like a stock Debian.
mkdir -p /etc/lightdm/lightdm-gtk-greeter.conf.d
cat > /etc/lightdm/lightdm-gtk-greeter.conf.d/50-wrolpi.conf <<'EOF'
[greeter]
banner-message-enable=true
banner-message-text=WROLPi Portable
EOF
# LightDM auto-login — drop straight into the XFCE session as wrolpi without
# a password prompt. The wrolpi password ('wrolpi') is still required for
# sudo and screen unlock.
mkdir -p /etc/lightdm/lightdm.conf.d
cat > /etc/lightdm/lightdm.conf.d/50-wrolpi-autologin.conf <<'EOF'
[Seat:*]
autologin-user=wrolpi
autologin-user-timeout=0
autologin-session=xfce
EOF
# Drop the default Postgres cluster created by the postgresql-* package.
# On first boot, the persistence partition will be bind-mounted over
# /var/lib/postgresql; with the default cluster baked into the squashfs, the
# mount would mask it and Postgres would fail to start against an empty data
# dir. Phase 2's firstboot service creates a fresh cluster directly on the
# persistence partition.
# Version derived from the installed cluster so this stays correct if the
# package list is later bumped past postgresql-15.
PG_VERSION=$(pg_lsclusters -h | awk 'NR==1{print $1}')
[ -n "${PG_VERSION}" ] || (echo "ERROR: no Postgres cluster found to drop" && exit 1)
pg_dropcluster --stop "${PG_VERSION}" main
# Protect blob files from accidental modification or deletion.
if [ -d /opt/wrolpi-blobs ] && ls /opt/wrolpi-blobs/* >/dev/null 2>&1; then
chown -R root:root /opt/wrolpi-blobs
chmod 444 /opt/wrolpi-blobs/*
chattr +i /opt/wrolpi-blobs/*
fi
set +x
echo
echo "======================================================================"
echo "9999-wrolpi.hook.chroot completed"
echo "======================================================================"
echo

View file

@ -1,22 +0,0 @@
[Unit]
Description=WROLPi Portable bootstrap (persistence partition + DB)
Documentation=https://github.com/lrnselfreliance/wrolpi
After=local-fs.target systemd-tmpfiles-setup.service
Before=postgresql.service wrolpi-api.service wrolpi-app.service wrolpi-kiwix.service wrolpi-controller.service wrolpi-first-startup.service wrolpi.target
Wants=local-fs.target
# live-boot uses one of these mount points depending on version: newer
# releases default to /run/live/medium, older ones use /usr/lib/live/mount/medium
# or /lib/live/mount/medium. Cover all three so the unit is correctly ordered
# after whichever mount actually exists.
RequiresMountsFor=/usr/lib/live/mount/medium /run/live/medium /lib/live/mount/medium
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/sbin/wrolpi-portable-bootstrap.sh
StandardOutput=journal
StandardError=journal
TimeoutStartSec=600
[Install]
WantedBy=multi-user.target

View file

@ -1,209 +0,0 @@
#!/usr/bin/env bash
# WROLPi Portable bootstrap — runs at every boot via
# wrolpi-portable-bootstrap.service. Idempotent throughout.
#
# First boot: the persistence partition does not yet exist. This script
# creates it in the free space at the end of the boot drive, formats it as
# ext4 with label `persistence`, writes a persistence.conf so future boots
# pick it up automatically via live-boot, bind-mounts the source dirs over
# /var/lib/postgresql, /etc/postgresql, and /media/wrolpi, creates a fresh
# Postgres cluster, runs initialize_api_db.sh, and enables the WROLPi
# runtime services.
#
# Subsequent boots: live-boot's persistence machinery has already done the
# bind mounts (because the kernel cmdline includes `persistence
# persistence-label=persistence persistence-media=removable`). Cluster and
# DB already exist. Most steps no-op; we always regenerate the leaf TLS
# certificate to pick up the current hostname/IP, and we always ensure the
# wrolpi-* services are enabled and started.
set -euo pipefail
log() { echo "[wrolpi-portable-bootstrap] $*"; }
# Show progress on the Plymouth splash so the user sees what's happening
# during the 30-60s first-boot work. Plymouth is up between initramfs and
# lightdm starting; we block lightdm via systemd ordering, so the splash
# owns the screen for our entire run. Falls back to log-only if Plymouth
# isn't active (e.g. running this script manually over SSH).
progress() {
plymouth display-message --text="$1" 2>/dev/null || true
log "$1"
}
progress "Preparing WROLPi Portable..."
# --- Identify the boot drive ---------------------------------------------
# live-boot uses one of these mount points depending on version / config.
BOOT_PART=""
for mp in /usr/lib/live/mount/medium /run/live/medium /lib/live/mount/medium; do
if BOOT_PART=$(findmnt -nr -o SOURCE "$mp" 2>/dev/null) && [ -n "$BOOT_PART" ]; then
break
fi
done
if [ -z "$BOOT_PART" ]; then
log "ERROR: could not locate live medium mount point"
exit 1
fi
BOOT_DRIVE=$(lsblk -no PKNAME "$BOOT_PART")
BOOT_DRIVE="/dev/${BOOT_DRIVE}"
log "live medium $BOOT_PART on $BOOT_DRIVE"
# --- Find or create persistence partition --------------------------------
PERSIST_PART=$(blkid -L persistence 2>/dev/null || true)
if [ -z "$PERSIST_PART" ]; then
progress "Creating persistence partition on $BOOT_DRIVE..."
# Compute next 1-MiB-aligned start sector after the highest-ending existing
# partition. We need MAX(start+size), not LAST: iso-hybrid layouts list a
# small ESP (sdb2) AFTER the main data partition (sdb1) in sfdisk -d output
# even though sdb1 ends much later on disk. Parse via sed because sfdisk -d
# pads values with spaces ("start= 64,") which breaks naive awk.
LAST_END=$(sfdisk -d "$BOOT_DRIVE" \
| sed -n 's|^/dev/[^ ]* *: *start= *\([0-9]\+\), *size= *\([0-9]\+\).*|\1 \2|p' \
| awk '{e=$1+$2; if(e>m) m=e} END{print m+0}')
if [ "${LAST_END:-0}" -lt 1 ]; then
log "ERROR: failed to compute LAST_END from $BOOT_DRIVE partition table"
exit 1
fi
START=$(( ((LAST_END + 2047) / 2048) * 2048 ))
# --no-reread --force: required because the live medium is mounted from
# this same drive. Adding a new partition entry past the mounted one is
# safe; we use partx -a below to inform the running kernel without forcing
# a re-read of the whole partition table.
echo "start=$START, type=83" | sfdisk --append --no-reread --force "$BOOT_DRIVE"
partx -a "$BOOT_DRIVE" 2>/dev/null || true
# Identify the new partition by the start sector we just assigned to it.
# We deliberately avoid `lsblk | tail -1` here: if partx -a or udev have
# not registered the new node yet, that fallback would return the
# existing last partition (typically the ESP or the live data partition)
# — and the mkfs.ext4 -F below would wipe it. Matching on start sector
# is unambiguous: no existing partition can share it. Retry up to 5s
# for udev to settle and the partition node to appear.
PERSIST_PART=""
for _i in 1 2 3 4 5; do
udevadm settle || true
PERSIST_PART=$(lsblk -lnpo NAME,START "$BOOT_DRIVE" 2>/dev/null \
| awk -v s="$START" '$2 == s {print $1; exit}')
[ -n "$PERSIST_PART" ] && [ -b "$PERSIST_PART" ] && break
PERSIST_PART=""
sleep 1
done
if [ -z "$PERSIST_PART" ] || ! [ -b "$PERSIST_PART" ]; then
log "ERROR: failed to locate the newly created persistence partition"
exit 1
fi
progress "Formatting $PERSIST_PART as ext4..."
mkfs.ext4 -L persistence -F "$PERSIST_PART"
fi
# --- Mount persistence and prepare source-dir layout ---------------------
mkdir -p /mnt/persistence
mountpoint -q /mnt/persistence || mount "$PERSIST_PART" /mnt/persistence
mkdir -p /mnt/persistence/postgresql-data \
/mnt/persistence/postgresql-config \
/mnt/persistence/wrolpi
chown postgres:postgres /mnt/persistence/postgresql-data /mnt/persistence/postgresql-config
chmod 0700 /mnt/persistence/postgresql-data
chown wrolpi:wrolpi /mnt/persistence/wrolpi
# Write persistence.conf so live-boot's initramfs persistence machinery can
# apply these bind mounts before systemd starts on subsequent boots.
#
# `bind` (not `union`) is critical: `union` instructs live-boot to apply an
# OverlayFS union mount, which PostgreSQL explicitly does not support and
# which would silently corrupt data from the second boot onward. On first
# boot bind_if_unmounted below does a real bind mount; on subsequent boots
# live-boot's initramfs reads this file and must do the same.
cat > /mnt/persistence/persistence.conf <<EOF
/var/lib/postgresql bind,source=postgresql-data
/etc/postgresql bind,source=postgresql-config
/media/wrolpi bind,source=wrolpi
EOF
# --- Bind mounts (idempotent — only mount if not already mounted) -------
bind_if_unmounted() {
local src=$1 dest=$2
mkdir -p "$dest"
mountpoint -q "$dest" || mount --bind "$src" "$dest"
}
bind_if_unmounted /mnt/persistence/postgresql-data /var/lib/postgresql
bind_if_unmounted /mnt/persistence/postgresql-config /etc/postgresql
bind_if_unmounted /mnt/persistence/wrolpi /media/wrolpi
# --- Postgres cluster ----------------------------------------------------
# Derive version from the installed Postgres binary so this stays correct if
# the package list is later bumped past postgresql-15. This matches what
# the chroot hook does when dropping the default cluster.
PG_VERSION=$(ls /usr/lib/postgresql 2>/dev/null | sort -n | tail -1)
if [ -z "$PG_VERSION" ]; then
log "ERROR: no postgresql server package found under /usr/lib/postgresql"
exit 1
fi
if ! pg_lsclusters -h | awk '{print $1,$2}' | grep -q "^${PG_VERSION} main$"; then
progress "Setting up Postgres cluster..."
pg_createcluster "${PG_VERSION}" main
fi
progress "Starting Postgres..."
systemctl start "postgresql@${PG_VERSION}-main.service"
# --- WROLPi DB schema + initial config import ----------------------------
if ! sudo -iu postgres psql -lqt | cut -d \| -f 1 | grep -qw wrolpi; then
progress "Initializing WROLPi database (this can take a minute)..."
/opt/wrolpi/scripts/initialize_api_db.sh
fi
# --- Certificates --------------------------------------------------------
# CA persists in /media/wrolpi/config/ssl/ (created on first run). Leaf
# cert is regenerated every boot so SANs reflect the current hostname/IP.
progress "Generating TLS certificate..."
/opt/wrolpi/scripts/generate_certificates.sh
# --- Caddy: swap onboarding stub for the full Caddyfile if needed --------
if ! diff -q /opt/wrolpi/etc/raspberrypios/Caddyfile /etc/caddy/Caddyfile >/dev/null 2>&1; then
progress "Configuring Caddy..."
cp /opt/wrolpi/etc/raspberrypios/Caddyfile /etc/caddy/Caddyfile
systemctl restart caddy || true
fi
# --- /media/wrolpi ownership + config dir --------------------------------
chown wrolpi:wrolpi /media/wrolpi 2>/dev/null || true
mkdir -p /media/wrolpi/config
chown wrolpi:wrolpi /media/wrolpi/config
# Signal to the Controller that the primary drive is set up. The Controller
# treats /media/wrolpi/config/controller.yaml as the "drive is configured"
# sentinel (see controller.lib.config.is_primary_drive_mounted); without it
# the Controller renders its onboarding wizard and offers the persistence
# partition as an unconfigured candidate. Empty YAML is valid and yields
# the defaults.
if [ ! -e /media/wrolpi/config/controller.yaml ]; then
install -o wrolpi -g wrolpi -m 0644 /dev/null /media/wrolpi/config/controller.yaml
fi
# --- Enable and start the WROLPi runtime services -----------------------
# These units declare `After=wrolpi-portable-bootstrap.service` (via our
# Before= here in this unit), so a blocking `systemctl --now` would deadlock:
# the runtime units won't start until bootstrap finishes, and bootstrap won't
# finish until systemctl returns. Enable, then start with --no-block so
# systemd queues the starts after we exit.
progress "Starting WROLPi services..."
systemctl enable wrolpi-api.service wrolpi-app.service wrolpi-kiwix.service
systemctl start --no-block wrolpi-api.service wrolpi-app.service wrolpi-kiwix.service
progress "WROLPi Portable is ready."
log "bootstrap complete"

View file

@ -1,86 +0,0 @@
apt-transport-https
aria2
autoconf
bind9-dnsutils
build-essential
bzip2
ca-certificates
calibre
cargo
catdoc
chromium
cpufrequtils
curl
dosfstools
ethtool
exfat-fuse
exfatprogs
fdupes
ffmpeg
firefox-esr
fonts-noto-cjk
fonts-noto-hinted
fonts-noto-unhinted
gcc
gcc-doc
gdisk
git
git-doc
gnupg-agent
gpsd
hostapd
htop
iotop
iperf3
kiwix
kiwix-tools
libboost-all-dev
libbz2-dev
libcairo2-dev
libcurl4-gnutls-dev
libffi-dev
libfreetype6-dev
libgeos++-dev
libgeos-dev
libicu-dev
libpq-dev
libproj-dev
libreoffice
libsqlite3-dev
libssl-dev
libtiff5-dev
libtool
libxml2-dev
netcat-openbsd
network-manager
nicstat
nodejs
npm
ntfs-3g
parted
pkg-config
postgresql-15
postgresql-doc
pv
python3-dev
python3-doc
python3-full
python3-pip
python3-venv
rsync
samba
smartmontools
software-properties-common
sudo
sysstat
tar
tcpdump
tmux
unrar-free
unzip
vim
vlc
wget
wpasupplicant
zim-tools
zlib1g-dev

View file

@ -1,2 +0,0 @@
*.pmtiles
*.tar.gz