wrolpi/repair.sh

218 lines
8.7 KiB
Bash
Raw Normal View History

#!/usr/bin/env bash
# This script will attempt to repair a WROLPi installation. It will not use internet.
cd /opt/wrolpi || (echo "Cannot repair. /opt/wrolpi does not exist" && exit 1)
# Re-execute this script if it wasn't called with sudo.
if [ $EUID != 0 ]; then
sudo "$0" "$@"
exit $?
fi
rpi=false
if (grep 'Raspberry Pi' /proc/cpuinfo >/dev/null); then
rpi=true
fi
set -e
set -x
# Stop services if they are running.
systemctl stop wrolpi-controller.service || :
systemctl stop wrolpi-api.service || :
systemctl stop wrolpi-app.service || :
systemctl stop wrolpi-kiwix.service || :
systemctl stop caddy || :
# Clear Python bytecode cache from virtual environments to fix potential corruption.
echo "Clearing Python bytecode cache..."
find /opt/wrolpi/venv -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || :
find /opt/wrolpi/venv -name "*.pyc" -delete 2>/dev/null || :
find /opt/wrolpi/controller/venv -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || :
find /opt/wrolpi/controller/venv -name "*.pyc" -delete 2>/dev/null || :
find /opt/wrolpi-help/venv -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || :
find /opt/wrolpi-help/venv -name "*.pyc" -delete 2>/dev/null || :
# Create the WROLPi user
grep wrolpi: /etc/passwd || useradd -md /home/wrolpi wrolpi -s "$(command -v bash)"
# Reset any inadvertent changes to the WROLPi repo. Restore ownership in case this repair script fails.
git config --global --add safe.directory /opt/wrolpi
git reset HEAD --hard
chown -R wrolpi:wrolpi /opt/wrolpi
# Scripts are run by systemd services (e.g. wrolpi-cert-renew.service) which fail with
# "Permission denied" if the executable bit was lost.
chmod +x /opt/wrolpi/*.sh /opt/wrolpi/scripts/*.sh
# Copy configs to system.
mkdir -p /etc/caddy
cp /opt/wrolpi/etc/raspberrypios/Caddyfile /etc/caddy/Caddyfile
mkdir -p /var/www
cp /opt/wrolpi/etc/raspberrypios/50x.html /var/www/50x.html
Hotspot captive portal: welcome page with the WROLPi's address (#709) * Hotspot captive portal: welcome page with the WROLPi's address Devices that join the hotspot probe a fixed plain-HTTP URL to check for Internet access. NetworkManager's shared-mode dnsmasq now resolves those probe hostnames (and `wrolpi`) to the hotspot address via an include the Controller writes when the hotspot starts, the probes land on the Controller (port 80), and are redirected to /portal: "This is your WROLPi, its address is 10.42.0.1, open WROLPi / open the Controller". Only the probe hostnames are hijacked so the hotspot keeps forwarding everything else when the WROLPi has an uplink. The include is corrected and the connection re-activated once if NetworkManager picked a non-default subnet. Writing the include is best-effort; the hotspot starts regardless. `hotspot.captive_portal` (controller.yaml, default true) toggles the probe redirects and the include; like the other hotspot settings it applies on the next hotspot start, so saving while the hotspot is up reports restart_required and both Controller UIs say so. The status endpoint gains the hotspot address and portal URL, shown in both UIs. repair.sh only creates the include directory so a disabled setting survives upgrades. Native only: Docker has no hotspot. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * captive portal: address review * React: re-read the hotspot address whenever the HotspotToggle starts or stops the hotspot, matching the fallback UI's refresh-on-status. * A WPA3 start that reverts to WPA2 leaves the hotspot up, so it now gets the captive portal address correction too. * The corrective `nmcli connection up` logs a warning on a non-zero exit; dnsmasq would otherwise keep the old address silently. * HotspotStatusResponse.captive_portal defaults to true like everywhere else. * Trim design narrative from the module docstring and React comments. * The include-before-activate test now observes the include at the moment of the `nmcli ... hotspot` call instead of inferring from call order. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * captive portal: never forward any record type for probe hosts `address=` only synthesizes A/AAAA answers. iOS also asks for the HTTPS record type; dnsmasq forwarded that upstream, Apple's answer aliased to captive.g.aaplimg.com, and the phone fetched the genuine "Success" from Apple instead of the portal. Each probe host now also gets `local=`, so dnsmasq answers every type itself, and the CDN alias is a probe host too. Observed with dnsmasq query logging on a hotspot with an eth0 uplink. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * captive portal: let acknowledged clients finish so the OS offers "Done" A phone's sign-in sheet only shows "Done" once the OS re-probes and gets the genuine success reply. The Controller now remembers each client (by hotspot IP) that has loaded the portal page and answers its later probes with the reply each OS expects (Apple "Success" HTML, Android 204, Windows "Microsoft Connect Test", ...). The set is cleared when the hotspot starts so a new session shows the page again. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * captive portal: acknowledge on Continue, not on view iOS fetches the portal page in the background before opening the sheet, then re-probes after the sheet's own navigation. Acknowledging on view made that re-probe return "Success" as the sheet's content. Phones only re-check the network (and offer "Done") after a full-page navigation, so the welcome page now has a Continue button leading to /portal/continue, which is the only place a client is acknowledged. The Continue page says to tap Done and links to WROLPi and the Controller. Also hijack clients3.google.com (older Android probe host). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * help: bump wrolpi-help to the captive portal and Speed Test docs Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * captive portal: comments describe the Continue design; clear acks on stop Three comments still described acknowledging on view, which c72a7f7b removed. Acknowledged clients are now also cleared when the hotspot stops, so a lease reused in a later session sees the portal page. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-13 20:09:48 -06:00
# The Controller writes the hotspot captive-portal dnsmasq include here when the hotspot starts.
# Never copy a file into it: the include only exists while the setting is enabled.
mkdir -p /etc/NetworkManager/dnsmasq-shared.d
# Generate certificate for HTTPS.
/opt/wrolpi/scripts/generate_certificates.sh
# Install map fonts from blob (needed for MapLibre map labels).
MAP_FONTS_BLOB=/opt/wrolpi-blobs/map-fonts.tar.gz
if [ -f "${MAP_FONTS_BLOB}" ]; then
mkdir -p /opt/wrolpi/modules/map/static/fonts
tar -xzf "${MAP_FONTS_BLOB}" -C /opt/wrolpi/modules/map/static/
fi
# Start Caddy quickly so user can access Controller from React UI
systemctl start caddy || :
# WROLPi needs a few privileged commands.
cp /opt/wrolpi/etc/raspberrypios/90-wrolpi /etc/sudoers.d/90-wrolpi
chmod 0440 /etc/sudoers.d/90-wrolpi
# Verify this new file is valid.
visudo -c -f /etc/sudoers.d/90-wrolpi
# Update the Portable bootstrap on boxes installed from the Live USB; it is
# otherwise frozen at image-build time.
LIVE_INCLUDES=/opt/wrolpi/debian-live-config/config/includes.chroot
if [ -f /usr/local/sbin/wrolpi-bootstrap.sh ]; then
cp "${LIVE_INCLUDES}/usr/local/sbin/wrolpi-bootstrap.sh" /usr/local/sbin/wrolpi-bootstrap.sh
chmod 0755 /usr/local/sbin/wrolpi-bootstrap.sh
if [ -f /etc/systemd/system/wrolpi-bootstrap.service ]; then
cp "${LIVE_INCLUDES}/etc/systemd/system/wrolpi-bootstrap.service" /etc/systemd/system/wrolpi-bootstrap.service
fi
fi
# Install the systemd services
cp /opt/wrolpi/etc/raspberrypios/wrolpi*.service /etc/systemd/system/
cp /opt/wrolpi/etc/raspberrypios/wrolpi-*.timer /etc/systemd/system/
cp /opt/wrolpi/etc/raspberrypios/wrolpi.target /etc/systemd/system/
systemctl enable wrolpi-controller.service
systemctl enable wrolpi-api.service
systemctl enable wrolpi-app.service
systemctl enable wrolpi-kiwix.service
systemctl enable wrolpi-help.service
Moving WROLPi-managed mounts off /etc/fstab onto a drive-resident fstab.yaml. A WROLPi drive can move between hosts and across deployment styles (Portable, RPi, Debian). /etc/fstab is host-specific and on the Portable image it's part of a read-only squashfs, so persisting secondary mounts there was both wrong-host and lost-on-reboot. This change makes /media/wrolpi/config/fstab.yaml the single source of truth for WROLPi- managed mounts and introduces a new wrolpi-mounts.service that reconciles live state to match. Architecture: - The Controller writes intent (fstab.yaml) but never calls mount(8) or umount(8) itself. Every mutation triggers `systemctl restart wrolpi-mounts.service`; the Controller then verifies the outcome via the same MountExecutor abstraction the service uses. - wrolpi-mounts.service runs a thin entry point that constructs a real Reconciler with a SubprocessMountExecutor and calls apply(). The reconciler mounts what's in fstab.yaml but missing from the live state, and unmounts paths it previously claimed that are no longer in fstab.yaml. A managed-set file in /run/wrolpi-mounts/managed prevents the reconciler from ever touching user-mounted paths it didn't own. - One-time migration (marker-gated) lifts both /etc/fstab WROLPi entries and any controller.yaml drives.mounts entries into fstab.yaml. /etc/fstab is backed up to /etc/fstab.wrolpi-migration.backup; migrated lines are commented in place with a "migrated to fstab.yaml:" prefix so a host admin can hand-recover. Test approach: dependency injection in place of patching. The MountExecutor protocol has SubprocessMountExecutor (production) and FakeMountExecutor (in-memory, records calls). The Reconciler accepts an executor in its constructor, so the 20 reconciler tests + 13 migration tests + 18 fstab_yaml tests use real files in tmp_path with zero mock.patch calls. test_disks_api.py drops to mocks only for the host-touching read helpers; the mount/unmount paths now run the real Reconciler against the fake executor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 22:02:30 -06:00
systemctl enable wrolpi-mounts.service
systemctl enable wrolpi-cert-renew.timer
# Repair Controller (offline-safe - no pip install)
repair_controller() {
echo "Checking WROLPi Controller..."
# Check venv exists
if [ ! -f /opt/wrolpi/controller/venv/bin/python ]; then
echo "ERROR: Controller venv missing!"
echo "Run install.sh or upgrade.sh (requires internet) to reinstall."
return 1
fi
# Check venv is functional
if ! /opt/wrolpi/controller/venv/bin/python --version > /dev/null 2>&1; then
echo "ERROR: Controller venv is corrupted!"
echo "Run install.sh or upgrade.sh (requires internet) to reinstall."
return 1
fi
# Check systemd service is installed
if [ ! -f /etc/systemd/system/wrolpi-controller.service ]; then
echo "Controller systemd service not installed, copying..."
cp /opt/wrolpi/etc/raspberrypios/wrolpi-controller.service /etc/systemd/system/
systemctl daemon-reload
fi
# Ensure service is enabled
if ! systemctl is-enabled wrolpi-controller > /dev/null 2>&1; then
echo "Enabling Controller service..."
systemctl enable wrolpi-controller
fi
echo "Controller repair completed"
}
# Run Controller repair if venv exists (skip on fresh install)
if [ -d /opt/wrolpi/controller/venv ]; then
repair_controller || echo "Controller repair failed, continuing..."
fi
# Start Controller so user can monitor repair in UI.
systemctl start wrolpi-controller
/usr/bin/systemctl daemon-reload
# WROLPi needs a few privileged commands.
cp /opt/wrolpi/etc/raspberrypios/90-wrolpi /etc/sudoers.d/90-wrolpi
chmod 0440 /etc/sudoers.d/90-wrolpi
# Verify this new file is valid.
visudo -c -f /etc/sudoers.d/90-wrolpi
Postgres -> SQLite hard cutover: the database moves into the media directory. The database now lives at <media>/config/wrolpi.db so a library (files, configs, AND database) travels with the drive between WROLPi systems. Core: - wrolpi/db.py: lazy SQLite engine factory (WAL, busy_timeout=30s, foreign_keys=ON); get_db_curs yields a plain sqlite3.Cursor; native sqlite3 paramstyle everywhere (no compatibility shims); global sqlite3 adapters keep raw-SQL datetime formats identical to the ORM's. - wrolpi/fts.py: FTS5 external-content search replacing tsvector/GIN. Injection-proof websearch->FTS5 translator, bm25(10,4,2,1) reproducing A/B/C/D weighting, fast(abc)/deep column filtering, snippet headlines. Stemming contract pinned by dedicated tests (watch/watches/watched/ watching are interchangeable). - wrolpi/schema_ddl.py: SQLite triggers shared verbatim by the alembic baseline and test DBs; fixes the channel video_count DELETE/UPDATE bugs and a latent total_size bug (it used the video id as channel id). - wrolpi/db_bootstrap.py: startup auto-create/migrate; refuses databases written by a newer WROLPi (db_version_mismatch); SQLite 3.40 floor; network-filesystem refusal; never creates/migrates/trusts a database beneath an unmounted production media directory (shadow-DB guard). - Alembic: 85 Postgres-era migrations squashed into one baseline; revision ids are dates (2026_07_10_1350) so filenames self-sort and alembic_version says when a schema was created; render_as_batch=True. - All raw Postgres SQL rewritten (ANY->json_each IN, array_agg @> -> HAVING COUNT(DISTINCT), unnest->executemany/UPDATE..FROM, generate_series histograms->strftime GROUP BY, DISTINCT ON-> row_number(), writable CTEs->two statements, jsonb_*->json_*, explicit NULLS FIRST/LAST, LIMIT-before-OFFSET). Simple statements converted to SQLAlchemy instead (flag upsert, download retry/summary/tag checks, channel-by-tag estimate, Directory delete). - Config imports count skipped entries; background dumps are blocked after a lossy import so partial imports can never overwrite YAML configs (they are the sole source of truth for rebuilds now). - Search tolerates explicit JSON nulls for limit/offset (Postgres treated OFFSET NULL as 0; SQLite errors). - file_group.id/doc_section.id render as INTEGER PRIMARY KEY on SQLite (BIGINT is not a rowid alias; FTS5 content_rowid requires it). Ops teardown: docker-compose db service, CI postgres sidecars, initialize_api_db.sh, controller postgresql service, psycopg2-binary all removed. upgrade.sh stops/disables postgresql (data is NOT deleted; `systemctl enable --now postgresql` restores it); systemd units no longer Want postgresql.service. Existing-user upgrade: first boot creates+migrates the DB, configs import (tags recreate their FileGroups from disk), a refresh rebuilds the library index. QA'd on the 10.0.0.8 RPi: rebuild matched the Postgres baseline exactly; survived an uncontrolled hard reboot with clean integrity checks. Accepted behavior changes: FTS5 has no stopwords; bm25 length normalization shifts some rank orders; fast-path d_headline is un-highlighted; all-negative queries match nothing. Deferred: Debian/trixie package lists (incl. sqlite3 CLI), Cypress re-baseline, VACUUM INTO snapshots, exFAT/NTFS warning, auto-kicked rebuild refresh with progress UX. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 13:59:20 -06:00
# WROLPi uses SQLite inside the media directory; the API creates/migrates the database at
# startup. Never fatal: the media drive may not be mounted yet.
(cd /opt/wrolpi && /opt/wrolpi/venv/bin/python3 /opt/wrolpi/main.py db upgrade) || \
echo "WARNING: db upgrade failed (media directory unavailable?); the API will retry at startup."
# Create the media directory. This should be mounted by the maintainer.
[ -d /media/wrolpi ] || mkdir /media/wrolpi
# Create config directory if external drive is mounted, and is empty.
if grep -qs /media/wrolpi /proc/mounts && [ -z "$(ls -A /media/wrolpi)" ] && [ ! -d /media/wrolpi/config ]; then
mkdir /media/wrolpi/config
fi
Bake the app build into images; rebuild on-device only when stale (#536) * Bake the app build into images; rebuild on-device only when stale Previously the Raspberry Pi image shipped without app/build/ and compiled the React bundle on first boot via repair.sh -- a multi-minute, memory-hungry step that can OOM a 2GB Pi. The Debian Live ISO already baked the build in, so the two paths had drifted. Add scripts/build_app.sh as the single source of truth: it hashes the build inputs (src/, public/, package.json, package-lock.json, tsconfig.json) and runs `npm run build` (with GENERATE_SOURCEMAP=false) only when build/ is missing or those inputs changed, then writes build/.build-stamp. Wire all three call sites to it: - pi-gen 03-run-chroot.sh now builds after npm install, baking build/ into the Pi image like the Debian hook already does. - debian-live 9999-wrolpi.hook.chroot uses it so the baked build is stamped. - repair.sh uses it, so first-boot/manual-repair/install/upgrade all become no-ops when the shipped build is current. Result: fresh images ship a stamped build/, so a weak Pi never compiles on boot or repair; an on-device rebuild happens only when an upgrade actually changes app/. build_app.sh never runs npm install, so repair.sh keeps its offline guarantee. * build_app.sh: rebuild when built output is incomplete, not just missing Addresses review feedback: a matching stamp only proves the inputs are unchanged, not that the output survived. repair.sh exists to fix corruption, so a build/ with index.html + stamp but a missing JS/CSS asset must still rebuild. Verify every file referenced by build/asset-manifest.json exists (and that the manifest itself parses); rebuild otherwise. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 15:19:15 -06:00
# Build the frontend app -- only if build/ is missing or the app source changed.
# The shipped image already contains a current build/, so on a healthy install
# (including a weak Pi) this is a no-op; it only rebuilds after an upgrade that
# changed app/, or if build/ went missing. Stays offline: it never npm-installs.
/opt/wrolpi/scripts/build_app.sh
# Change owner of the media directory, ignore any errors because the drive may be fat/exfat/etc.
chown wrolpi:wrolpi /media/wrolpi 2>/dev/null || echo "Ignoring failure to change media directory permissions."
# Ensure the config directory is owned by wrolpi so the API can write config files and the SQLite DB.
[ -d /media/wrolpi/config ] && chown -R wrolpi:wrolpi /media/wrolpi/config 2>/dev/null || :
# Remove immutable flag from blob files before chown (may not exist or may not be set).
chattr -i /opt/wrolpi-blobs/* 2>/dev/null || :
# Ship committed blobs (e.g. the default Zim served by Kiwix when the media dir
# has no working zims) into /opt/wrolpi-blobs. repair.sh runs on every install
# and upgrade, so this keeps the blobs current there just like the image builds
# do. It runs after the immutable flag is cleared above (so existing blobs can
# be overwritten) and before immutability is re-applied below.
mkdir -p /opt/wrolpi-blobs
if [ -d /opt/wrolpi/wrolpi/blobs ] && ls /opt/wrolpi/wrolpi/blobs/* >/dev/null 2>&1; then
cp /opt/wrolpi/wrolpi/blobs/* /opt/wrolpi-blobs/
fi
chown -R wrolpi:wrolpi /home/wrolpi /opt/wrolpi*
# Protect blob files from accidental modification or deletion.
# These files are critical for initializing WROLPi.
if [ -d /opt/wrolpi-blobs ] && ls /opt/wrolpi-blobs/* >/dev/null 2>&1; then
chown -R root:root /opt/wrolpi-blobs
chmod 444 /opt/wrolpi-blobs/*
chattr +i /opt/wrolpi-blobs/*
fi
# Copy MOTD once the repair has been successful.
cp /opt/wrolpi/etc/raspberrypios/motd/30-wrolpi.motd /etc/update-motd.d/30-wrolpi
chmod +x /etc/update-motd.d/*
# Copy desktop shortcuts to existing users (new users get them from /etc/skel).
if [[ ${rpi} == true ]]; then
for user_home in /home/wrolpi /home/pi; do
if [ -d "${user_home}/Desktop" ]; then
cp /opt/wrolpi/etc/raspberrypios/*.desktop "${user_home}/Desktop/"
chown -R "$(basename ${user_home})":"$(basename ${user_home})" "${user_home}/Desktop/"
fi
done
# Update skeleton for future users.
mkdir -p /etc/skel/Desktop
cp /opt/wrolpi/etc/raspberrypios/*.desktop /etc/skel/Desktop/
fi
systemctl restart wrolpi-help
systemctl start wrolpi.target
set +x
echo "Repair has completed. You may run /opt/wrolpi/help.sh to check system status."