Commit graph

32 commits

Author SHA1 Message Date
lrnselfreliance
ad56134851
install.sh installs on Debian 13 (#830)
install.sh only accepted Debian 12, which cannot run this version, and
rejected Debian 13.  It now requires Debian 13 or newer on both PCs and
Raspberry Pis.  The install scripts point to the drive wizard instead of
manual mounting.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:51:40 -06:00
Roland
420e4957c9 help.sh: move DB integrity scan out of startup diagnostics
PRAGMA quick_check reads every page of the database, so on a large DB or
a slow (e.g. USB spinning) disk it can run for many minutes, making
help.sh (run by the wrolpi-help-script service) appear to hang.

Drop the quick_check from help.sh's fast startup diagnostics and add a
standalone scripts/check_database_integrity.sh for on-demand checks.
The new script runs sqlite3 as the wrolpi user (WAL -shm access),
reports elapsed time, and supports --full for an exhaustive
integrity_check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 19:47:35 -06:00
Roland
ddb8b4b590
help.sh: add Kiwix service, port 9085, and zim-corruption checks
The Kiwix section only checked library.xml and the Caddy HTTPS proxy on 8085,
which made a common failure hard to diagnose: kiwix-serve down while the
wrolpi-kiwix unit still reads "running" (its ExecStart is a bash wrapper), and
corrupt/incomplete zims silently missing from the library.

- Check the wrolpi-kiwix systemd unit exists and the service is up (parity
  with the api/app/controller sections).
- Check kiwix-serve is actually listening on 9085 and responds over plain
  HTTP there, isolating a kiwix failure from a Caddy proxy failure on 8085.
- Compare *.zim files on disk to <book> entries loaded in library.xml; a
  shortfall flags corrupt or incompletely-downloaded zims.

Verified against a healthy Debian 13 RPi (10.0.0.9).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 17:42:14 -06:00
Roland
e01795ad7e
Debian 13 (trixie) rebase: docker, debian-live-config, and pi-gen builds.
All three build targets move from Debian 12/bookworm to Debian 13/trixie
for fresh installs; existing Debian 12 systems remain supported in place.

Docker: api/help on python:3.13-trixie, controller on 3.13-slim-trixie,
app on node:20-trixie (matches trixie's nodejs 20), samba/zim on
debian:13.  Full backend suite passes on Python 3.13 in the api container.

debian-live-config: --distribution trixie; postgres teardown for the
SQLite cutover (pg_dropcluster hook block removed, wrolpi-bootstrap.sh no
longer creates a cluster or calls the deleted initialize_api_db.sh — the
API creates wrolpi.db itself on first start).  README documents the
SQLite layout and drops Etcher recommendations.

pi-gen: upstream has no trixie branch — the arm64 branch now defaults to
RELEASE=trixie; KEYBOARD_KEYMAP=en_US is rejected by trixie's
console-setup (ckbcomp validates), use 'us'; stop stamping a Postgres
.pgpass into /etc/skel.

Package lists (both targets): postgresql-15/postgresql-doc/libpq-dev
deleted; libtiff5-dev renamed to libtiff-dev; software-properties-common
and cpufrequtils removed from trixie (Controller CPU throttle falls back
to sysfs, dead cpufreq-set sudoers rule removed); locales and gnupg added
(no longer arrive transitively under --apt-recommends false); sqlite3 CLI
added for reset_tags.sh.

help.sh: replace the Postgres service/port/socket/cluster/psql checks
(all false FAILEDs after the SQLite cutover) with SQLite checks:
wrolpi.db exists, PRAGMA quick_check, file_group table + row count, run
as the wrolpi user so root never creates the -shm sidecar.

Both build scripts take a WROLPI_BUILD_DIR override so development builds
can run beside the release pipeline's build trees; the sudo re-exec
passes it through explicitly (a plain sudo would drop it).

test_common.py: the truncate tests hard-coded sys.getsizeof results from
the 3.11 object layout (str/list shrank in 3.12); assert the contract
(fits budget, prefix, monotonic in budget) so they pass on 3.11 and 3.13.

Verified: trixie ISO (3.7G) and pi-gen desktop image (4.3G) both built
from scratch and passed their sanity checks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 15:43:08 -06:00
Roland
715035e5dc
help.sh: port 80 is the Controller's, not Caddy's; fix IPv4 port regex.
The Caddyfile only binds 443/8084/8085/8086 and reverse-proxies to the
Controller on port 80, which the Controller health check already covers.
The IPv4 alternative in the port grep was missing its colon, so only the
IPv6 ":::" listeners ever matched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 16:00:53 -06:00
Roland
0e67cd6c74 help.sh: only print OK/FAILED/Note lines; robust hostname SAN check.
Connect to Postgres directly instead of via `sudo -i -u wrolpi` so
sudo/PAM stop spamming the journal when help.sh runs as a systemd
service.  Drop the raw pg_lsclusters table and the per-file PMTiles
listing.  Use `openssl x509 -checkhost` for the hostname SAN check so a
short hostname (e.g. "pi") cannot substring-match another SAN and
wrapped SAN lines cannot cause a false failure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 16:00:43 -06:00
Roland
d6b019c24e Certificates: help.sh validates the CA and leaf certs; fix renew exec bit.
help.sh now detects invalid Caddy certificates: expired or unparseable
CA/leaf certs, a leaf expiring within 30 days, a leaf that does not match
its private key or the current hostname, and Caddy serving a stale cert
after regeneration.  It also checks that the renewal script is executable
and its timer is enabled.

scripts/renew_certificates.sh was committed without the executable bit,
so wrolpi-cert-renew.service failed with "Permission denied".  Set the
bit and make repair.sh restore executable bits on all scripts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 16:00:43 -06:00
Roland
3180a29e82
help.sh: real Postgres cluster diagnostics.
The umbrella postgresql.service reports active with every cluster down,
which made two cluster failures look green.  Show pg_lsclusters, check
the first cluster is online, verify the data directory is under
/var/lib/postgresql, and reveal a mount over /var/lib/postgresql (the
Portable bind layout — wrong on an installed system).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 20:30:30 -06:00
Roland
e0324ce0fc Migrating Controller from port 8087 to 80 so that it is more discoverable. Reformatting Controller dashboard to make header at the top. Reformatting navigation bar to handle the new links which prevented Admin from being visible in some screens. 2026-04-09 15:01:05 -06:00
Roland
b8a532c8d9
Key changes:
- Add MapLibre GL JS viewer with light/dark themes, terrain layer,
  and embedded static map page for coordinate sharing
- Add map pin support with persistent map_pins.yaml config
- Add map region subscriptions and downloader with progress tracking,
  sourcing pre-extracted PMTiles from a cold CDN
- Add map export to PNG and GPS coordinate copy via context menu
- Add MapViewer React component alongside reworked Map page
- Add map catalog and region-building script (build_map_regions.py)
- Remove OSM import/render scripts, leaflet files, Apache configs,
  renderd service, and osm-map-dumper Docker container
- Update Caddyfile to serve PMTiles and map static assets
- Update build/install/repair scripts for Protomaps dependencies
- Drop map_file table, add/drop map_subscription table (alembic)
- Adding button to skip primary drive mounting during onboarding.
2026-04-04 13:48:19 -06:00
Roland
fbb71e012d Replace nginx with Caddy for TLS termination and reverse proxying.
- Replace nginx with Caddy as the web server for both Docker and native
  (RPi/Debian) deployments. Caddy handles HTTPS termination, reverse
  proxying to all services (API, React app, Controller, Kiwix, Help, Map),
  and serves the media/download file browsers.
- Generate a Root CA and signed leaf certificates for HTTPS. The Root CA
  uses Name Constraints to restrict signing to local addresses only
  (.local, localhost, and RFC 1918 private IP ranges), mitigating MITM
  risk if the CA key is compromised.
- Add an HTTP landing page that guides users through one-time CA
  certificate installation on iOS, Android, macOS, Windows, Linux, and
  Firefox, with auto-redirect when the cert is already trusted.
- Add trailing-slash redirects for /controller, /docs, /media, and
  /download so bare paths don't fall through to the React catch-all.
- Add a Root CA download link on the Settings page under Configs.
- Update help.sh diagnostics for IPv6 listener detection and HTTP/2
  status codes.
- Remove all nginx config files and the https_proxy Docker container.
- Bump wrolpi-help submodule with certificate documentation.
2026-03-07 18:48:44 -07:00
Roland
b9158b60fe Creating Controller service which will control WROLPi services, admin, SMART, disks, and status. nginx will fallback to Controller UI when React UI is unavailable (such as during upgrades). 2025-12-15 15:48:40 -07:00
Roland
00c86e1a1a
Searching wrolpi user's home directory for browser profiles. 2025-05-02 17:24:51 -06:00
Roland
16e359b339
Recommending refreshing files when database has no files. 2025-01-11 11:06:43 -07:00
Roland
95a524ba33
Adding under-voltage/over-current detection to help script. 2025-01-10 11:31:41 -07:00
Roland
8e4d173791
Converting map blob to use "custom" postgres dump format so restore can use multiple threads. Renaming map db blob from gis-map.dump.gz to map-db-gis.dump. 2025-01-02 12:04:03 -07:00
Roland
7324e83626
Creating script which migrates map database to new database for custom niceness. 2025-01-01 16:41:21 -07:00
Roland
7d6b7114a7
Adding TRACE level for logging beyond debug. Reworking async_client for tests, not all tests need the App to handle signals, so any test that needs signals can use the new await_switches fixture which primes the app. 2024-12-11 11:52:33 -07:00
Roland
af8bd7ba34
Cleaning up some extra data from video info json. 2024-10-16 18:55:59 -06:00
Roland
53d2402d32
Only files that have been tagged will be returned when new 'Any' tag button is used to filter file search results. 2024-10-16 18:55:56 -06:00
Roland
00c4e6ece0
nginx needs to serve downloads from the media directory. 2024-10-02 17:10:05 -06:00
Roland
04dbca7bb7
Checking for ebook files in help. 2024-10-01 11:29:57 -06:00
Roland
38de2f9b40
Moving HTTP kiwix/help services to 9085/9086 ports. Nginx will serve HTTPS proxy for kiwix/help services at the old ports. Modifying apache2 to use HTTPS to serve map.
Using new https_proxy docker container to serve HTTPS for zim, map and help.
2024-06-12 21:38:51 -06:00
Roland
63256d01c1
Checking that python venv scripts can be run in help.sh. 2024-06-12 16:38:46 -06:00
Roland
ab82213006
Checking if media directory is a mounted drive in help script. 2024-03-16 19:57:13 -06:00
Roland
9dd7f48d45
Adding Help service to help script. 2023-11-23 17:02:03 -07:00
Roland
b020fbfeee
Requiring sudo for help script, it will change to other users as needed. 2023-10-09 13:25:13 -06:00
Roland
51453ad6cc
Using wrolpi user to check for database in help script. 2023-10-09 13:16:07 -06:00
Roland
6e0f9a999c
Combining install scripts into a new repair script. Removing create_db script in favor of repair script. 2023-10-03 12:15:29 -06:00
Roland
1fff928d4e
Installing Kiwix packages and service in Raspberry Pi and Debian installer scripts. 2023-07-11 16:52:18 -06:00
Roland
7f3606dece
Testing that wrolpi config file exists (and is served by nginx) in help script. 2023-05-09 14:24:26 -06:00
Roland
07493ae3f3
Creating help script which displays a diagnostic summary. 2023-04-30 17:50:09 -06:00