wrolpi/etc/raspberrypios
Roland d9303a0ea8
Postgres -> SQLite hard cutover: the database moves into the media directory.
The database now lives at <media>/config/wrolpi.db so a library (files,
configs, AND database) travels with the drive between WROLPi systems.

Core:
- wrolpi/db.py: lazy SQLite engine factory (WAL, busy_timeout=30s,
  foreign_keys=ON); get_db_curs yields a plain sqlite3.Cursor; native
  sqlite3 paramstyle everywhere (no compatibility shims); global sqlite3
  adapters keep raw-SQL datetime formats identical to the ORM's.
- wrolpi/fts.py: FTS5 external-content search replacing tsvector/GIN.
  Injection-proof websearch->FTS5 translator, bm25(10,4,2,1) reproducing
  A/B/C/D weighting, fast(abc)/deep column filtering, snippet headlines.
  Stemming contract pinned by dedicated tests (watch/watches/watched/
  watching are interchangeable).
- wrolpi/schema_ddl.py: SQLite triggers shared verbatim by the alembic
  baseline and test DBs; fixes the channel video_count DELETE/UPDATE
  bugs and a latent total_size bug (it used the video id as channel id).
- wrolpi/db_bootstrap.py: startup auto-create/migrate; refuses databases
  written by a newer WROLPi (db_version_mismatch); SQLite 3.40 floor;
  network-filesystem refusal; never creates/migrates/trusts a database
  beneath an unmounted production media directory (shadow-DB guard).
- Alembic: 85 Postgres-era migrations squashed into one baseline;
  revision ids are dates (2026_07_10_1350) so filenames self-sort and
  alembic_version says when a schema was created; render_as_batch=True.
- All raw Postgres SQL rewritten (ANY->json_each IN, array_agg @> ->
  HAVING COUNT(DISTINCT), unnest->executemany/UPDATE..FROM,
  generate_series histograms->strftime GROUP BY, DISTINCT ON->
  row_number(), writable CTEs->two statements, jsonb_*->json_*, explicit
  NULLS FIRST/LAST, LIMIT-before-OFFSET).  Simple statements converted
  to SQLAlchemy instead (flag upsert, download retry/summary/tag checks,
  channel-by-tag estimate, Directory delete).
- Config imports count skipped entries; background dumps are blocked
  after a lossy import so partial imports can never overwrite YAML
  configs (they are the sole source of truth for rebuilds now).
- Search tolerates explicit JSON nulls for limit/offset (Postgres
  treated OFFSET NULL as 0; SQLite errors).
- file_group.id/doc_section.id render as INTEGER PRIMARY KEY on SQLite
  (BIGINT is not a rowid alias; FTS5 content_rowid requires it).

Ops teardown: docker-compose db service, CI postgres sidecars,
initialize_api_db.sh, controller postgresql service, psycopg2-binary all
removed.  upgrade.sh stops/disables postgresql (data is NOT deleted;
`systemctl enable --now postgresql` restores it); systemd units no
longer Want postgresql.service.

Existing-user upgrade: first boot creates+migrates the DB, configs
import (tags recreate their FileGroups from disk), a refresh rebuilds
the library index.  QA'd on the 10.0.0.8 RPi: rebuild matched the
Postgres baseline exactly; survived an uncontrolled hard reboot with
clean integrity checks.

Accepted behavior changes: FTS5 has no stopwords; bm25 length
normalization shifts some rank orders; fast-path d_headline is
un-highlighted; all-negative queries match nothing.

Deferred: Debian/trixie package lists (incl. sqlite3 CLI), Cypress
re-baseline, VACUUM INTO snapshots, exFAT/NTFS warning, auto-kicked
rebuild refresh with progress UX.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 14:09:37 -06:00
..
motd Updating MOTD to suggest ip addresses on login. 2025-08-18 11:33:44 -06:00
50x.html Migrating Controller from port 8087 to 80 so that it is more discoverable. Reformatting Controller dashboard to make header at the top. Reformatting navigation bar to handle the new links which prevented Admin from being visible in some screens. 2026-04-09 15:01:05 -06:00
90-wrolpi Key changes: 2026-04-04 13:48:19 -06:00
Caddyfile Migrating Controller from port 8087 to 80 so that it is more discoverable. Reformatting Controller dashboard to make header at the top. Reformatting navigation bar to handle the new links which prevented Admin from being visible in some screens. 2026-04-09 15:01:05 -06:00
Caddyfile.onboarding Migrating Controller from port 8087 to 80 so that it is more discoverable. Reformatting Controller dashboard to make header at the top. Reformatting navigation bar to handle the new links which prevented Admin from being visible in some screens. 2026-04-09 15:01:05 -06:00
controller-logo.png Creating WROLPi Controller desktop icon. 2026-02-14 17:05:57 -07:00
controller.desktop Migrating Controller from port 8087 to 80 so that it is more discoverable. Reformatting Controller dashboard to make header at the top. Reformatting navigation bar to handle the new links which prevented Admin from being visible in some screens. 2026-04-09 15:01:05 -06:00
gnome-disks.desktop Migrating Controller from port 8087 to 80 so that it is more discoverable. Reformatting Controller dashboard to make header at the top. Reformatting navigation bar to handle the new links which prevented Admin from being visible in some screens. 2026-04-09 15:01:05 -06:00
wrolpi-api-dev.service Postgres -> SQLite hard cutover: the database moves into the media directory. 2026-07-12 14:09:37 -06:00
wrolpi-api.service Postgres -> SQLite hard cutover: the database moves into the media directory. 2026-07-12 14:09:37 -06:00
wrolpi-app-dev.service Adding wrolpi-app-dev which runs frontend in dev mode. 2025-12-18 18:01:45 -07:00
wrolpi-app.service Adding wrolpi-app-dev which runs frontend in dev mode. 2025-12-18 18:01:45 -07:00
wrolpi-cert-renew.service Replace nginx with Caddy for TLS termination and reverse proxying. 2026-03-07 18:48:44 -07:00
wrolpi-cert-renew.timer Replace nginx with Caddy for TLS termination and reverse proxying. 2026-03-07 18:48:44 -07:00
wrolpi-controller.service Postgres -> SQLite hard cutover: the database moves into the media directory. 2026-07-12 14:09:37 -06:00
wrolpi-first-startup.service Creating script which runs on first startup only. This is used to copy /etc/skel files to first user's home directory. 2024-06-18 16:17:54 -06:00
wrolpi-fix-media-permissions.service Creating script that fixes media directory file and directory permissions, adding it to the Controller UI. 2026-03-05 17:58:21 -07:00
wrolpi-help-files.desktop Moving HTTP kiwix/help services to 9085/9086 ports. Nginx will serve HTTPS proxy for kiwix/help services at the old ports. Modifying apache2 to use HTTPS to serve map. 2024-06-12 21:38:51 -06:00
wrolpi-help-script.service Controller: add help.sh to the Maintenance Scripts. 2026-07-09 16:00:43 -06:00
wrolpi-help.desktop A directory containing files that have been tagged is maintained automatically by WROLPi. 2024-10-16 18:56:05 -06:00
wrolpi-help.service Modifying niceness of WROLPi services to facilitate API startup. 2025-03-03 19:28:35 -07:00
wrolpi-kiwix.service Modifying niceness of WROLPi services to facilitate API startup. 2025-03-03 19:28:35 -07:00
wrolpi-mounts.service Merging wrolpi-portable into debian-live-config; unifying Postgres on /media/wrolpi/config/postgresql. 2026-05-24 12:44:05 -06:00
wrolpi-repair.service Enhancing Controller to run scripts, starting with the repair and upgrade scripts for onboarding use. 2026-02-25 17:08:32 -07:00
wrolpi-upgrade.service upgrade.sh delegates to wrolpi-upgrade service so that logs are visible in controller, even if launched from terminal. 2026-02-03 16:42:06 -07:00
wrolpi.desktop Moving HTTP kiwix/help services to 9085/9086 ports. Nginx will serve HTTPS proxy for kiwix/help services at the old ports. Modifying apache2 to use HTTPS to serve map. 2024-06-12 21:38:51 -06:00
wrolpi.target Postgres -> SQLite hard cutover: the database moves into the media directory. 2026-07-12 14:09:37 -06:00