wrolpi/docker-compose.yml
Roland d9303a0ea8
Postgres -> SQLite hard cutover: the database moves into the media directory.
The database now lives at <media>/config/wrolpi.db so a library (files,
configs, AND database) travels with the drive between WROLPi systems.

Core:
- wrolpi/db.py: lazy SQLite engine factory (WAL, busy_timeout=30s,
  foreign_keys=ON); get_db_curs yields a plain sqlite3.Cursor; native
  sqlite3 paramstyle everywhere (no compatibility shims); global sqlite3
  adapters keep raw-SQL datetime formats identical to the ORM's.
- wrolpi/fts.py: FTS5 external-content search replacing tsvector/GIN.
  Injection-proof websearch->FTS5 translator, bm25(10,4,2,1) reproducing
  A/B/C/D weighting, fast(abc)/deep column filtering, snippet headlines.
  Stemming contract pinned by dedicated tests (watch/watches/watched/
  watching are interchangeable).
- wrolpi/schema_ddl.py: SQLite triggers shared verbatim by the alembic
  baseline and test DBs; fixes the channel video_count DELETE/UPDATE
  bugs and a latent total_size bug (it used the video id as channel id).
- wrolpi/db_bootstrap.py: startup auto-create/migrate; refuses databases
  written by a newer WROLPi (db_version_mismatch); SQLite 3.40 floor;
  network-filesystem refusal; never creates/migrates/trusts a database
  beneath an unmounted production media directory (shadow-DB guard).
- Alembic: 85 Postgres-era migrations squashed into one baseline;
  revision ids are dates (2026_07_10_1350) so filenames self-sort and
  alembic_version says when a schema was created; render_as_batch=True.
- All raw Postgres SQL rewritten (ANY->json_each IN, array_agg @> ->
  HAVING COUNT(DISTINCT), unnest->executemany/UPDATE..FROM,
  generate_series histograms->strftime GROUP BY, DISTINCT ON->
  row_number(), writable CTEs->two statements, jsonb_*->json_*, explicit
  NULLS FIRST/LAST, LIMIT-before-OFFSET).  Simple statements converted
  to SQLAlchemy instead (flag upsert, download retry/summary/tag checks,
  channel-by-tag estimate, Directory delete).
- Config imports count skipped entries; background dumps are blocked
  after a lossy import so partial imports can never overwrite YAML
  configs (they are the sole source of truth for rebuilds now).
- Search tolerates explicit JSON nulls for limit/offset (Postgres
  treated OFFSET NULL as 0; SQLite errors).
- file_group.id/doc_section.id render as INTEGER PRIMARY KEY on SQLite
  (BIGINT is not a rowid alias; FTS5 content_rowid requires it).

Ops teardown: docker-compose db service, CI postgres sidecars,
initialize_api_db.sh, controller postgresql service, psycopg2-binary all
removed.  upgrade.sh stops/disables postgresql (data is NOT deleted;
`systemctl enable --now postgresql` restores it); systemd units no
longer Want postgresql.service.

Existing-user upgrade: first boot creates+migrates the DB, configs
import (tags recreate their FileGroups from disk), a refresh rebuilds
the library index.  QA'd on the 10.0.0.8 RPi: rebuild matched the
Postgres baseline exactly; survived an uncontrolled hard reboot with
clean integrity checks.

Accepted behavior changes: FTS5 has no stopwords; bm25 length
normalization shifts some rank orders; fast-path d_headline is
un-highlighted; all-negative queries match nothing.

Deferred: Debian/trixie package lists (incl. sqlite3 CLI), Cypress
re-baseline, VACUUM INTO snapshots, exFAT/NTFS warning, auto-kicked
rebuild refresh with progress UX.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 14:09:37 -06:00

155 lines
4.6 KiB
YAML

# Development containers, these should only be used if you are developing.
# The database is SQLite inside the media directory (./test/config/wrolpi.db); no db service.
version: '3'
services:
controller: # WROLPi Controller Service
build:
context: .
dockerfile: docker/controller/Dockerfile
restart: unless-stopped
ports:
- "8080:80"
volumes:
# Controller code (for development hot-reload)
- ./controller:/app/controller
# Docker socket for container management
- /var/run/docker.sock:/var/run/docker.sock
# Host system info (read-only)
- /proc:/host/proc:ro
- /sys:/host/sys:ro
# Media directory (needed for Samba share path validation)
- ${MEDIA_DIRECTORY:-./test}:/media/wrolpi
environment:
- DOCKERIZED=true
- MEDIA_DIRECTORY=/media/wrolpi
# Override CMD to enable auto-reload for development
command: ["python", "-m", "controller.serve"]
healthcheck:
test: [ 'CMD-SHELL', 'curl -f http://127.0.0.1:80/api/health || exit 1' ]
interval: 30s
timeout: 5s
start_period: 10s
retries: 3
api:
depends_on:
controller:
condition: service_started
build:
context: .
dockerfile: docker/api/Dockerfile
environment:
# Sanic 25.x wants 'spawn' but our code uses 'fork' for multiprocessing.
# This tells Sanic to accept the existing start method.
- SANIC_START_METHOD_SET=1
volumes:
- './main.py:/opt/wrolpi/main.py'
- './wrolpi:/opt/wrolpi/wrolpi'
- './modules:/opt/wrolpi/modules'
- './scripts:/opt/wrolpi/scripts'
- './test:/opt/wrolpi/test'
- './test:/media/wrolpi'
- './alembic:/opt/wrolpi/alembic'
- './alembic.ini:/opt/wrolpi/alembic.ini'
ports:
- ${REACT_APP_API-0.0.0.0:8081}:8081
user: '${UID-1000}:${GID-1000}'
healthcheck:
test: [ 'CMD-SHELL', 'curl http://127.0.0.1:8081/api/echo' ]
interval: 2m
archive: # Python/Sanic wrapper around Singlefile and Readability.
build:
context: .
dockerfile: docker/archive/Dockerfile
volumes:
- './docker/archive/main.py:/app/main.py'
ports:
- '8083:8080'
healthcheck:
test: [ 'CMD-SHELL', 'curl http://127.0.0.1:8080/' ]
interval: 2m
app: # React App
depends_on:
- api
build:
context: .
dockerfile: docker/app/Dockerfile
volumes:
- './app/public:/app/public'
- './app/src:/app/src'
- './app/package.json:/app/package.json'
- '.env:/app/.env'
command: 'npm run start'
stdin_open: true
healthcheck:
test: [ 'CMD-SHELL', 'curl http://127.0.0.1:3000' ]
interval: 1m
user: '${UID-1000}:${GID-1000}'
environment:
# Force React to use Caddy port to connect to it's websocket.
- WDS_SOCKET_PORT=0
web: # Caddy reverse proxy - handles HTTPS for all services.
depends_on:
- app
- zim
- help
build:
context: .
dockerfile: docker/web/Dockerfile
ports:
- ${WEB_HOST-0.0.0.0}:${WEB_HTTPS_PORT-8443}:443
- '8084:8084'
- '8085:8085'
- '8086:8086'
volumes:
- './docker/web/Caddyfile:/etc/caddy/Caddyfile'
- './icon.ico:/etc/caddy/favicon.ico'
- './icon.png:/etc/caddy/icon.png'
- './test:/opt/media'
- './modules/map/static:/opt/wrolpi/modules/map/static:ro'
- './blobs:/opt/wrolpi-blobs:ro'
environment:
- MEDIA_DIRECTORY=/opt/media
# Set EXTRA_SANS to include the host's LAN IP in the TLS certificate.
# Example: EXTRA_SANS=IP:10.0.0.5
- EXTRA_SANS=${EXTRA_SANS:-}
healthcheck:
test: [ 'CMD-SHELL', 'wget --spider --no-check-certificate https://127.0.0.1:443 || exit 1' ]
interval: 1m
zim: # kiwix-serve
build:
context: .
dockerfile: docker/zim/Dockerfile
volumes:
- './test:/media/wrolpi'
- './docker/zim/entrypoint.sh:/entrypoint.sh'
healthcheck:
test: [ 'CMD-SHELL', 'curl http://127.0.0.1:80' ]
interval: 1m
samba: # Samba file sharing - started/stopped by the controller based on configured shares.
build:
context: .
dockerfile: docker/samba/Dockerfile
restart: "no"
depends_on:
controller:
condition: service_healthy
ports:
- "445:445"
volumes:
- ${MEDIA_DIRECTORY:-./test}:/media/wrolpi
help: # The WROLPi Help documentation.
build:
context: .
dockerfile: docker/help/Dockerfile
volumes:
- './docker/help/wrolpi-help:/opt/wrolpi-help'
healthcheck:
test: [ 'CMD-SHELL', 'curl http://127.0.0.1:8086' ]
interval: 1m