diff --git a/docs/controller/services.md b/docs/controller/services.md index e032a1b..6a2220f 100644 --- a/docs/controller/services.md +++ b/docs/controller/services.md @@ -10,13 +10,11 @@ The following services are managed by the Controller: |-------------------|--------|----------------------------------| | wrolpi-controller | 80 | System management (this service) | | wrolpi-api | 8081 | Backend API server | -| wrolpi-app | 5000 | React frontend application | -| wrolpi-help | 8084 | Documentation server | +| wrolpi-app | 3000 | React frontend application | +| wrolpi-help | 8086 | Documentation server | | wrolpi-kiwix | 8085 | Offline wiki server (Kiwix) | -| renderd | - | Map tile rendering | -| apache2 | - | Map tile server | -| postgresql | - | Database server | | caddy | 80/443 | Web server and reverse proxy | +| smbd / nmbd | 445 | Samba file sharing | Development services (shown only when running): @@ -36,7 +34,7 @@ service is running. ### Stopping a Service Click the **Stop** button to stop a running service. Use caution when stopping critical services -like Caddy or postgresql. +like Caddy. ![Service stop button](services-stop-button.png) @@ -103,10 +101,11 @@ Caddy handles HTTPS termination and reverse proxying for all services in a singl | `api` | 8081 | Backend API | | `app` | - | React frontend | | `controller` | 8080 | Controller service (HTTP) | -| `db` | 5432 | PostgreSQL database | | `help` | 8086 | Help documentation (via Caddy) | -| `map` | 8084 | Map tile server (via Caddy) | | `zim` | 8085 | Kiwix wiki server (via Caddy) | +| `samba` | 445 | Samba file sharing | + +The API database is SQLite at `/media/wrolpi/config/wrolpi.db` (no database container). *The following screenshot is from a healthy WROLPi running in Docker containers* diff --git a/docs/system/configs.md b/docs/system/configs.md index 583cd12..c224bf3 100644 --- a/docs/system/configs.md +++ b/docs/system/configs.md @@ -30,6 +30,7 @@ All configs are stored in the `config` directory of your media directory: `/medi | `fstab.yaml` | The WROLPi-managed mount table for your drives. | | `backup/` | Automatic dated backups of your configs (see below). | | `ssl/` | Your generated [HTTPS certificates](certificates.md). | +| `wrolpi.db` | The [SQLite database](databases.md) (search index; rebuilt from files + configs). | ## The Source-of-Truth Model diff --git a/docs/system/databases.md b/docs/system/databases.md index 575c826..0d0d33b 100644 --- a/docs/system/databases.md +++ b/docs/system/databases.md @@ -1,16 +1,20 @@ -# Databases +# Database -WROLPi has two major Postgresql databases, the main (API) database, and the map database. +WROLPi uses a single [SQLite](https://sqlite.org/) database stored in the media directory. +There is no separate database service — the API opens the file directly. -## Main Database +## Location -This contains the tables for the API. +`/media/wrolpi/config/wrolpi.db` -### Service +The database lives next to the [YAML config files](configs.md) so a user's entire library +(files + configs + database) travels with the media drive. WAL sidecars (`wrolpi.db-wal`, +`wrolpi.db-shm`) may appear alongside it while the API is running. -`postgresql@15-main.service` +The database is created automatically on first API start. To wipe and rebuild it (you will need +to refresh your files afterward), see [Reset API Database](getting-help.md#reset-api-database). -### Tables +## Tables | Name | Purpose | Relations | `/opt/wrolpi` File | |------------------|---------------------------------------------------------|------------------------------|--------------------------------| @@ -26,7 +30,7 @@ This contains the tables for the API. | file_group | Each record is a group of files that share a stem. | | `wrolpi/files/models.py` | | inventory | A collection of items. | | `modules/inventory/models.py` | | item | An item in an inventory. | inventory_id | `modules/inventory/models.py` | -| map_file | Data about PBF files (if they have been imported, etc.) | | `modules/map/models.py` | +| map_file | Data about map files (if they have been imported, etc.) | | `modules/map/models.py` | | tag | Data about a Tag (name, color, etc.) | | `wrolpi/tags.py` | | tag_file | A FileGroup that has been tagged. | tag_id, file_group_id | `wrolpi/tags.py` | | tag_zim | An entry (article) that was tagged in a Zim file. | tag_id, zim_id, zim_entry | `modules/zim/models.py` | @@ -35,10 +39,9 @@ This contains the tables for the API. | zim | A Zim file | | `modules/zim/models.py` | | zim_subscription | A recurring download of a Zim file. | | `modules/zim/models.py` | -## Map Database +Full-text search uses SQLite FTS5 virtual tables (created alongside the main schema). -This contains tables for the OpenStreetMap. +## Maps -### Service - -`postgresql@15-map.service` +Map data is **not** stored in this database. Maps use PMTiles files under `/media/wrolpi/map/`, +served by Caddy. See the [Map module](../modules/map/index.md). diff --git a/docs/system/getting-help.md b/docs/system/getting-help.md index c6c2093..93173e5 100644 --- a/docs/system/getting-help.md +++ b/docs/system/getting-help.md @@ -34,20 +34,24 @@ Example output: ```text WROLPi version: 0.13.1-beta -OK: Running on Raspberry Pi +OK: Running on Raspberry Pi 5 OK: Found wrolpi user OK: The WROLPi directory exists OK: The WROLPi blobs directory exists OK: The WROLPi main script exists OK: The WROLPi config file exists -OK: Postgres found -OK: Port 5432 is occupied -OK: Postgres is using file socket -OK: Found wrolpi database +OK: Found wrolpi database at /media/wrolpi/config/wrolpi.db +OK: WROLPi database passes quick_check OK: WROLPi database is initialized OK: WROLPi database has files +OK: Controller Python virtual environment exists +OK: WROLPi Controller systemd exists +OK: WROLPi Controller service is up +OK: WROLPi Controller health check passed +OK: WROL Mode is inactive + OK: WROLPi Python virtual environment exists OK: WROLPi main can be run OK: Sanic can be run @@ -59,16 +63,16 @@ OK: WROLPi app directory exists OK: WROLPi app exists OK: WROLPi app systemd exists OK: WROLPi app service is up -OK: WROLPi app responded with interface -OK: Port 80 is occupied -OK: Caddy systemd exists +OK: WROLPi app responded with UI -OK: Found map database -OK: Map database is initialized -OK: renderd systemd exists -OK: Map app responded -OK: Map initialization blob exists -OK: Leaflet.js exists +OK: Caddy is installed +OK: Caddy systemd exists +OK: Caddy service is running +OK: Caddy HTTPS proxy responded on :443 + +OK: Map viewer responded +OK: Map static assets exist +OK: Found 3 PMTiles map files OK: The kiwix library file exists OK: Kiwix app responded @@ -76,8 +80,8 @@ OK: Kiwix app responded OK: The media directory exists OK: Can modify media directory OK: Media directory is a mounted drive -OK: Media directory files are served by Caddy -OK: Config can be fetched from Caddy +OK: Media directory files are served by caddy +OK: Config can be fetched from caddy OK: Media directory is owned by wrolpi user OK: Singlefile can be run @@ -91,7 +95,7 @@ OK: Help mkdocs can be run OK: Help service is running OK: Can ping 1.1.1.1 -OK: Can ping wrolpi.org +OK: Can ping one.one.one.one ``` The output above will help an experienced developer narrow down the cause of an issue. @@ -100,8 +104,8 @@ The output above will help an experienced developer narrow down the cause of an ### Reset API Database -The `wrolpi` database contains the tables and indexes which allow you to search or even browse your WROLPi. A script is -provided which will delete and re-create this database. **Warning you will need to refresh your files!** Run it if the -repair script above fails: +The SQLite database at `/media/wrolpi/config/wrolpi.db` contains the tables and indexes which allow you to search or +browse your WROLPi. A script is provided which will delete this database (the API recreates it on next start). +**Warning: you will need to refresh your files!** Run it if the repair script above fails: `/opt/wrolpi/scripts/reset_api_db.sh` diff --git a/docs/system/index.md b/docs/system/index.md index 6314bcb..a4d206d 100644 --- a/docs/system/index.md +++ b/docs/system/index.md @@ -33,9 +33,11 @@ HTTPS with [self-signed certificates](certificates.md). ### DB -| Software Project | Purpose | -|-------------------------------------------|----------------------------------------------------------------------| -| [Postgresql](https://www.postgresql.org/) | Facilitates searching of files, scheduling downloads, and much more. | +| Software Project | Purpose | +|-------------------------------|----------------------------------------------------------------------| +| [SQLite](https://sqlite.org/) | Facilitates searching of files, scheduling downloads, and much more. | + +The database file lives at `/media/wrolpi/config/wrolpi.db`. See [Databases](databases.md). ### React App @@ -47,11 +49,11 @@ HTTPS with [self-signed certificates](certificates.md). ### Map -| Software Project | Purpose | -|-------------------------------------------------|-------------------------------------------| -| [OpenStreetMap](https://www.openstreetmap.org/) | Provides the Map data. | -| [osmium](https://osmcode.org/osmium-tool/) | Used to merge map PBF files. | -| [osm2pgsql](https://osm2pgsql.org/) | Used to import PBF files into Postgresql. | +| Software Project | Purpose | +|-------------------------------------------------|--------------------------------------------------| +| [OpenStreetMap](https://www.openstreetmap.org/) | Provides the Map data. | +| [PMTiles](https://protomaps.com/) | Offline map tiles stored as files under `map/`. | +| [MapLibre GL JS](https://maplibre.org/) | Map viewer in the browser. | ### Zim diff --git a/docs/system/special-directories.md b/docs/system/special-directories.md index 0c0a425..8321e41 100644 --- a/docs/system/special-directories.md +++ b/docs/system/special-directories.md @@ -22,8 +22,9 @@ The normal media directory is `/media/wrolpi` `/media/wrolpi/config/` -This directory contains the configuration files of the WROLPi. These files are considered the "source of truth" and -their contents will alter the behavior of the WROLPi. See [How Configs Work](configs.md). +This directory contains the configuration files of the WROLPi and the SQLite database (`wrolpi.db`). Config files +are considered the "source of truth" and their contents will alter the behavior of the WROLPi. +See [How Configs Work](configs.md) and [Databases](databases.md). ## Tags Directory @@ -38,8 +39,8 @@ files. See [Tags](tags.md). `/opt/wrolpi-blobs/` -This directory contains the files necessary to repair a WROLPi. For example, the `map-db-gis.dump` file contains a -Postgresql dump of the map database. **It is best to leave this directory as-is.** +This directory contains files necessary to repair a WROLPi (for example map fonts and other offline assets). +**It is best to leave this directory as-is.** ## WROLPi Help diff --git a/docs/system/systemd.md b/docs/system/systemd.md index 43a4783..75b72b5 100644 --- a/docs/system/systemd.md +++ b/docs/system/systemd.md @@ -12,11 +12,14 @@ See [Service Management](../controller/services.md) for details. For advanced users or scripting, services can be controlled directly with these commands: -| Service | Status | Logs | Restart | Stop | Start | -|-----------|-----------------------------------------------|------------------------------------------------|------------------------------------------------|---------------------------------------------|----------------------------------------------| -| API | `systemctl status wrolpi-api` | `journalctl -u wrolpi-api` | `systemctl restart wrolpi-api` | `systemctl stop wrolpi-api` | `systemctl start wrolpi-api` | -| React App | `systemctl status wrolpi-app` | `journalctl -u wrolpi-app` | `systemctl restart wrolpi-app` | `systemctl stop wrolpi-app` | `systemctl start wrolpi-app` | -| Map | `systemctl status renderd` | `journalctl -u renderd` | `systemctl restart renderd` | `systemctl stop renderd` | `systemctl start renderd` | -| Zim | `systemctl status wrolpi-kiwix` | `journalctl -u wrolpi-kiwix` | `systemctl restart wrolpi-kiwix` | `systemctl stop wrolpi-kiwix` | `systemctl start wrolpi-kiwix` | -| API DB | `systemctl status postgresql@15-main.service` | `journalctl status postgresql@15-main.service` | `systemctl restart postgresql@15-main.service` | `systemctl stop postgresql@15-main.service` | `systemctl start postgresql@15-main.service` | -| Map DB | `systemctl status postgresql@15-map.service` | `journalctl status postgresql@15-map.service` | `systemctl restart postgresql@15-map.service` | `systemctl stop postgresql@15-map.service` | `systemctl start postgresql@15-map.service` | +| Service | Status | Logs | Restart | Stop | Start | +|------------|---------------------------------|------------------------------|----------------------------------|-------------------------------|--------------------------------| +| Controller | `systemctl status wrolpi-controller` | `journalctl -u wrolpi-controller` | `systemctl restart wrolpi-controller` | `systemctl stop wrolpi-controller` | `systemctl start wrolpi-controller` | +| API | `systemctl status wrolpi-api` | `journalctl -u wrolpi-api` | `systemctl restart wrolpi-api` | `systemctl stop wrolpi-api` | `systemctl start wrolpi-api` | +| React App | `systemctl status wrolpi-app` | `journalctl -u wrolpi-app` | `systemctl restart wrolpi-app` | `systemctl stop wrolpi-app` | `systemctl start wrolpi-app` | +| Zim | `systemctl status wrolpi-kiwix` | `journalctl -u wrolpi-kiwix` | `systemctl restart wrolpi-kiwix` | `systemctl stop wrolpi-kiwix` | `systemctl start wrolpi-kiwix` | +| Caddy | `systemctl status caddy` | `journalctl -u caddy` | `systemctl restart caddy` | `systemctl stop caddy` | `systemctl start caddy` | +| Help | `systemctl status wrolpi-help` | `journalctl -u wrolpi-help` | `systemctl restart wrolpi-help` | `systemctl stop wrolpi-help` | `systemctl start wrolpi-help` | + +The SQLite database is a file under `/media/wrolpi/config/` — there is no database systemd service. +Maps are served by Caddy from PMTiles files (no map rendering service). diff --git a/docs/system/variables.md b/docs/system/variables.md index 67909a7..43397a0 100644 --- a/docs/system/variables.md +++ b/docs/system/variables.md @@ -12,18 +12,6 @@ Download behavior can be modified by using these variables: |---------------------------------|-------|---------------| | `SIMULTANEOUS_DOWNLOAD_DOMAINS` | `Int` | 4 | -### Database - -Connecting to the database can be modified using these variables: - -| Name | Type | Default Value | -|---------------|----------|---------------| -| `DB_HOST` | `String` | '127.0.0.1' | -| `DB_PORT` | `Int` | 5432 | -| `DB_NAME` | `String` | 'wrolpi' | -| `DB_USER` | `String` | 'wrolpi' | -| `DB_PASSWORD` | `String` | 'wrolpi' | - ### Raspberry Pi Raspberry Pi behavior can be modified by using these variables: