mirror of
https://github.com/lrnselfreliance/wrolpi.git
synced 2026-11-19 14:36:22 -05:00
Create your own off-grid library
https://wrolpi.org
* A secondary WROLPi copies its primary's library over HTTP A WROLPi with role "secondary" pulls from its primary: a manifest of every FileGroup's files (/api/sync/manifest), then each missing or changed file from the primary's Caddy /media/ (resumable, written to a hidden partial and renamed with the primary's mtime), a refresh, then the library configs, imported as the primary wrote them. wrolpi.yaml contributes only its library settings. Files deleted on the primary are kept. A secondary never saves the synced configs and never downloads. Its API answers 503 SYNCING while a sync runs, and every page warns that it is a copy. It syncs in a daily window or on "Sync now"; WROL Mode stops it. On a Pi 4 (no AES instructions) the client uses TLS 1.2 ChaCha20, which pulls ~110 MB/s from Caddy versus ~41 MB/s for rsync over ssh. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Sync has its own /admin tab; a sync's reads from the primary are bounded The sync settings and progress move from Settings (and the component gallery, which is for library components) to /admin/sync. A download never writes past the size in the primary's manifest, and configs, listings, and manifest pages are capped. A primary address may not carry a user name or password (it is logged and shown). While a sync runs, the API answers only the exact requests that show its progress or stop it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A sync keeps the secondary's replaced configs in today's config backup The first copy of the day is kept, like a save, so a config backup import undoes a sync's config changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Docker's Caddy logs every request `docker compose logs -f web` shows each request with its status, size and duration, e.g. a secondary's /media/ downloads during a sync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A sync reports its progress as it downloads, and refuses quietly The status updated only when a page of 500 FileGroups finished, so a page of large files looked stuck; it now updates every 2 seconds. Requests refused while syncing are answered by the middleware instead of raised, so the UI's polling no longer logs an error each time. The primary's version is stored without its trailing newline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A WROLPi can back up to a second drive; sync settings live in sync.yaml The Sync tab's Backup Drive copies the media directory to a second mounted drive with rsync during the sync window (hourly), or on "Back Up Now". Nothing on the backup drive is deleted. The Tags and playlists directories are left out (WROLPi rebuilds them from the configs), and the database is written as an SQLite snapshot rather than copied as files, so the drive can be mounted as the media directory and start. A backup is written only while the chosen drive (by UUID) is mounted where it was chosen, and is not the media or system drive; rsync is stopped if the drive disappears. Not in Docker, which cannot reach other drives. Role, primary, window and backup drive move from wrolpi.yaml to sync.yaml: they belong to this machine, and are never copied from a primary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * The Sync tab shows the configured backup drive getSettings copies each setting by name and did not list the backup drive, so the Sync tab showed "None" while one was configured. It also dropped `version`. A test now fails for any setting the API sends that getSettings drops. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A backup can go to a folder on its drive, and remembers its last success The Backup Drive panel takes a folder on the chosen drive. The drive is checked through the folder's mount point: the folder is written only when the drive with the saved UUID is mounted there, so a folder left on the system drive by an unplugged drive, or on another drive mounted inside the chosen one, is refused. A missing folder is created once the drive is checked. When the last backup succeeded is saved in sync.yaml, so it is known after a restart. "Stop Backup" is disabled while the database snapshot is written (it runs to the end), and a finished backup clears the request to stop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * The backup drive has its own window The sync window was shown in the Sync panel but also scheduled the backup drive. The Backup Drive panel now has its own window (sync.yaml backup_window_start/end), saved with the drive; the sync window is shown, and used, only for a secondary's sync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Sync and the backup drive can each exclude folders A secondary or a backup drive may be smaller than the primary's library. Each Sync tab panel has its own excluded folders in sync.yaml, chosen in a folder browser: a secondary browses its primary's folders (through Caddy), the backup drive browses this WROLPi's media directory. Excluded folders are never copied; what was copied before is kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * The folder browser does not offer lost+found Every ext4 drive has a root-only lost+found folder; a sync and a backup never copy it, so it is not offered for exclusion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A sync never writes into config or tags, whatever their case On a media drive that ignores case (exFAT, NTFS, macOS), a manifest path like Config/wrolpi.db is the real config folder. The protected and unlisted top-level folders are now compared case-insensitively in the manifest, in resolve_destination, in excluded-folder validation, and in both folder listings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A backup writes the database snapshot only when the database changed Every backup rewrote the whole database snapshot (2.5 GB on a busy WROLPi), even on the window's idle hourly runs. The snapshot is now skipped when the live database (its size and mtime, and those of a non-empty WAL) and the drive's copy are both unchanged since the last snapshot. The record is in memory, so the first backup after a restart writes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Remove the unused save_sync_config switch Nothing activated it; SyncConfig saves directly. The test server's log_message override is called by http.server, so it goes in the vulture baseline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|---|---|---|
| .github | ||
| .run | ||
| alembic | ||
| app | ||
| controller | ||
| debian-live-config | ||
| docker | ||
| etc | ||
| modules | ||
| npm-tools | ||
| pi-gen | ||
| release | ||
| scripts | ||
| test | ||
| wrolpi | ||
| .coveragerc | ||
| .dockerignore | ||
| .gitignore | ||
| .gitmodules | ||
| .gitpod.yml | ||
| .mcp.json.example | ||
| alembic.ini | ||
| conftest.py | ||
| demo_link.jpg | ||
| docker-compose.yml | ||
| help.sh | ||
| icon.ico | ||
| icon.png | ||
| INSTALL.md | ||
| install.sh | ||
| join_discord.png | ||
| LICENSE | ||
| main.py | ||
| pytest.ini | ||
| README.md | ||
| repair.sh | ||
| requirements.lock | ||
| requirements.txt | ||
| test_all.sh | ||
| UPGRADE.md | ||
| upgrade.sh | ||
WROLPi
Create your own off-grid library.
WROLPi is a self-contained collection of software to help you survive the world Without Rule of Law.
WROLPi is intended to be run on a Raspberry Pi with an optional external drive attached. It serves up it's own Wi-Fi network so that any user with a laptop/tablet/phone can connect and use the library created by the maintainer.
Table of Contents
Features
- Videos
- Web Archives
- Maps
- Wikipedia
- File search
- eBooks (EPUB / PDFs)
- Wi-Fi Hotspot
- Universal search
- Food Inventory management
- One-Time pad generator, encrypter/decrypter
- Synchronizer/duplicator
- Food storage calculator
Module features matrix
| Module | View | Search | Download | Project |
|---|---|---|---|---|
| Videos | yes | yes | yes | yt-dlp |
| Web Archives | yes | yes | yes | Singlefile |
| Wikipedia | yes | yes | yes | Kiwix |
| eBooks | yes | yes | planned | EPUB/PDF |
| Map | yes | yes | yes | Protomaps |
| Podcasts/Audio | yes | planned | planned |
Demo
Download
Images are available at wrolpi.org
Try WROLPi!
You can try out WROLPi by running the docker containers.
- Install docker-ce and docker-compose
- Copy the latest WROLPi code.
git clone https://github.com/lrnselfreliance/wrolpi.git
- Change directory into the code base.
cd wrolpi
- Initialize git submodules
git submodule update --init
- Build the containers.
docker-compose build --parallel
- Create volumes.
docker volume create --name=openstreetmap-datadocker volume create --name=openstreetmap-rendered-tiles
- Initialize the database (SQLite in the media directory).
docker-compose run --rm api db upgrade
- Start the docker containers.
docker-compose up
- Browse to WROLPi.
More Docker documentation is available in docker/README.md
Install WROLPi
Full installation instructions are in INSTALL.md. Pick the option that matches your hardware:
- WROLPi Portable (Live USB) — boot a complete WROLPi from a USB stick on any x86 PC or laptop. Nothing is installed to the computer; the first boot creates a persistence partition on the USB so your library, database, and configuration survive reboots.
- Debian 13 — install WROLPi onto an x86 PC or laptop.
- Raspberry Pi — install WROLPi onto a Raspberry Pi with an external drive.
- Docker — run the WROLPi containers on an existing Linux host.
Charter
Guiding Principals
- Storage is cheaper than power.
- Two is one, one is none.
- Run silently.
- Primary, secondary, tertiary.
- Secure as a bookshelf in your home.
Physical properties
- A WROLPi instance should be capable of running with a minimal of hardware:
- Raspberry Pi
- SD card
- External USB hard drive
- Power supply and cables
- Wi-Fi USER device such as a phone, tablet, or laptop.
- A WROLPi instance should consume a minimal amount of electricity during WROL event. It is expected power will be scarce when WROLPi is needed most.
User expectations
- A WROLPi instance should be run and maintained by a person (MAINTAINER) with a moderate amount of Linux and Raspberry Pi experience. It is expected that they should be able to do this using only the documentation on their WROLPi.
Software properties and capabilities
- A WROLPi instance should be able to "factory-reset" itself without any outside services.
- WROLPi should function completely without any internet services.
- A user should have easy access to their data if WROLPi fails:
- For example, a user can watch their videos by navigating a short and intuitive directory structure and opening the video in their preferred video player.
- WROLPi should be self-documented. The UI should contain a tutorial for USERS as well as the MAINTAINER.
- If the UI isn't functional, the code should be documented such that a user can restore functionality.
- WROLPi should favor pre-processing, rather than processing during a WROL event. Such as re-encoding a video, or processing captions. This is to ensure that when a user adds content during non-WROL time, the processing for optimum performance is already done for a WROL event.
Join!
