mirror of
https://github.com/lrnselfreliance/wrolpi.git
synced 2026-10-26 19:01:39 -04:00
Create your own off-grid library
https://wrolpi.org
06ac0d2055
All three bugs confirmed; the two suggestions taken. [bug] `video_modeler` awaited ffprobe -- a subprocess per video, up to 20 per batch -- inside its `commit=True` session. With IMMEDIATE as the default that holds the write lock for the whole batch, which is the same starvation this branch is curing in `video_cleanup`. (It was already partly true before: the batch's first `session.flush([video])` took the lock ahead of the remaining ffprobes.) The batch is now read, ffprobed with no transaction open, then written. `get_or_create_ffprobe_json` in modules/videos/common.py is the path-only entry point that makes that possible; `Video.get_ffprobe_json` delegates to it and keeps appending the cache file to its FileGroup. `validate()` -- and the poster generation it can trigger -- is still inside the write transaction, and now says so. [bug] `test_download_dispatched_outside_immediate_transaction` probed `getattr(_immediate_txn, 'active', False)`, which a ContextVar never has, so it passed unconditionally. It now probes the actual write lock from a competing connection under `production_like_sessions`. Mutation-checked: moving the dispatch back inside the claim transaction fails it. [bug] `asyncio.create_task` snapshots the current Context, so a flag held for a write session's lifetime was inherited by every task spawned inside it -- and the parent's `reset()` cannot reach a child's copy, leaving that task issuing BEGIN IMMEDIATE for its *read* sessions. `create_downloads` spawns `dispatch_downloads` exactly that way, so every user-created download hit it. The flag now lives only across the session's opening `connection()` call, which also means the write lock is taken when the session opens rather than at the caller's first statement. [suggestion] `renew_recurring_downloads` scanned every recurring Download under the write lock, including the usual cycle where nothing is due. It reads first now and takes the lock only for the rows that change, re-checking status since the dispatcher may have claimed one in between. [nit] Parenthesized the directory OR in the claim join. `probe_write_lock_is_held` moved to conftest alongside `write_lock_held_briefly`; three suites use it now. Suite: 1712 passed, 5 skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .circleci | ||
| .github | ||
| .run | ||
| alembic | ||
| app | ||
| controller | ||
| debian-live-config | ||
| docker | ||
| etc | ||
| modules | ||
| pi-gen | ||
| release | ||
| scripts | ||
| test | ||
| wrolpi | ||
| .coveragerc | ||
| .dockerignore | ||
| .gitignore | ||
| .gitmodules | ||
| .gitpod.yml | ||
| .mcp.json.example | ||
| alembic.ini | ||
| conftest.py | ||
| demo_link.jpg | ||
| docker-compose.yml | ||
| help.sh | ||
| icon.ico | ||
| icon.png | ||
| INSTALL.md | ||
| install.sh | ||
| join_discord.png | ||
| LICENSE | ||
| main.py | ||
| pytest.ini | ||
| README.md | ||
| repair.sh | ||
| requirements.txt | ||
| test_all.sh | ||
| UPGRADE.md | ||
| upgrade.sh | ||
WROLPi
Create your own off-grid library.
WROLPi is a self-contained collection of software to help you survive the world Without Rule of Law.
WROLPi is intended to be run on a Raspberry Pi with an optional external drive attached. It serves up it's own Wi-Fi network so that any user with a laptop/tablet/phone can connect and use the library created by the maintainer.
Table of Contents
Features
- Videos
- Web Archives
- Maps
- Wikipedia
- File search
- eBooks (EPUB / PDFs)
- Wi-Fi Hotspot
- Universal search
- Food Inventory management
- One-Time pad generator, encrypter/decrypter
- Synchronizer/duplicator
- Food storage calculator
Module features matrix
| Module | View | Search | Download | Project |
|---|---|---|---|---|
| Videos | yes | yes | yes | yt-dlp |
| Web Archives | yes | yes | yes | Singlefile |
| Wikipedia | yes | yes | yes | Kiwix |
| eBooks | yes | yes | planned | EPUB/PDF |
| Map | yes | yes | yes | Protomaps |
| Podcasts/Audio | yes | planned | planned |
Demo
Download
Images are available at wrolpi.org
Try WROLPi!
You can try out WROLPi by running the docker containers.
- Install docker-ce and docker-compose
- Copy the latest WROLPi code.
git clone https://github.com/lrnselfreliance/wrolpi.git
- Change directory into the code base.
cd wrolpi
- Initialize git submodules
git submodule update --init
- Build the containers.
docker-compose build --parallel
- Create volumes.
docker volume create --name=openstreetmap-datadocker volume create --name=openstreetmap-rendered-tiles
- Initialize the database (SQLite in the media directory).
docker-compose run --rm api db upgrade
- Start the docker containers.
docker-compose up
- Browse to WROLPi.
More Docker documentation is available in docker/README.md
Install WROLPi
Full installation instructions are in INSTALL.md. Pick the option that matches your hardware:
- WROLPi Portable (Live USB) — boot a complete WROLPi from a USB stick on any x86 PC or laptop. Nothing is installed to the computer; the first boot creates a persistence partition on the USB so your library, database, and configuration survive reboots.
- Debian 12 — install WROLPi onto an x86 PC or laptop.
- Raspberry Pi — install WROLPi onto a Raspberry Pi with an external drive.
- Docker — run the WROLPi containers on an existing Linux host.
Charter
Guiding Principals
- Storage is cheaper than power.
- Two is one, one is none.
- Run silently.
- Primary, secondary, tertiary.
- Secure as a bookshelf in your home.
Physical properties
- A WROLPi instance should be capable of running with a minimal of hardware:
- Raspberry Pi
- SD card
- External USB hard drive
- Power supply and cables
- Wi-Fi USER device such as a phone, tablet, or laptop.
- A WROLPi instance should consume a minimal amount of electricity during WROL event. It is expected power will be scarce when WROLPi is needed most.
User expectations
- A WROLPi instance should be run and maintained by a person (MAINTAINER) with a moderate amount of Linux and Raspberry Pi experience. It is expected that they should be able to do this using only the documentation on their WROLPi.
Software properties and capabilities
- A WROLPi instance should be able to "factory-reset" itself without any outside services.
- WROLPi should function completely without any internet services.
- A user should have easy access to their data if WROLPi fails:
- For example, a user can watch their videos by navigating a short and intuitive directory structure and opening the video in their preferred video player.
- WROLPi should be self-documented. The UI should contain a tutorial for USERS as well as the MAINTAINER.
- If the UI isn't functional, the code should be documented such that a user can restore functionality.
- WROLPi should favor pre-processing, rather than processing during a WROL event. Such as re-encoding a video, or processing captions. This is to ensure that when a user adds content during non-WROL time, the processing for optimum performance is already done for a WROL event.
Join!
