mirror of
https://github.com/lrnselfreliance/wrolpi.git
synced 2026-11-19 14:36:22 -05:00
Create your own off-grid library
https://wrolpi.org
* A move neither scans every FileGroup nor stalls the downloader's writes
Two stalls made a 109 file move take three minutes on a large library:
- Planning joined the move_sources temp table to file_group. The temp
table has no statistics, so with a real library's statistics SQLite
scanned all of file_group (233k rows, 2.4 GB) instead of looking up 109
paths in the primary_path index. CROSS JOIN fixes the join order.
- The move's write transaction yielded to the event loop between chunks.
The process that moves files also downloads; a download's write then
waited on the move's lock synchronously, so the event loop could not
resume the move: each write waited out the 30 s busy_timeout and failed
("Unable to renew downloads!"). The transaction now runs in a thread
(inline in tests, which share one connection), so another writer waits
only until the move commits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A cancelled move waits for its thread however often it is cancelled
Review fix: the move waited for its thread on the first cancellation by
awaiting the thread's future again, unshielded. Shutdown cancels twice
(run_loops cancels the file worker loop and stops waiting after 15 s, then
asyncio.run cancels every pending task, including the thread's future), so
the second cancellation raised while the thread was still in its
transaction, and handle_move reverted the files the thread then committed.
The move now waits on a threading.Event the thread sets when it leaves its
transaction, absorbing every cancellation meanwhile. A cancelled future
reads "done" while its thread still runs, so it cannot be that signal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|---|---|---|
| .github | ||
| .run | ||
| alembic | ||
| app | ||
| controller | ||
| debian-live-config | ||
| docker | ||
| etc | ||
| modules | ||
| npm-tools | ||
| pi-gen | ||
| release | ||
| scripts | ||
| test | ||
| wrolpi | ||
| .coveragerc | ||
| .dockerignore | ||
| .gitignore | ||
| .gitmodules | ||
| .gitpod.yml | ||
| .mcp.json.example | ||
| alembic.ini | ||
| conftest.py | ||
| demo_link.jpg | ||
| docker-compose.yml | ||
| help.sh | ||
| icon.ico | ||
| icon.png | ||
| INSTALL.md | ||
| install.sh | ||
| join_discord.png | ||
| LICENSE | ||
| main.py | ||
| pytest.ini | ||
| README.md | ||
| repair.sh | ||
| requirements.lock | ||
| requirements.txt | ||
| test_all.sh | ||
| UPGRADE.md | ||
| upgrade.sh | ||
WROLPi
Create your own off-grid library.
WROLPi is a self-contained collection of software to help you survive the world Without Rule of Law.
WROLPi is intended to be run on a Raspberry Pi with an optional external drive attached. It serves up it's own Wi-Fi network so that any user with a laptop/tablet/phone can connect and use the library created by the maintainer.
Table of Contents
Features
- Videos
- Web Archives
- Maps
- Wikipedia
- File search
- eBooks (EPUB / PDFs)
- Wi-Fi Hotspot
- Universal search
- Food Inventory management
- One-Time pad generator, encrypter/decrypter
- Synchronizer/duplicator
- Food storage calculator
Module features matrix
| Module | View | Search | Download | Project |
|---|---|---|---|---|
| Videos | yes | yes | yes | yt-dlp |
| Web Archives | yes | yes | yes | Singlefile |
| Wikipedia | yes | yes | yes | Kiwix |
| eBooks | yes | yes | planned | EPUB/PDF |
| Map | yes | yes | yes | Protomaps |
| Podcasts/Audio | yes | planned | planned |
Demo
Download
Images are available at wrolpi.org
Try WROLPi!
You can try out WROLPi by running the docker containers.
- Install docker-ce and docker-compose
- Copy the latest WROLPi code.
git clone https://github.com/lrnselfreliance/wrolpi.git
- Change directory into the code base.
cd wrolpi
- Initialize git submodules
git submodule update --init
- Build the containers.
docker-compose build --parallel
- Create volumes.
docker volume create --name=openstreetmap-datadocker volume create --name=openstreetmap-rendered-tiles
- Initialize the database (SQLite in the media directory).
docker-compose run --rm api db upgrade
- Start the docker containers.
docker-compose up
- Browse to WROLPi.
More Docker documentation is available in docker/README.md
Install WROLPi
Full installation instructions are in INSTALL.md. Pick the option that matches your hardware:
- WROLPi Portable (Live USB) — boot a complete WROLPi from a USB stick on any x86 PC or laptop. Nothing is installed to the computer; the first boot creates a persistence partition on the USB so your library, database, and configuration survive reboots.
- Debian 13 — install WROLPi onto an x86 PC or laptop.
- Raspberry Pi — install WROLPi onto a Raspberry Pi with an external drive.
- Docker — run the WROLPi containers on an existing Linux host.
Charter
Guiding Principals
- Storage is cheaper than power.
- Two is one, one is none.
- Run silently.
- Primary, secondary, tertiary.
- Secure as a bookshelf in your home.
Physical properties
- A WROLPi instance should be capable of running with a minimal of hardware:
- Raspberry Pi
- SD card
- External USB hard drive
- Power supply and cables
- Wi-Fi USER device such as a phone, tablet, or laptop.
- A WROLPi instance should consume a minimal amount of electricity during WROL event. It is expected power will be scarce when WROLPi is needed most.
User expectations
- A WROLPi instance should be run and maintained by a person (MAINTAINER) with a moderate amount of Linux and Raspberry Pi experience. It is expected that they should be able to do this using only the documentation on their WROLPi.
Software properties and capabilities
- A WROLPi instance should be able to "factory-reset" itself without any outside services.
- WROLPi should function completely without any internet services.
- A user should have easy access to their data if WROLPi fails:
- For example, a user can watch their videos by navigating a short and intuitive directory structure and opening the video in their preferred video player.
- WROLPi should be self-documented. The UI should contain a tutorial for USERS as well as the MAINTAINER.
- If the UI isn't functional, the code should be documented such that a user can restore functionality.
- WROLPi should favor pre-processing, rather than processing during a WROL event. Such as re-encoding a video, or processing captions. This is to ensure that when a user adds content during non-WROL time, the processing for optimum performance is already done for a WROL event.
Join!
