1acb148c90
* Singlefile runs under Deno 2.8.3; browser leftovers are reaped.
single-file-cli is designed for Deno. Under node 18 (Debian 12/RPi OS) it
cannot reach the browser CDP on IPv6-enabled hosts ("localhost" resolves to
::1 first with no IPv4 fallback) and it never exits after a successful save,
blocking the download worker forever.
The native path and the Archive docker container now run single-file with
"deno run". Deno is pinned at v2.8.3 everywhere because yt-dlp 2026.06.09
requires at least 2.3.0 (our node is Debian's v18, below yt-dlp's v22
minimum). Deno >= 2.3 kills the browser single-file spawns (piped stderr
that single-file never reads), so the browser is run through
singlefile_browser_wrapper.sh which discards its stderr.
Commands are run in their own process group which is killed when they
finish, reaping the browser single-file leaves behind. The Archive
downloader now has a ten minute timeout. Install scripts install Deno
(previously only upgrade.sh and the image builds did).
* Singlefile writes to a file instead of dumping to stdout.
Deno >= 2.3 truncates large --dump-content output at exit; compressed
(SingleFileZ) singlefiles of large pages lost their ZIP tail, failed the
compression check in write_archive_files, and were destroyed by the HTML
prettifier (blank white archives). single-file now writes its result to a
temporary file which WROLPi reads back.
As insurance, a singlefile marked compressed (<html data-sfz) whose ZIP is
unreadable is now stored verbatim instead of being prettified.
* Compressed Archive screenshots and readability images are no longer broken.
The index.html inside a compressed (SingleFileZ) singlefile references its
resources by relative ZIP paths. Screenshots are now taken from the fully
extracted ZIP so those references resolve. Readability images are inlined
as data URIs read from the ZIP (readability-extractor absolutizes relative
srcs against the page URL, so both forms are matched).
* Compressed Archives are created by converting the uncompressed capture.
The page is always captured uncompressed; readability, screenshot, and
title come from that canonical page (its images are inline data URIs, so
no ZIP extraction or image inlining is needed). When compress_singlefile
is set, a second single-file run converts the captured file into a
compressed (SingleFileZ) self-extracting file. The conversion is a local
file:// operation - the page is not fetched again, and single-file
preserves the original URL header when re-saving its own pages. If the
conversion fails, the uncompressed singlefile is stored instead.
* Consolidating duplicate compressed-singlefile tests.
The write_archive_files tests (verbatim write, title fallback, broken-ZIP
guard, readability inlining) are combined into one test; the helper probes
and the setting-plumbing tests are each merged into one. The do_singlefile
compression-failure scenario and the timeout constant are folded into the
main do_singlefile test, which shared their setup.
* Addressing review comments: removeprefix, PermissionError when killing.
* Bumping version to 0.24.1-beta.
* Install scripts assert the deno.zip hash to avoid malicious or invalid files.
The hashes are the official sha256sum assets of the Deno v2.8.3 release,
per architecture.
* Archive Dockerfile asserts the deno.zip hash to avoid malicious or invalid files.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| config | ||
| files | ||
| build.sh | ||
| README.md | ||
Debian Live Config
Build a bootable amd64 ISO that boots into a full WROLPi system on any x86 PC. Users can run WROLPi directly off the USB (a persistence partition is created on the first boot so the database and media directory survive reboots) or install to disk via the Calamares launcher on the desktop.
The chroot-phase build is identical for both modes; the resulting squashfs is what runs in live mode AND what Calamares copies onto the target disk.
See https://live-team.pages.debian.net/live-manual/html/live-manual/index.en.html
Build
sudo ./build.sh # build from 'release' branch
sudo ./build.sh -b master # build from a specific branch
Requires Debian / Ubuntu with live-build installed. The resulting ISO is
written to this directory as WROLPi-v${VERSION}-amd64.iso.
Flash with Etcher, Rufus, Raspberry Pi Imager, or:
sudo dd if=WROLPi-v${VERSION}-amd64.iso of=/dev/sdX bs=4M status=progress conv=fsync
Upgrading an existing WROLPi USB
A WROLPi USB carries two partitions: the ISO (boot) partition and a
persistence partition created on first boot that holds your library,
database, and configuration. To move to a newer ISO without losing the
persistence partition, use scripts/wrolpi-usb.sh from any Linux host:
sudo ./scripts/wrolpi-usb.sh upgrade WROLPi-v${VERSION}-amd64.iso /dev/sdX
It backs up the partition table, overwrites only the ISO partition, and re-adds the persistence partition entry at its original location. The ISO partition reserves 8 GiB of headroom so a larger future ISO still fits ahead of the persistence partition.
Converting an existing data drive (not supported)
There is no in-place conversion of a drive that already holds your data
into a WROLPi USB — flashing the ISO erases the whole drive, and there
is no safe, filesystem-agnostic way to graft WROLPi onto a full drive.
scripts/wrolpi-usb.sh install only prints this procedure:
- Back up everything on the drive to another disk.
- Flash the whole drive with the ISO (
dd/Etcher/Rufus — this erases it). - Boot once so WROLPi creates its persistence partition and sets itself up.
- Copy your data into
/media/wrolpi/, then let WROLPi refresh/repair to index the restored files.
Afterwards, future ISO upgrades preserve your library via the
upgrade command above.
Layout on disk
The drive carries everything WROLPi needs. Postgres data lives alongside the user's library so a drive moved between hosts keeps its full state:
/media/wrolpi/
videos/ ← user-facing media
archives/
zims/
map/
tags/
config/
wrolpi.yaml
controller.yaml
fstab.yaml
ssl/
postgresql/
data/ ← bind-mounted to /var/lib/postgresql
config/ ← bind-mounted to /etc/postgresql
Booting via Ventoy or other multiboot USBs
Booting WROLPi via Ventoy, YUMI, Easy2Boot, AIO Boot, GLIM, or any other tool that loop-mounts the ISO works, but only in ephemeral mode.
When the live medium appears under /dev/loop*, /dev/dm-*, or
/dev/mapper/* (the kernel's tell-tale for a loop/device-mapper-backed
file rather than a real partition), wrolpi-bootstrap.sh refuses to
create a persistence partition. The underlying disk in that case is the
user's Ventoy stick, not a blank target — appending a new ext4 partition
to it would risk corrupting the user's other ISOs and data.
Instead the script mounts a tmpfs at /media/wrolpi (sized to half of
RAM by default), and the rest of the boot is unchanged. Postgres
bind-mounts and DB initialisation all succeed, just into RAM. The user
sees a "Running ephemerally…" notice in the first-boot dialog.
Everything written to the WROLPi library — downloads, archives, video
metadata, the database — is lost on reboot. This mode is intended for
trying WROLPi out without committing a USB stick; for any real use,
flash the ISO directly to its own drive with dd or Etcher.
Direct (dd/Etcher) boot is distinguished by the live medium living under
/dev/sd*, /dev/nvme*, or /dev/mmcblk*; this is the only shape
where wrolpi-bootstrap.sh will touch the partition table.
(Future work: integrate with Ventoy's native persistence.dat flow so
the user can opt into a persistent ephemeral-style image without
needing a dedicated drive.)
Implementation status
- Phase 1 — merged live + chroot config: Single
debian-live-config/produces a Debian Live ISO with the full WROLPi stack, XFCE auto-login aswrolpi, persistence partition created on first boot. - Phase 2 — Debian Installer: Standard
debian-installerbundled vialb config --debian-installer live --debian-installer-gui true. GRUB exposes "Live", "Install", and "Graphical Install" entries. Live boots into WROLPi; Install/Graphical Install copy the running squashfs onto the target disk via Debian'slive-installerflow. (We previously tried Calamares-as-the-installer; the XFCE desktop-icon click path proved unreliable in the wrolpi auto-login session and the upstream d-i path is better-tested.) - Phase 3 — Ventoy/multiboot safety:
wrolpi-bootstrap.shdetects when it's been booted via a loop-mounted ISO (Ventoy and friends) and falls back to a tmpfs-backed/media/wrolpiinstead of writing a new partition onto the user's multiboot stick. - Release: CI on tag, hosting, wrolpi.org docs.