* Add Playlists: ordered, mixed-type collections
A playlist (Collection kind='playlist') is a manually-curated, ordered list that
mixes item types. Squash of the playlists feature branch (24 commits).
Data model & API:
- CollectionItem is polymorphic via item_kind: 'file' (FileGroup), 'zim'
(zim_id + zim_entry), or 'url' (link to anything the WROLPi browser opens,
e.g. /map?lat=&lon=&z=). CHECK constraint enforces exactly one kind; per-kind
unique constraints; migration a7b1c2d3e4f5.
- Endpoints: create playlist; add/remove/reorder items (idempotent adds: 201 new,
200 existing); GET returns the ordered items. All mutations wrol_mode_check'd.
Config (source of truth):
- playlists.yaml (PlaylistsConfig) stores each playlist's metadata AND ordered
items (file by relative path, zim by path+entry, url inline); round-trips a
DB wipe. Supports backup import (DB_CONFIG_FILE_NAMES).
- The redundant generic CollectionsConfig (collections.yaml) is deleted:
channels/domains own their configs, authors/subjects are re-derived.
On-disk sync (sync_playlists_directory switch):
- Each playlist materializes into the configurable Playlists Directory
(playlists_destination, Settings UI input, excluded from indexing) with
position-prefixed entries: NNNN_<file> hard links (copy fallback across
filesystems) and NNNN_<title>.html stubs for zim/url items.
- Tagged playlists are namespaced like channels: <playlists>/<tag>/<name>;
a tag change moves the directory and prunes the empty old tag dir.
- Deletion safety: managed files are only removed when they are our marked
stubs or still have another hard link (st_nlink > 1) -- never a last copy.
Stray loose files at the playlists root are removed; stub URLs are
scheme-allowlisted (no javascript:/data:).
Frontend:
- /playlists list page (CollectionTable, name filter, New Playlist modal, Tag
column) between Files and Zim in the nav.
- View-only detail page (Name link) and edit page (Edit button): themed table
with model icons/colors + poster thumbnails, items link to their model page
(video/archive/doc) or preview, up/down reorder, remove, add-URL form,
single-tag selector; all editing disabled in WROL Mode.
- Reusable green "Add to Playlist" button on Video/Archive/Doc detail pages,
the FilePreview toolbar, Zim search results, and Map pins (added as a /map
URL item with a map-marker icon).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Playlist edit page parity: rename, custom directory, tag modal move suggestion
Match the ChannelEditPage/DomainEditPage UX on the playlist edit page:
Back/View buttons, an edit form (name, description, directory), and the
shared CollectionTagModal with the violet "Tag" button. The tag modal now
suggests moving the playlist to its managed tag location
(playlists/<tag>/<name>); accepting the suggestion stores directory=None so
the sync keeps auto-managing the location. Tag buttons on the Channel and
Domain edit pages are now violet to match.
Backend: update_collection learns rename (with per-kind validation and
duplicate checks) and playlist custom directories (cleanup of the abandoned
directory, config round-trip, commit-before-switch to avoid the background
sync racing pre-commit state). tag_collection's duplicated untag/tag
branches are merged, and the per-kind config-save dispatch is consolidated
into _activate_config_save().
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Deduplicate Collection error handling and Tag creation
Delete the duplicate UnknownCollection in wrolpi/collections/errors.py and
re-export the canonical one from wrolpi/errors.py, which carries
status_code=NOT_FOUND (the local copy inherited 500). Replace the repeated
query/raise pattern in reorganize.py with Collection.find_by_id.
Add Tag.get_or_create_tag and use it for collection tagging; flush the new
Tag before returning so callers can read .id immediately -- previously
tagging a collection with a brand-new tag name set tag_id=None. Regression
test included.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Persist finalize_download mutations; isolate child-download enqueue
A channel/RSS Download could get stuck permanently "pending". Root cause:
the phase-split dispatch in signal_download_download passes the Download
loaded in an already-closed session to finalize_download, so it is
DETACHED. ChannelDownloader.finalize_download sets
`download.sub_downloader = 'video'` (and info_json/collection_id) on that
detached object, so the write never reaches the DB. The status block then
re-queries the Download fresh, reads sub_downloader=None, and calls
create_downloads(downloader_name=None) ->
`InvalidDownload: Cannot find downloader with name None`.
That exception was raised inside the status-commit transaction (outside the
inner try), so it (a) rolled back the parent's complete(), reverting it to
the `pending` set by started(), and (b) was swallowed by the outer bare
`except Exception`, which only logs. Result: the parent never failed, never
completed — it sat `pending` forever, re-crashing the same way on every
restart.
Fixes:
- finalize_download is now given a Download re-loaded in (attached to) its
own session, so its mutations persist.
- Child-download creation moves to its own transaction after the parent's
status is committed. A failure there is logged and cannot roll back or
strand the parent.
The existing test harness reuses one shared DB session for every
get_db_session call, so detached-object and rollback bugs never reproduced
under it (which is why this shipped). Add `production_like_sessions` to
conftest.py — a context manager handing out a fresh Session per call, like
production — and two regression tests that fail without the fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Re-attach Download in prepare_download too (consistency)
prepare_download receives the same detached Download as finalize_download
did, so VideoDownloader.prepare_download's writes to the row
(destination/tag_names) are not reliably persisted. Re-load the Download
attached to the prepare session, mirroring the finalize fix, so the row's
metadata is accurate.
Unlike sub_downloader (which the status block re-reads and feeds to
create_downloads), nothing downstream re-reads these values, so this is a
consistency/cosmetic improvement, not a functional bug — and it cannot be
made to fail in a test (the loaded Download's session outlives the worker),
so no dedicated test is added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Log (not swallow) session-close failures in production_like_sessions
Addresses Greptile review: surface a failed session.close() at debug level
instead of silently passing. engine.dispose() still does the authoritative
cleanup, so the close failure remains non-fatal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A failing SMART self-assessment predicts imminent drive failure, so it
deserves a prominent warning. useDriveHealth() reads the assessment
already present in smart_stats (pySMART reports PASS / FAIL / null) and
flags any drive whose assessment is FAIL. Nav.js renders a red warning
icon in its own always-on slot — like the power warning, and unlike the
single-slot priority chain where a transient high temp could hide it.
Frontend-only: the assessment field is already in the cached payload.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Warn in the navbar when a hard drive runs hot.
The navbar already warns about high CPU temperature; add the equivalent
for drive temperature, since an overheating disk risks data loss and a
user running an off-grid library won't be watching for it.
- controller/lib/smart.py: build_smart_stats() returns the per-drive list
plus warning thresholds (55C warn / 65C critical) so the frontend stays
dumb. Reading SMART wakes a spun-down drive, so drives in standby are
detected with `smartctl -n standby` (which works through the USB-SATA
bridges where `hdparm -C` only reports "unknown") and NOT read; their
last-known reading is reused instead.
- controller/lib/status_worker.py: collect SMART on a slow 60s cadence
(cached on app.state) rather than the 5s status loop, so the poll does
not keep USB drives awake. The result flows into the same cached payload
via a new collect_all_status(smart_stats=...) kwarg.
- Frontend: useDriveTemperature() reports the hottest drive; Nav.js shows
a yellow/red `hdd` icon with a popup, mirroring the CPU warning.
Verified on hardware: smart_stats reports real temps and SMART runs once
a minute, not every 5s.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Address Greptile review on HDD temperature warning.
- _drive_is_asleep also treats SLEEP mode as asleep, not just STANDBY.
smartctl -n standby exits "Device is in SLEEP mode, exit(2)" for the
deeper power mode; matching only "STANDBY" meant a SLEEP drive was read
and woken — the exact failure the check prevents. (TDD: added a failing
SLEEP test, then fixed.)
- useDriveTemperature uses ?? instead of || for the temperature fallback
to express "fall back only on null/undefined".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Applied directly since the dependabot PR could not auto-rebase after the
grouped merges.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
alembic 1.18.4 requires SQLAlchemy>=1.4.23, but the project pins
SQLAlchemy==1.3.22, making the dependency set unresolvable. This slipped
through because the backend pytest suite is not a required CI check
(only controller-tests and jest-tests are).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Real drives report attribute 194 raw values like "52 (Min/Max 16/57)"
which int() cannot parse, so every drive's temperature was null.
Prefer pySMART's `temperature` property, which parses these formats
and also works for NVMe devices that have no ATA attribute table.
Verified on real hardware (WD100EMAZ over USB-SAT): 51-53C now
reported where null was returned before.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The /api/disks/unmount endpoint only removed entries from fstab.yaml and
asked the reconciler to unmount, but the reconciler deliberately never
unmounts paths it has not claimed. Foreign mounts (udisks2 desktop
automounts, manual mounts) therefore could not be unmounted from the UI,
failing with a confusing "check journalctl" error.
- Fall back to a direct umount(8) when a reconcile leaves the requested
mount point live.
- Reject unmounting the primary WROLPi drive (/media/wrolpi) with a
clear 400, normalized to catch path traversal.
- Require the normalized mount point to be strictly under /media/
before any direct umount, since the fallback runs as root.
- Show "System" instead of an Unmount button for /media/wrolpi in both
the React UI and the Controller's simplified UI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A SIGTERM/kill (e.g. `systemctl stop`) skips the EXIT-trap cleanup, leaving
/var/tmp/wrolpi-build-pi-gen behind. The next build's `mv pi-gen-bookworm-arm64
wrolpi-build-pi-gen` would then nest into the stale dir and fail. Remove the
leftover (and intermediate zip/unzip dir) at startup, mirroring the debian
build, stripping immutable flags off blob files first.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The watcher read latest.json through the CDN, which caches it ~5 min. A
catch-up timer fire right after a release published (Persistent=true) read the
stale cached tag, mismatched, and kicked off a redundant multi-hour rebuild of
the just-published version. Read the manifest from the strongly-consistent
Spaces origin for the dedup so it always reflects the latest publish.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- In get_effective_video_settings: on unparseable sleep_requests, delete
the key instead of forcing 0.0. This lets the global config default
(already present from _extract_global_inheritable) take effect, as
suggested by Greptile.
- In handleInputEvent: add explicit `value !== ''` guard before the
isNaN check. Prevents silent NaN in form state for empty number
fields (isNaN('') === false in JS).
Both changes are strict improvements with no behavior change for
valid inputs.
`sleep_requests` (and other numeric inheritable video download settings)
could arrive as strings ('' or '0.75') from the frontend NumberInputForm
and DownloadRequest bodies. These were persisted into Download.settings
(JSONB) and later triggered:
if effective['sleep_requests'] > 0:
TypeError: '>' not supported between instances of 'str' and 'int'
in VideoDownloader.execute_download (and similar sites).
Root cause: DownloadSettings.__post_init__ only filtered None/[] and did
no type coercion, even though the dataclass declared Optional[float]/int.
Changes (minimal targeted fix):
- Add _coerce_optional_number helper and apply it to all numeric fields
in DownloadSettings.__post_init__ (the existing sanitization boundary).
- Centralize legacy string coercion for persisted data in the single
read chokepoint: get_effective_video_settings.
- Simplify VideoDownloaderConfig.sleep_requests getter.
- Frontend: use parseFloat for type=number inputs + null defaults for
sleep_requests (prevents '' from ever being submitted).
- Add focused regression test for string coercion path.
- Revert unrelated change to string field filtering behavior.
No NaN/Inf speculation, no duplicated try/float guards at every call site,
no behavior change for non-numeric settings.
Builds take hours and a failure almost always needs a code fix, not a blind
retry -- retrying 3x burned ~5h overnight. Default to a single attempt: on
failure, record the marker, fire the notify hook, and wait for a human to
investigate and clear it. The MAX_RETRIES override remains for N attempts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When the release runs under the wrolpi-release systemd service, HOME is unset,
so `git config --global --add safe.directory /opt/wrolpi` in the pi-gen chroot
aborts with "fatal: $HOME not set" and the build fails at stage 3. Manual
shell builds worked only because the shell provided HOME. Export HOME=/root in
the chroot hook (self-contained), and also set Environment=HOME=/root in the
service unit template.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The build ended with `mv "${SCRIPT_DIR}"/*.iso "${DEST}"`. A leftover ISO from
a previous build made the glob match two files, so `mv` treated the target as
a directory and failed ("No such file or directory") -- after a successful
37-minute lb build. Take the single ISO out of the freshly-wiped BUILD_DIR via
a nullglob array and copy it to the final name; robust to leftovers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Greptile (PR #448):
- C1 (P1): release-watch.sh compared the GitHub tag (minus 'v') against the
manifest's .version (from version.txt content). If a tag and its version.txt
ever diverged, the guard never matched and it rebuilt every hour forever.
Compare the manifest's .tag against the release tag directly -- exact and
immune to version-string formatting. Drops the now-unused normalize().
- C2 (P2): document that .tag is the built git ref (release tag for automated
builds, branch for manual) and that consumers use .version for display.
- C3 (P2): document that newest-by-publish-date is the intended trigger and
that prereleases are deliberately included (every release is -beta), so we
do not filter on .prerelease as suggested.
Also add an optional completion notify hook: release.sh invokes
$WROLPI_RELEASE_NOTIFY with a success/failure message (no-op if unset, skipped
on dry-run). The public script names no path or tool; the real notifier stays
on the host (commented Environment= placeholder in the service unit).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scripts/upgrade.sh runs `npm install`, which rewrites app/package-lock.json
and leaves the working tree dirty. The next upgrade's `git checkout ${BRANCH}`
then aborts with "Your local changes would be overwritten by checkout" before
the following `git reset --hard` can discard them, failing the upgrade with
exit 4.
Use `git checkout -f` so the branch switch discards local modifications; the
reset --hard immediately after already forces the tree to match origin, so the
intent (blow away local changes) is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
upgrade.sh delegates to wrolpi-upgrade.service at the very start, but that
unit is only installed by repair.sh, which runs near the end of the upgrade.
On a box that has never run a repair since the unit was introduced, the
delegation fails immediately with "Unit wrolpi-upgrade.service not found",
the service never runs (so nothing is logged to the journal), and
/opt/wrolpi/upgrade.log is never created.
Install/refresh the unit from the checked-out tree and daemon-reload right
before starting the service. The delegation block already runs as root, so
no sudoers changes are needed. This is idempotent and also self-heals a
stale unit file.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- New INSTALL.md consolidates all install paths: WROLPi Portable (Live
USB) as the headline option, plus Debian 12, Raspberry Pi, and Docker.
- README install sections collapsed into a single Install section that
links to INSTALL.md.
- UPGRADE.md gains an "Upgrading a WROLPi Portable USB" section covering
scripts/wrolpi-usb.sh upgrade, and documents that in-place conversion
of an existing data drive is not supported (backup -> flash -> restore).
- wrolpi-usb.sh install now prints that supported procedure instead of a
"not implemented" stub; debian-live-config/README.md documents the same.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Treat virtual disks (vd*/xvd*/hd*) as direct, not ephemeral. A blank
VM disk imaged with the ISO can safely hold a persistence partition;
previously these fell through the wildcard to loop mode and never
accumulated state.
- Move the ephemeral note_action inside the mountpoint guard so a manual
wrolpi-bootstrap.service restart mid-session doesn't re-fire the "NOT
writing to disk" first-boot dialog.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
wrolpi-bootstrap.sh now classifies the live medium by device-node
pattern. Real partitions (/dev/sd*, /dev/nvme*, /dev/mmcblk*) continue
to get a persistence partition appended. Loop/device-mapper sources
(/dev/loop*, /dev/dm-*, /dev/mapper/*) — the marker for Ventoy/YUMI/
Easy2Boot/etc — mount a tmpfs at /media/wrolpi instead, so we never
sfdisk-append a partition onto the user's multiboot stick.
First-boot zenity dialog branches on mode and tells ephemeral users
their data won't persist; README documents the behaviour and the
deferred persistence.dat work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address Greptile review on PR #447:
- Fall back to the minimum preset when an unrecognized/stale
water_calculator_preset is read from localStorage, so a bad persisted
value can no longer crash the component on mount.
- Remove the unused Media import.
- Simplify the metric branch of waterWeight (volume * 1 -> volume).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses calculator review feedback:
- Add an "Additional daily demand" (Extra) field for pets, livestock,
guests, or sick household members not covered by the per-person rates.
- Make the preset choice explicit buttons with a note that switching
resets rates to the preset defaults.
- Clamp rate and extra inputs to non-negative via a shared helper that
preserves in-progress decimal entry (so "1.5" is typeable).
- Persist the duration slider to local storage (with an out-of-range
guard), matching the people-count/metric/preset persistence.
- Relabel "Total Weight" as "Water Weight" to avoid implying it includes
container tare weight.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New offline prepper calculator under Calculators:
- Men/women/children/pregnant-nursing counts with editable per-person
daily rates (drinking + hygiene/dish use).
- Minimum/Comfortable presets grounded in CDC/FEMA (1 gal/person/day
floor, store 2 weeks) and National Academies adequate-intake figures;
Pregnant/Nursing adds the CDC/Ready.gov +0.5/+1 gal. Sources cited in
the Daily Use info popup.
- Duration slider on a clean ladder (days → weeks → months → years) so it
never lands on awkward values like "68 days".
- Containers Needed table listing every container (IBC tote → 500 mL
bottle) with capacity and count + spare for the total volume.
- Total water and total weight (8.345 lb/gal, 1 kg/L).
- Imperial/metric, preset, and people counts persist via local storage.
Pure functions (dailyDemand, totalWater, containersNeeded, waterWeight,
formatDuration) covered by 21 unit tests. Registered in the calculator
list and global app search.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address Greptile review: remove the per-render and per-keystroke console.debug
calls (they fired on every React render and every field edit). Keep the single
strategic solveTrain log, which fires only on a settled (debounced) solve.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a SUGGESTED_APPS entry so the Drive Ratio calculator shows under "Apps"
in the global search, matching the other calculator entries.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A new "Drive Ratio" calculator under /more/calculators for belt, gear, and
chain-sprocket drives, supporting a compound train of 2–6 wheels on a single
shared belt/chain (constant size·rpm across the train).
- Pure, unit-tested math: size·rpm solver, ratio, torque, surface/belt speed,
belt length, sprocket pitch diameter, chain length, gear pitch diameter.
- Mode toggle (Pulley/Gear/Sprocket) with imperial/metric units; torque in
lb·ft or N·m.
- Add/remove wheels (2–6), each with a colorblind-friendly color shared between
its inputs and the schematic SVG diagram (Paul Tol bright palette).
- Dependency-free inline-SVG diagram: colored wheels threaded by one belt/chain,
or meshing gear teeth; auto-fitting viewBox.
- Per-mode details: belt/chain length and center distance (2-wheel only, with an
info popup), per-wheel pitch diameters, and constant train speed.
- Guards against impossible geometry (wheels overlapping) and ideal-torque note.
- Integer teeth enforced in gear/sprocket modes; pulley diameters stay continuous.
Also fix Theme.Header dropping the theme's text color when a caller passes a
`style` prop (now merges {...t.style, ...props.style}).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
numfmt with a negative argument:
Under `set -euo pipefail` the upgrade subcommand would die at
`numfmt --to=iec ... $headroom_bytes` whenever the new ISO was larger
than the headroom — numfmt treats the leading `-` as an option flag,
exits non-zero, and the script aborts before printing the descriptive
back-up-and-reflash error message just below. Splitting positive
and negative cases and feeding numfmt the absolute value fixes it;
the eventual error message now actually shows up.
Silent umount during sanity check:
`umount "$child" 2>/dev/null || true` was swallowing both stderr and
the exit code. A busy partition (open file handle from a daemon
that auto-opened the USB) would survive the loop, and `cmd_upgrade`
would proceed straight to `dd if=<iso> of=<dev>` — writing to a
block device whose partition is still mounted, corrupting the
filesystem on that partition. Adds a post-loop assertion: re-read
MOUNTPOINTS and die with an `lsof` hint if anything is still mounted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
