The Disks page treated /media/wrolpi as fully protected, so a
manually-mounted primary drive could not be persisted from the UI — and
the Persist toggle would have been wrong anyway: it wrote fstab.yaml,
which the wrolpi-mounts reconciler refuses to apply for the primary
mount (RESERVED_MOUNT_POINTS). Route Persist for the primary mount to
the host's /etc/fstab (as onboarding does), report its entry in the
fstab list endpoint, and enable the toggle in both UIs while keeping
Unmount blocked. Mounts outside /media (e.g. /boot/efi, which showed
an Unmount button) are now entirely untouchable System mounts.
Also:
- Run `ntfsfix -d` before mounting ntfs/ntfs3 volumes so the Windows
dirty flag (Fast Startup, unclean unplug) cannot block mounting;
non-fatal and a no-op on clean volumes.
- Add uid/gid fstab options for ntfs/ntfs3, matching mount_drive().
- Log reconcile diagnostics (desired/already-mounted/skipped with
reasons) so an idle wrolpi-mounts run explains itself in the journal.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Semantic UI may activate a Dropdown.Item from the keyboard without
following its href, which would save the slicer preference without
launching the slicer. Prevent the default anchor navigation and call
window.location.assign() so every activation path behaves the same.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a split button to the STL and 3MF preview toolbars that opens the
model directly in a desktop slicer via its URL scheme
(scheme://open?file=<absolute media URL>). The slicer downloads the
file itself over the network.
Offers Orca Slicer, Bambu Studio, and Cura; the last-used slicer is
remembered in localStorage and becomes the split button's default,
since browsers cannot detect which protocol handlers are installed.
PrusaSlicer is deliberately omitted: it only downloads from domains
whitelisted by Prusa (printables.com, thingiverse.com, cults3d.com),
so it cannot fetch from a WROLPi.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On bootstrapped (portable) systems the cluster lives under
/media/wrolpi/config/postgresql, bind-mounted over /var/lib/postgresql;
the recursive chown of the config directory swept the cluster to
wrolpi:wrolpi, breaking new backends. No-op on normal installs where
that directory does not exist.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
If cluster files lose postgres ownership (stray recursive chown,
interrupted install), the postmaster stays up but every new backend
dies with FATAL: could not open file "global/pg_filenode.map":
Permission denied — and dropdb/psql cannot work either. Restore
ownership of /var/lib/postgresql and /etc/postgresql and restart
postgres before recreating the database, and start the API when done.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The caddy package install hit a dpkg conffile prompt because repair.sh
already wrote /etc/caddy/Caddyfile; under the tty-less
wrolpi-upgrade.service the prompt got EOF on stdin and apt exited 100.
Run the caddy/samba/gnome-disk-utility installs with
DEBIAN_FRONTEND=noninteractive and --force-confdef/--force-confold
(keep the current conffile; repair.sh manages the Caddyfile anyway),
and run `dpkg --configure -a` first so a box left half-configured by a
previously interrupted run is healed automatically.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
If the caddy keyring file already exists (left by a prior partial
install), gpg prompts to overwrite via /dev/tty, which fails under the
tty-less wrolpi-upgrade.service ("cannot open '/dev/tty'") and aborts
the upgrade. --batch --yes overwrites silently, making the caddy
install idempotent in all four scripts that share this pattern.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Same fix install.sh received in bf88ac9f: on a box with no local
release branch (installed before that patch), the tracked release/
directory makes a bare `git checkout -f release` fail with
"could be both a local file and a tracking branch", aborting upgrades.
`checkout -f -B <branch> origin/<branch>` is unambiguous and already
resets the branch to origin, replacing the separate reset --hard.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Global IPs from `hostname -I` (e.g. routable IPv6 assigned via SLAAC)
violated the Root CA's critical nameConstraints, failing leaf verification
with "permitted subtree violation" (error 47) and aborting
install/repair/upgrade. Filter SAN candidates, including EXTRA_SANS, to the
permitted private ranges and log skipped entries; normalize the IP:/DNS:
prefix case in EXTRA_SANS since openssl rejects lowercase prefixes.
Also guard the service stops in upgrade.sh so upgrading a box whose
install died before the units were installed can still recover.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* POST /api/hotspot/settings checks is_docker_mode() directly instead of
string-matching the error message (still 500, matching the sibling
hotspot/bluetooth/throttle endpoints).
* Escape \ ; , " in the WiFi QR code per the WPA QR format.
* Skip the controller.yaml write when a settings update changes nothing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Hotspot was reported as "not supported" on x86 machines (Ubuntu/Debian PCs)
because the Controller only watched the hardcoded default wlan0 device, while
predictable interface naming produces names like wlp2s0. The Settings page
saved hotspot_device to wrolpi.yaml, which the Controller never read.
The Controller now owns hotspot settings in controller.yaml:
* New endpoints: GET/POST /api/hotspot/settings, GET /api/hotspot/devices
(WiFi interfaces discovered via nmcli).
* Legacy hotspot_* values in wrolpi.yaml are migrated into controller.yaml
once on startup, unless controller.yaml already sets them.
* The Hotspot SSID/password/device fields moved from the Settings page to the
Control page next to the Hotspot toggle; the device is now a dropdown of
the WiFi devices found on the machine.
* The Controller fallback UI gets the same hotspot settings form for parity.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Etcher is privacy-invasive; replace it in INSTALL.md, the USB upgrade
tool messages, and the live-boot ephemeral warning.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds Tags, Configs, FileGroups, Downloads, and Upgrade docs, the
WROLPi Portable section in Getting Started, and a synced TOC.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
After a Channel refresh, log how many video/audio FileGroups are in the
Channel's directory, and how many are assigned to this Channel, other
Channels, or no Channel. Helps debug Videos missing from their Channel
after files are moved.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tag_file join in Channel.get_statistics() produced one row per tag,
so a Video with N Tags was counted N times in video_count, size, and
length. Count tagged Videos in a subquery instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a client-side filter above the File Browser table. Typing reduces
the visible rows to files/directories whose name matches, plus their
parent folders. A folder whose own name matches keeps its entire
subtree, and opening a matching folder shows its contents unfiltered.
* 'f' focuses the filter input, like the search inputs on other pages.
* An attached X button clears the filter; disabled while empty.
* A collapse-all (angle double up) button closes all open folders,
clearing the ?folders= query param and deselecting hidden children.
* The filter row is right-aligned and double-width on desktop, and
fills a single row on mobile.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The DeepSearchHint only appeared on zero results, so a fast search with
matches gave no indication that captions/contents were not searched.
Module searches have no deep estimate, so show a generic reminder
whenever a fast search is active: "More results may be available in
deep search." with results, "Not finding what you need?" without.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The file_group.textsearch tsvector indexes all four text columns;
d_text (captions, document text, zip contents) is ~90% of the data,
making ts_rank searches take seconds on large libraries. Add a
textsearch_abc generated column (title/author/description only, ~8x
smaller) and default all searches to it, with deep=true opting into the
full column.
Database:
- Migration adds the generated column and a GIN index (rewrites the
table; a few minutes on large libraries).
Backend:
- FilesSearchRequest, VideoSearchRequest, ArchiveSearchRequest, and
DocSearchRequest accept `deep`; each search function picks the column.
- /api/search_file_estimates returns both counts in one index-bound
query (COUNT FILTER on the abc subset): `file_groups` (fast) and
`file_groups_deep`.
UI (one coherent story across the count surfaces):
- The typing dropdown always suggests the deep count (what is findable).
- The tab badge shows the active mode's count so it matches the result
list.
- The gap is reconciled by a hint offering deep search: count-aware on
the global search ("No results in fast search. N results available in
deep search." / "N more results available..."), generic on the module
pages (no deep estimate there). The hint resets pagination.
- "Search Depth" toggle in the search filter modals, persisted as
?deep=true.
Tests that relied on d_text matches now pass deep=True, with fast-mode
assertions alongside. Also skip flaky test_download_channel on
CircleCI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A failed "Filter files by detecting device" only surfaced the download-mode
reminder down in the Connect segment, off-screen. Extract the message into a
shared renderer and show it both places, so the error appears right where the
user clicked on the "Choose from your WROLPi" tab.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- ESP_CHIP_ID_NAMES disagreed with the backend ESP_CHIP_IDS: it mapped 18 to
ESP32-C5 (18 is ESP32-P4) and omitted 17 entirely, so C5 firmware showed as
"chip id 17" and P4 firmware was mislabeled "ESP32-C5". Now matches the
backend (17=ESP32-C5, 18=ESP32-P4), with a test guarding it.
- Removed the stale filesSearch import/mock/beforeEach from the test; the
flasher no longer calls it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prevents the wrong-chip flash that bricks a board (e.g. an ESP32-S2 bootloader
in an ESP32 set): after connecting, each selected file's ESP image header is
checked against the device's chip_id and the offending row is flagged (a
warning, not a block). Changing the firmware selection now disconnects so the
check is always re-run against a freshly detected chip.
- "Choose from your WROLPi" is now a table (File / Path / Size / Chip / Kind
/ add), served by a single /flasher/search call
- search_esp_firmware returns every .bin annotated when unfiltered (non-ESP
parts like partition tables, boot_app0, and littlefs stay pickable); a chip
filter still narrows to that chip's ESP images
- Detect-device filter now searches by the canonical chip name (CHIP_NAME,
e.g. "ESP32") instead of the variant description ("ESP32-D0WD-V3"), which
matched nothing
- Chip column uses deterministic, colorblind-safe colors; same-width chip
names always get distinct colors, and the palette has headroom to grow
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Browser-based ESP32/ESP8266 flasher (esptool-js + Web Serial); nothing is
uploaded to WROLPi, the flash is written straight to the device over USB.
Firmware sources (themed, tabbed): a saved set, the WROLPi media directory,
or a local file. Local files can be dropped anywhere on the page, which jumps
to the "Add from computer" dropzone. The media picker searches .bin files by
path and can detect the connected chip to filter firmware to matching images.
- Indexed file_group.suffix + path filter for fast .bin search (alembic
migration backfills via split_path_stem_and_suffix)
- modules/flasher: chip detection from ESP image headers, /search endpoint,
and flasher.yaml saved configurations (/configs CRUD)
- Flash progress modal with ETA, connect timeout, boot-mode hint, auto-
scrolling logs
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
YouTube's new listing layout means yt-dlp no longer returns `view_count`
in flat channel/playlist entries, so sorting by views raised KeyError and
the download failed and retried forever.
Sorting by views now requires per-video extraction (hundreds of requests
per channel), so the feature is deprecated rather than degraded:
- downloader: the 'views' branch raises UnrecoverableDownloadError so the
download fails cleanly (no retry loop) with a message to switch to
Newest/Oldest. May be revisited via the YouTube Data API.
- schema: download_order validation drops 'views' (newest/oldest/null only).
- UI: removed "Most Views" from the download-order dropdown.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
getSettings() in api.js remaps the /settings response through a fixed field
whitelist, which dropped download_daily_limit_global/per_domain before they
reached the SettingsContext. The values saved correctly but always rendered
empty. Add both fields to the mapping.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Settings.js: send 0 (not null) when a daily-limit field is cleared, so the
settings API actually clears a previously-saved limit instead of dropping the
null and keeping the old value.
- downloader.py: document that the daily limit is a best-effort soft cap; minor
bounded overshoot is possible when create_downloads' background dispatch overlaps
the perpetual cycle, and self-corrects the next cycle. A strict cross-process cap
is intentionally not implemented.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds two optional DownloadManager settings:
- download_daily_limit_per_domain: max non-recurring downloads per domain per day
- download_daily_limit_global: max non-recurring downloads across all domains per day
Recurring Channel/RSS catalog updates are never gated and never counted; only the
videos/archives they spawn and one-off downloads count. Both successful and failed
attempts consume a slot (counted once per download per day). Limits reset at local
midnight (configured timezone). Over-limit downloads stay "new" and resume the next day.
Implementation:
- New Download.last_download_attempt column (+ index, migration) set at dispatch time so
the count survives restarts and is correct across workers.
- dispatch_downloads gates non-recurring downloads: per-domain skips that domain, global
stops dispatching counted downloads for the cycle. daily_download_counts derives the
day's totals from the DB.
- Domain grouping uses normalized URLs via a new Downloader.normalize_domain method
(base strips www/m/mobile + port); VideoDownloader/ChannelDownloader override it to
collapse youtu.be, /shorts/, and tracking params to youtube.com. ArchiveDownloader is
the seam for future archive URL normalization.
- Settings wired through WROLPiConfig, the /settings API (GET/PATCH, non-negative
validation, 0 = unlimited), and the React Settings page.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The dark-theme rule hardcoded the inverted Accordion's title/content
background to #171616, which is the Modal content color. Inside a Modal
this matched, but on the /admin/status page (and any Segment, which
renders at #1B1C1D) the Accordion showed up as a darker band.
Make the inverted Accordion transparent so it inherits whatever
container it sits in, matching how the light theme already renders
accordions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix FileWorker status stranded mid-phase (e.g. stuck on "Cleanup")
The FileWorker stores its status in shared context and relies on an explicit
reset_status() in each handler. Two execution paths set the status (comparing/
upserting/deleting/modeling/indexing/cleanup) but could finish without ever
resetting it, freezing the UI at the last phase:
- refresh_sync() ran _apply_post_processing (final phase sets status='cleanup')
but never called reset_status(). Deleting files in the UI goes through
lib.delete -> refresh_sync, so any delete that changed the DB stranded the
worker at 'cleanup' (observed on 10.0.0.8 after deleting STL files).
- process_queue() wrapped the handler dispatch in try/finally with no except.
An exception in handle_count/handle_refresh (which, unlike handle_move/
handle_reorganize, have no try/except) escaped to the perpetual worker loop
and left the status frozen at whatever phase was running ("any state").
Fixes:
- refresh_sync(): reset_status() in a finally.
- process_queue(): on exception record a terminal 'error' status (so the worker
is not stranded mid-phase) and re-raise so callers and the perpetual loop
still observe the failure. This covers both handle_count and handle_refresh.
Tests: add replication tests for the synchronous-refresh, handler-error, and
count-error stranding paths, plus the previously-untested wait_for_job timeout.
* refresh_sync: leave terminal 'error' status on failure
Address Greptile review: refresh_sync reset to idle in a finally that also ran
on failure, so a synchronous refresh that raised after a phase was set reported
a clean 'idle' to the status endpoint instead of the failure (the exception was
still raised to the caller). Make it symmetric with process_queue: reset to idle
only on success, record status='error' and re-raise on failure.
Add test_refresh_sync_records_error_status_on_failure.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The tracked release/ directory collides with the release branch name, so a
bare `git checkout release` is ambiguous ("could be both a local file and a
tracking branch") and aborts a fresh install. Use `checkout -B <branch>
origin/<branch>`, which can never be parsed as a path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Restore the censored-video search filter
Censored videos (no longer publicly available — detected when a channel
refresh drops them from the channel list, or when comment downloads fail
with "video unavailable/private/removed") are already flagged on
FileGroup.censored, but there was no way to find them since the search
rework dropped the old filter.
Add a "Censored" toggle to the reworked search filter modal, shown only on
the Videos page (showCensored). When enabled the video search returns only
videos with FileGroup.censored = TRUE, tracked via a ?censored=true query
param (useSearchCensored).
* Move Availability filter below the Tags/Results row
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Surface coarse SMART health for USB-bridge drives
USB enclosures that reject ATA pass-through (e.g. Seagate Expansion) are
invisible to pySMART/DeviceList, so the drive shows no SMART at all on the
Controller page. Fall back to smartctl over the SCSI transport, which many
such bridges still answer, to report a coarse PASS/FAIL health plus drive
identity (model/serial/capacity).
The health (-d scsi -H) and identity (-d scsi -i) reads are issued as
separate calls: combining them makes smartctl suppress the health line when
the identity read reports "SMART support is: Unavailable" (which these
bridges do). The entry is marked smart_limited (labelled "(limited)" in both
UIs) so a green PASS is not mistaken for full monitoring — these drives
expose no attributes or temperature, and some bridges report "OK" statically.
* Harden SCSI health fallback (Greptile review)
- _scsi_health: treat any unrecognised status (e.g. "UNKNOWN", "N/A") as
unreadable (None) instead of a false FAIL. Only "OK" -> PASS and "FAIL*"
-> FAIL; anything else omits the drive rather than showing a red FAIL that
contradicts the smart_limited disclaimer.
- get_all_smart_status: isolate the SCSI supplement loop in its own
try/except so an error there cannot discard the pySMART results already
collected.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Consolidate search options into a single Filter modal
Reworks the file-search UI on the Dashboard, Videos, Archives, and Docs
pages. The crowded row of controls (sort, tags, limit, file-type filter,
date selector) is replaced with a single "Filter" button that opens one
comprehensive modal.
- Add SearchFilterModal + SearchFilterButton (violet + count badge when
active). Sections (Sort, File Type, Tags, Results Per Page, Published
Date) are shown per-page via props; layout uses a Semantic Grid.
- Selections are drafted locally and applied in a single updateQuery only
on close/Done, so the search isn't re-run on every field change.
- Results Per Page and Sort use buttons (fewer clicks, no clipped dropdown).
- Extract shared SearchControlBar (view toggle + input + Filter) for the
three results pages; move the global-search view toggle onto the tab row.
- Rework the Dashboard search line into one adaptive flex row; clear button
only shows when there's an active search.
- Remove now-dead FilesView limit/tag returns and SearchLimitDropdown.
* Address review: reset limit on Clear All; use replace over replaceAll
- handleClearAll now resets Results Per Page to the default page size
(DEFAULT_SEARCH_LIMIT = 24) so "Clear All" actually clears every setting.
- Replace start-anchored replaceAll(/^-/g) with replace(/^-/) for the sort key.
- Add a test covering the Clear All limit reset.
* Remove now-unused Media import from Files.js
The FilesSearchView refactor dropped the last <Media> usage in this file,
leaving an unused import (a build warning). Remove it.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
