POST /api/videos/channels failed with "database is locked" on
`INSERT INTO collection` on the docker WROLPi while background workers
(config saves, refreshes) were writing. The handler wrote through the
deferred request session: `create_channel` reads (the URL/name/directory
conflict check) and then INSERTs, and SQLite refuses that mid-transaction
lock upgrade instantly when another connection wrote in between --
busy_timeout is skipped for upgrades.
Same shape and same fix as POST /api/download (daf2ba39): the handler
now uses `get_db_session(commit=True)`, which begins IMMEDIATE so the
concurrent writer waits its turn, and which commits before the refresh
is fired so the refresh does not wait on a lock the request still holds.
The regression test commits a write from a second connection between
the handler's conflict check and its INSERT. The existing
`write_lock_held_briefly` helper cannot reproduce this: the Sanic test
client takes over a second to start the app, so its lock is released
before the handler runs. Before the fix the test fails with the
production error verbatim.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Refreshing a specific file (the Files page Refresh button) bypassed the
ignored_directories check that directory and global refreshes apply.
Refreshing a hard link inside the playlists directory therefore created a
second Video for the same file.
The direct-file path now drops existing files that live in an ignored
directory before comparing against the database. Deleted paths are kept
so stale records are still removed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Hotspot captive portal: welcome page with the WROLPi's address
Devices that join the hotspot probe a fixed plain-HTTP URL to check for
Internet access. NetworkManager's shared-mode dnsmasq now resolves those
probe hostnames (and `wrolpi`) to the hotspot address via an include the
Controller writes when the hotspot starts, the probes land on the Controller
(port 80), and are redirected to /portal: "This is your WROLPi, its address is
10.42.0.1, open WROLPi / open the Controller".
Only the probe hostnames are hijacked so the hotspot keeps forwarding
everything else when the WROLPi has an uplink. The include is corrected and
the connection re-activated once if NetworkManager picked a non-default
subnet. Writing the include is best-effort; the hotspot starts regardless.
`hotspot.captive_portal` (controller.yaml, default true) toggles the probe
redirects and the include; like the other hotspot settings it applies on the
next hotspot start, so saving while the hotspot is up reports
restart_required and both Controller UIs say so. The status endpoint gains
the hotspot address and portal URL, shown in both UIs. repair.sh only
creates the include directory so a disabled setting survives upgrades.
Native only: Docker has no hotspot.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: address review
* React: re-read the hotspot address whenever the HotspotToggle starts or
stops the hotspot, matching the fallback UI's refresh-on-status.
* A WPA3 start that reverts to WPA2 leaves the hotspot up, so it now gets
the captive portal address correction too.
* The corrective `nmcli connection up` logs a warning on a non-zero exit;
dnsmasq would otherwise keep the old address silently.
* HotspotStatusResponse.captive_portal defaults to true like everywhere else.
* Trim design narrative from the module docstring and React comments.
* The include-before-activate test now observes the include at the moment
of the `nmcli ... hotspot` call instead of inferring from call order.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: never forward any record type for probe hosts
`address=` only synthesizes A/AAAA answers. iOS also asks for the HTTPS
record type; dnsmasq forwarded that upstream, Apple's answer aliased to
captive.g.aaplimg.com, and the phone fetched the genuine "Success" from
Apple instead of the portal. Each probe host now also gets `local=`, so
dnsmasq answers every type itself, and the CDN alias is a probe host too.
Observed with dnsmasq query logging on a hotspot with an eth0 uplink.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: let acknowledged clients finish so the OS offers "Done"
A phone's sign-in sheet only shows "Done" once the OS re-probes and gets
the genuine success reply. The Controller now remembers each client (by
hotspot IP) that has loaded the portal page and answers its later probes
with the reply each OS expects (Apple "Success" HTML, Android 204, Windows
"Microsoft Connect Test", ...). The set is cleared when the hotspot
starts so a new session shows the page again.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: acknowledge on Continue, not on view
iOS fetches the portal page in the background before opening the sheet,
then re-probes after the sheet's own navigation. Acknowledging on view
made that re-probe return "Success" as the sheet's content. Phones only
re-check the network (and offer "Done") after a full-page navigation, so
the welcome page now has a Continue button leading to /portal/continue,
which is the only place a client is acknowledged. The Continue page says
to tap Done and links to WROLPi and the Controller.
Also hijack clients3.google.com (older Android probe host).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* help: bump wrolpi-help to the captive portal and Speed Test docs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: comments describe the Continue design; clear acks on stop
Three comments still described acknowledging on view, which c72a7f7b
removed. Acknowledged clients are now also cleared when the hotspot
stops, so a lease reused in a later session sees the portal page.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Speed Test calculator: ping, download, and upload against the API
Users asking whether the hotspot is fast enough, or how many people can
stream video at once, had no objective tool that works offline. This
adds one under Calculators > Tools & Reference.
Backend (wrolpi/speedtest.py, /api/speedtest):
* ping: tiny no-store JSON the browser times for round trip latency.
* download: streams slices of a pre-built random buffer for a clamped
number of seconds (0.1-15). Random so no proxy or compression can
flatter the result; slices so no bytes are generated per request.
* upload: streaming-body handler that reads and discards the body and
reports bytes and elapsed time. Requests over 8 MiB are refused.
* info: the client address Caddy forwarded plus how many tests are in
flight. Sanic now trusts one proxy hop (PROXIES_COUNT = 1) so
request.remote_addr is the browser, not Caddy. Nothing else read
that field before.
There is deliberately no one-test-at-a-time lock. Several people
testing at once is how a user learns how the link shares between
streamers, so the server only counts in-flight tests (a shared_ctx
Value created pre-fork so every worker sees the same number) and the
UI reports how many others ran alongside.
Both Caddyfiles route /api/speedtest/* unbuffered, as the SSE chat
route already is, so throughput is measured live rather than in bursts.
Frontend:
* speedtest.js holds the logic with no React in it (bucketed
throughput, steady-state average that drops the first second, ping
stats, stream-capacity tiers, hotspot address check) so Jest drives
it with fake fetch and streams.
* SpeedTestCalculator runs 20 pings, a 10 s download, and a 10 s upload
with a live figure and rolling trace, then shows Mbps and MB/s, ping
verdict, the client address with a hotspot badge, and a table of how
many 480p..4K streams the download speed supports. Upload is
sequential POSTs of one 4 MiB Blob because Safari cannot stream a
request body. Nothing is saved.
* Sparkline is a new inline-SVG ui component (no chart dependency),
added to /theme-sample and checked in all four themes. A missing
sample is a gap, not a plunge to zero.
The page states that it measures the network path, not the drive that
actually serves video files.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* speedtest: bound uploads in time, close the Start race, address review
Review on #708 found three bugs and some polish:
* Start could be clicked again while the first fetchInfo() was still
pending, because the phase only left `idle` after that await. A
second click made a second AbortController and a second concurrent
run, and Cancel only reached the later one. start() now refuses when
a controller already exists and enters a `starting` phase
synchronously, so the button is Cancel before the first await.
* The upload phase could overrun the advertised 10 s by however long
the in-flight 4 MiB POST took: on a slow hotspot, the very link this
tool exists for, that is tens of seconds. runUpload() now aborts the
in-flight request at the phase deadline (counting only completed
requests) and uses 1 MiB bodies so a few-Mbps link still completes
several inside the phase.
* The server upload sink was capped by size only, so a slow or paused
body could hold a worker until 8 MiB eventually arrived. It now also
enforces MAX_UPLOAD_SECONDS (15 s, matching download) and answers
408 when exceeded. Both caps together are what "cannot hold a
worker" actually requires.
Also: a NaN `seconds` now gets a NaN message instead of "must not be
negative"; the module, Sparkline, and gallery docstrings are trimmed
to purpose; and the upload counter is covered for the mid-stream 413,
the 408, and a broken body.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* docs: browse walks the file_group date index instead of probing every doc
The unfiltered /api/docs/search drove its join from `doc`, so SQLite probed
every doc's file_group row for the sort key before returning the first page.
With 20k docs scattered through a 10 GB database on a spinning disk, the first
request took about a minute cold (0.07s warm).
Use the same CROSS JOIN shape as the archive browse: file_group is the outer
table, SQLite walks file_group_published_datetime_idx and probes doc's covering
unique index on file_group_id. Both sides stay index-only and the scan stops at
the page limit. Measured cold: ~60s -> 0.6s, identical rows.
Filtered searches and the size/title orderings keep the ORM query.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: trim browse comments, stabler plan assertion
Review feedback: drop the benchmark narrative from docstrings, state only the
non-obvious constraint at the call site, and assert the doc covering-index probe
by shape rather than by autoindex name.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* zim: recompute the outdated-Zims flag after startup and on Delete
The Dashboard shows the "Outdated Zim Files" banner from the persisted
`outdated_zims` flag, but the modal lists files from a live scan. The two
disagreed: the flag was restored from the DB on startup and never
recomputed once the outdated file had been removed, so the banner showed
with nothing to delete, and the modal displayed an endless placeholder.
- The startup recompute now runs after the DB is up and configs are
imported, so it overrides the value `init_flags` restored from the DB.
Previously it ran before the DB was up, and the restored value won.
- DELETE /api/zim/outdated always recomputes the flag, even when nothing
was deleted, so clicking Delete clears a stale banner.
- The modal re-scans when opened and says when nothing was found instead
of showing a placeholder.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* zim: enforce restore-then-scan ordering; no Kiwix restart on empty delete
- Startup calls flags.init_flags() then flag_outdated_zim_files() right
after db_up, before the DB config import, so the disk scan always
overwrites the DB-restored value in one local sequence.
- DELETE /api/zim/outdated recomputes the flag in a finally.
- remove_outdated_zim_files skips refresh and the Kiwix restart when no
file was deleted.
- Trim narrative comments to the constraint they enforce.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
COUNT(*) OVER() forced SQLite to visit every matching file_group
before LIMIT, which is a full table scan of the (large) file_group
table — ~22s cold on a Pi for videos ordered by download date.
Count separately and cache for 60s by filters so indexed orders can
stop after one page. Past-the-end pages now report the true total.
Videos/archives/docs pagers keep totalPages across page changes.
* videos: codec download preferences with transcode and strict-fail options
Add video_codecs/audio_codecs ordered preference lists to the video,
channel, and RSS downloaders, alongside the existing resolution
selector. Codec preference is codec-major: a preferred codec at a
lower resolution beats another codec at a higher resolution.
When 'transcode' is enabled, a downloaded file whose streams do not
match the preferences is re-encoded with ffmpeg after download (only
the mismatching stream; targets are extensible maps, currently h264
and aac/opus/mp3). When 'strict_codecs' is enabled and transcode is
not, a download with no matching format fails permanently instead of
falling back.
Empty codec lists produce a byte-identical yt-dlp command to before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* videos: honor strict_codecs when transcode cannot produce the preference
Review fixes:
- strict_codecs is now suppressed only when transcoding can produce a
preferred codec for every non-empty preference list (new
transcode_can_satisfy_codecs); av1-only preferences with transcode
enabled fail under strict instead of silently keeping the file.
- No partial transcode: an unfixable video mismatch blocks an
audio-only conversion (and vice versa); the mismatch falls through
to strict/keep.
- fsync the transcode output before rename, and the directory after,
so power loss on a Pi cannot leave a truncated final file.
- mp3 yt-dlp pattern matches mp4a.6B as well as mp4a.6b.
- ffmpeg maps use 0:V:0 (excludes attached pictures) and 0🅰️0?
(tolerates missing audio).
- Strict toggle is no longer disabled by transcode in the UI; popup
wording corrected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* videos: serialize transcodes machine-wide via shared_ctx lock
Sanic runs multiple worker processes, so an asyncio primitive cannot
serialize transcodes across download workers. A multiprocessing.Lock
in shared_ctx now guarantees only one ffmpeg transcode runs at a time
on the machine; waiting workers poll the lock asynchronously so the
event loop stays responsive while a transcode holds it for hours.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* videos: transcode-lock review fixes
- Document that a crashed lock holder requires an API restart; this is
accepted rather than adding owner-death recovery.
- Move the 'Transcoding' user notification to after the lock is
acquired, so it fires when work begins rather than before a
potentially hours-long wait.
- Soften the lock test docstring to what it measures (in-process
wait/release); trim the acquire docstring to the polling rationale.
- Give transcode_lock its own section header in contexts.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
test_search_videos_does_not_block_event_loop asserted a wall-clock
threshold (echo < 0.4s), which fails on loaded CI machines from
scheduler noise alone. Replace the timing with ordering: the mocked
search parks its thread on an Event, the echo is sent only once the
handler is provably inside the query, and the test asserts the echo
completes while the query is still held open. Events are only waited
on off the event loop, so a regression fails the ordering assertion
instead of deadlocking.
Verified both directions: passes repeatedly against the real endpoint,
and fails when the endpoint is changed to run the search on the loop.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* files: left-align the preview toolbar, Open joins the button row
The modal actions row packs its children to the right, which suits a
confirming button but stranded the preview's toolbar. The toolbar now
fills the row so its buttons read from the left, with Open first in
the same row as Download. On phones the tags selector always takes its
own line instead of squeezing in beside the buttons.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* tags: the add-tag button is violet like every other tag button
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* share: a real button everywhere but the navbar corner
The share trigger was a bare anchor-wrapped glyph, which matched the
navbar's other corner icons but looked out of place beside real buttons
in the file preview toolbar. It is an IconButton by default now; the
navbar keeps the bare glyph via asIcon.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* files: readable text previews in dark themes
A same-origin text document inherits the page's color-scheme, so dark
themes rendered the browser's default light text on the preview's
forced white background. Force a light color-scheme on the preview
iframe so text is always black on white.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* files: text previews follow the theme
Instead of pinning the preview iframe to a light scheme, follow the
theme: dark, night, and amber render the document light-on-dark against
the theme's panel color; light stays black-on-white.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Uploaded singlefiles are base64-encoded into JSON before being sent to the
archive container, so an 88MB upload becomes a ~118MB request. Sanic's default
REQUEST_MAX_SIZE of 100MB rejected it with a 413 before the handler ran, and the
API surfaced this as KeyError: 'singlefile'.
Only the Docker deployment is affected; Debian/RPi extract in-process.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Completed once-downloads are never written to the downloads config, and
importing the config (which runs on every startup) deleted any DB Download
the config did not list. That erased the last_download_attempt rows the
per-domain and global daily limits count from, so a restart reset the count.
Keep completed once-downloads on import; pending/failed and recurring
downloads still follow the config as the source of truth.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Fix API stalls: non-blocking switch worker, threaded searches, indexed Videos page
Three response-time fixes found by benchmarking the read-only API endpoints
hot and cold on real libraries:
* `switch_worker` runs on the Sanic worker's event loop every 0.1s and called
the blocking `multiprocessing.Event.wait(timeout=1)`. With no switch pending
(the normal state) this froze the loop for a second at a time; about a third
of all requests, including trivial ones, took ~1,000 ms. It now polls
`is_set()` and returns immediately.
* The search handlers (videos, files, archives, docs, zims) and `/api/statistics`
ran synchronous SQLite queries on the event loop, so one slow search stalled
every other request on that worker. They now run via `asyncio.to_thread`.
* The unfiltered Videos page (`POST /api/videos/search` with only an order)
used `mimetype LIKE 'video/%' OR ...` plus `COUNT(*) OVER()`, which visits
every video's index entry before LIMIT applies: seconds cold on a large
library on a spinning disk, and ~10 s hot for a deep page. It now seeks each
exact mimetype on the (mimetype, effective_datetime) index for `limit + offset`
rows and merges them, with the total from a separate count cached for 60 s.
Results are identical to the generic query.
Also adds `scripts/api_perf.py` (times every read-only endpoint against a host,
optionally restarting the API over SSH for cold numbers) and
`scripts/api_perf_compare.py` (merges per-host results). Host profiles live in a
gitignored JSON file; an example is included.
Measured on an 8-core docker host with a 10 GB library: median hot request
1,007 ms -> 22 ms; Videos page 1 hot 287 -> 108 ms, page 50 1,345 -> 120 ms.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Serve the video description from its own endpoint instead of Video.__json__
`Video.__json__` computed `description` as `c_text or get_video_description()`,
where the fallback opens and parses the video's info json from disk. The list
endpoints serialize every result with `__json__`, so any page containing videos
without an indexed description (common: many videos have none) read their info
json files on every request, cold, for a field the list never displays. The
description is not part of search either: search runs against the FTS copy of
`c_text`, so the fallback never fed the index.
Drop `description` from `__json__` and serve it from
`GET /api/videos/<id>/description`, like comments and captions. The endpoint
returns the indexed `c_text`, falling back to the info json only there, for a
row indexed before descriptions were stored. The player page fetches it through
`useVideoExtras` alongside the comments.
Tests assert the list and detail responses never touch the info json, and cover
the endpoint's indexed, fallback, and not-found cases.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address review: thread-local zim sessions, fast-path equivalence test, comments
* Zim search, single-zim search, and search estimates run libzim off the event
loop, each opening its own session inside the worker thread rather than
passing the request's SQLAlchemy Session across threads.
* The Videos fast-path test now runs the same requests through the generic
query and asserts identical rows and paging for both date orders. The one
intentional difference is documented: past the end, the fast path still
reports the true total.
* switch_worker returns before its try block so the empty path does not clear
the event; its comment states the invariant only. The copied "synchronous
SQLite" comments are reduced to one line in one place.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* videos: a collaboration video's channel directory conflict killed the download
YouTube collaboration videos appear in every collaborator's feed but their
metadata names only the primary uploader's channel. When that channel's
computed directory already belonged to an existing Channel that no longer
matched by source_id/url/name, channel creation raised
ChannelDirectoryConflict and the download failed unrecoverably. Now the
channel that owns the directory is used instead of creating a conflicting one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: trim comments that restated the code
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
NAME_PARSER's suffix whitelist had no audio suffixes (and no mkv/m4v),
so a correctly-named audio file got no published date and sorted to the
very end of its Channel -- which reads as missing. The whitelist now
carries the media suffixes a Video FileGroup can lead with.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* videos: an uploaded audio file never appeared in its Channel
The upload flow asks Video.can_model, which only accepted video/*, so an
uploaded .flac was indexed with no Video row -- and the batch modeler's
`indexed != True` gate then excluded it on every later refresh, leaving
the file permanently invisible in its Channel.
can_model now accepts audio/* like the batch modeler always has, and the
modeler gates on the Video row being missing (mirroring the doc_modeler
fix for the same class of bug) so already-stuck files heal on the next
refresh. Audio playlists (.m3u detects as audio/x-mpegurl) are excluded
from both paths and unmodeled by video_cleanup.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: widen the playlist denylist, prove modeler termination
The denylist gains audio/x-scpls and audio/scpls -- .pls detection
varies by libmagic version, and any audio/* playlist type must be
refused wherever it comes back. Modeling one file is extracted as
_model_video so a test can force persistent failure and prove the loop
terminates; failed_ids is recorded before the PYTEST re-raise so no
failure path can re-select an id forever. Playlist mimetypes are bound
through json_each like the rest of video_cleanup, and the comments
state the invariants instead of the history.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: the playlist test must not depend on libmagic's m3u opinion
CI's libmagic detects .m3u as text/plain where a Mac's says
audio/x-mpegurl. The invariant is that a playlist mimetype is never
modeled by either path, so the test forces the mimetype like the other
denylist tests instead of asserting what detection returned.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* common: config writes must be atomic and version-checked under the save lock
A config file must never be observable in a truncated state: writes go to a
temporary file in the same directory and are renamed into place. The version
check in ConfigFile.save() reads the config file, so it must hold
config_save_lock like the write does.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* collections: item writes must begin as SQLite writers
The add/remove/reorder item endpoints read before their first write; a
deferred transaction upgrading at that write fails instantly with "database
is locked" when any other connection wrote in between, bypassing
busy_timeout. Open their sessions with get_db_session(commit=True) so the
transaction takes the write lock at BEGIN, matching post_download.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: error responses must not commit the write session; trim comments
Map exceptions to 4xx outside the get_db_session(commit=True) block so an
error response never commits, and serialize inside the session. Close the
mkstemp descriptor when fdopen never took ownership. Trim comments and test
docstrings to what the code and assertions actually cover.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* app: the navbar corner wrapped onto a second row on small phones
Every status-icon slot carried a 1.5em separator margin -- ~80px across
three slots -- which wrapped the mobile bar's whole corner onto a second
row on any screen narrower than about 380px. The margin is now 0.25em on
both bars: the corner's gap and the slots' own padding still space the
icons, and on desktop the freed room just goes back to the links that
useOverflowNav folds into "More".
The mobile bar is extracted as MobileNav (mirroring DesktopNav) so a
Cypress suite can mount it and hold it to one row at phone widths.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: prove 320px with a faithful fixture, trim the CSS narration
The suite claimed 360px covered narrower devices because the fixture was
wider than production, but the fixture also lacked the real theme
toggle's 0.8em right margin, so the safety margin was overstated. The
fixture now matches a phone -- 32px logo home link, the toggle's margin,
and a coarse pointer so the search link drops its keyboard hint, with a
test asserting the hint is really absent -- and 320px is measured
directly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The channel downloader refreshes a Channel's files before downloading,
already passing send_events=False, but the parameter was never plumbed
into the file worker, so every pre-download refresh notified the user.
FileTask now carries send_events and handle_refresh skips the
Refreshing/Refreshed events when it is False. User-requested refreshes
and global refreshes still send Events.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* e2e: exercise the FileBrowser footer buttons against real files
Sixteen Cypress tests covering Rename, Move, Delete (including the
tagged-file force-delete flow), Ignore/Unignore, Upload, and scoped
Refresh, with no API mocking: fb:* tasks seed a cypress-fb/ sandbox
inside the media directory, the UI mutates it through the real API, and
assertions check the disk as well as the table. Skipped in CI, which
runs only the React dev server.
Background jobs (move, refresh) only queue work and worker_status reads
idle while jobs wait, so cy.refreshFiles / cy.waitForEvent poll the
events feed for completion events newer than the server's clock, and
fail loudly when one never arrives. Every suite cleans up
unconditionally: sandbox files, DB rows via a final refresh, test tags
and their tags/ directories, and any leaked ignored-directory config.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* e2e: act on review — drain stragglers, teardown that cannot be skipped
Events carry no job id, so waitForEvent cannot prove WHICH job finished.
The rename tests now drain the parent-directory refresh their rename
queues before ending, so the next test's refresh wait cannot be
satisfied by a straggler while the DB still reflects the old tree; move
waits match the destination in the completion message.
Teardown must always reach the disk wipe: a failed command aborts the
rest of its hook, so the tag cleanup and the DB-purge refresh are now
best-effort (failOnStatusCode: false, no assertions), and the unignore
runs after every test instead of once per spec. History and
forward-looking product notes trimmed from comments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
perpetual_shutdown is set by the real before_server_stop listener, so any
earlier async_client teardown in the same pytest-xdist worker leaves it
True on the module-global app and the watchdog returns before reclaiming.
Failed under -nauto, passed alone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Starting the file-worker pump in every Sanic worker made 5 processes
race the shared status dict, the busy flag, and the same directories.
Non-owners now run only a reclaim watchdog: they refresh nothing unless
the owner PID is gone or its heartbeat is stale.
Production wait_for_job polls every 0.1s instead of 100 Hz. A failed
rename includes the recorded job error in the 409 body.
A directory rename waits inline on a move job. When that move failed
(stale FileGroup rows colliding on primary_path), handle_move reverted
the files and sent a failure event but left the job pending. wait_for_job
only understands 'complete', so POST /api/files/rename spun until its
300s timeout and the Rename modal never closed.
The same sequence could kill file processing for the life of the process.
perpetual_signal treated any CancelledError as terminal and logged
nothing. Separately, perpetual_tasks_started is a shared Event that
survives the worker that set it: docker auto-reloads and worker crashes
left every later after_server_start as a no-op. Queued refreshes sat
untouched while worker_status reported idle. Restarting the API was the
only recovery.
Mark failed jobs failed and raise immediately from wait_for_job. Catch
unexpected cancellation and reschedule unless the app is actually
stopping. Track the perpetual-loop owner PID and retake the claim when
that process is gone; every Sanic worker still runs the file-worker
pump. Harden transfer_queue against the cross-process Empty race, and
stop request processes stealing from the shared public queue.
* upgrade: restart services when the upgrade fails before code changes
The upgrade script stops wrolpi-api and wrolpi-app before fetching, so a
failed fetch, signature verification, or checkout left the WROLPi down
with unchanged code on disk. The EXIT trap now restarts both services
for failures that occur before scripts/upgrade.sh begins modifying the
system. The post-checkout verification failure rolls the branch back to
the previously-running commit first, so services never restart on an
unverified commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* upgrade: only restart services when rollback lands on the known-good commit
Review fixes: a failed rollback no longer leaves the restart flag set, so
services cannot start on an unverified or mixed tree; the non-atomic
checkout failure path now rolls back too; trimmed narrative comments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* downloads: a batch of failed downloads that retried in lockstep
`calculate_next_download` gives every deferred download the same 3^attempts
backoff, measured from the moment it failed. Downloads that fail together
therefore retry together, and a synchronized burst against one host is what
gets a whole batch rejected -- which defers them all again, on the same
schedule. The backoff grows, but the batch never breaks up.
Jitter each retry forward by up to half its backoff. The backoff stays a
floor, so nothing is retried early; a recurring download is still capped at
its own frequency, so nothing is retried later than it was before.
Deferred scheduling is no longer an exact value, so the tests that pinned it
assert a floor and a ceiling instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* review: comments that narrated the design instead of the code
Trim the jitter comment to a single WHY and the test docstring to its summary
line. The reasoning that led here belongs in the PR, not beside the
arithmetic.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* downloads: a deferred once-download that nothing ever retried
Deferring a download only schedules it: `defer()` sets the status and
`calculate_next_download` sets `next_download`. Acting on that schedule was
`renew_recurring_downloads`, which filters on `frequency != None` -- so only
recurring downloads were ever marked "new" again.
A once-download has no frequency. One transient error (an HTTP 403 from the
host, a dropped connection) deferred it with a `next_download` that nothing
read, and it stayed deferred until someone retried it by hand.
`delete_old_once_downloads` did not clear it either -- that requires a
`last_successful_download`, which a download that never succeeded does not
have. So every transient failure added one more permanently-stuck row to the
downloads list.
Add `renew_deferred_once_downloads`, called from `do_downloads` alongside the
recurring renewal. `attempts` is preserved, so `calculate_next_download`'s
3^attempts backoff keeps growing across retries: a URL that never succeeds
backs off toward never instead of retrying at a fixed interval.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* review: comments that narrated the bug instead of the contract
The new docstrings restated why the bug existed and how the fix was reasoned
about. That history belongs in the PR, not in code that has to stay true.
Keep the invariant that a reader needs: attempts are not reset, so the backoff
grows. In the test, the assertions already document the contract.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ui: a download form that emptied itself after every submit
A successful submit reset the whole form, so adding a second download meant
re-choosing the tags, the destination and the video settings that the first
one had already been given.
It now empties only the URLs. Everything else stays for the next download,
and a Clear button beside Download puts the form back to its defaults when
the next one is unrelated.
`useForm` gains `clearPathsOnSuccess`; `clearOnSuccess` is untouched for the
forms that want the whole reset.
* review: the one download form that kept its URLs
Scrape took a `clearOnSuccess` prop that no caller passed, so mapping it to
the URL paths left create-mode scrape clearing nothing. It follows the same
rule as Videos and Archive now: the edit form supplies `onSuccess`, and only
that keeps the form as it was.
`clearPaths` also wrote every path it was given, so a `urls` form gained a
`url: ''` it never declared -- which the comment beside it and the test named
for it both said it would not do. It skips a path neither the defaults nor
the form has, and the test asserts the key is absent.
* ui: two upload dialogs that had drifted apart
The dashboard and the file browser each had their own upload component: two
dropzones, two overwrite toggles, two copies of the progress-bar rendering,
against the same `useUploadFile` state. The file browser's dialog was made
fullscreen with a drop target that grows; the dashboard's stayed a `large`
dialog with a strip to drop onto, because it was a different file.
They differ in one thing that is real: the file browser knows the destination
because the user selected it, and the dashboard has to ask. That is now the
`destination` prop -- `undefined` asks, a string uploads there, and '' is the
media directory rather than a missing destination.
Everything else is one component, so the next change to either entry point
lands on both.
* review: an upload dialog that looped on the media directory
Pressing Upload in the file browser with nothing selected passes `''` -- the
media directory. Three checks along that path read it as "no destination":
* The clearing effect ran, and `doClear` is a new function every render whose
own setState scheduled the next one. React ended it with "Maximum update
depth exceeded", 16 warnings deep, on the default Upload path. The effect
now clears only while asking, and `doClear` is stable.
* The title read `Upload to: /media/wrolpi/`.
* `doUpload` bailed, so a drop on the media root uploaded nothing.
All three predate this branch; moving the dialog is what put them behind one
component. Tests cover the title, what the hook is told, and that a fixed
destination is never the thing that empties the queue.
The upload dialog was 682x367 on a 1440x900 desktop -- 47% of the width, 41% of the
height -- and the drop target inside it was a strip about 90px tall. A small thing to
drag a file onto, with the file browser it came from covering most of the screen behind.
Upload and Move ask for `fullScreen`, which is edge to edge on a phone and 90% above the
tablet breakpoint: filling a desktop reads as the app having been replaced rather than as
something opened over it. Move holds a DirectorySearch above a row per path being moved,
which grows with the selection; 620px showed a handful and scrolled the rest.
Two things had to follow. Mantine sizes a fullscreen dialog's content to the viewport but
leaves the body at its content height, so the footer landed two thirds up the screen with
a quarter of the viewport empty beneath it -- the body takes the leftover now and the
actions ride to the bottom of it. And the drop target itself grows into the room, with a
floor so it stays large once progress bars claim their own space.
Rename and the confirmations keep their sizes: a fullscreen dialog for one text field or
one sentence reads as broken. New Folder keeps `large` too -- it is a text field and a
path preview, the same shape as rename, not the directory picker its name suggests.
`size='fullscreen'` is a DIFFERENT thing -- full width, height from the content, which is
what a log or a preview wants. Both are documented where the sizes are declared, and the
gallery grows a fullscreen example so the difference is visible in all four themes.
339 component tests pass, 69 suites / 1214 jest tests pass, npm run build is clean.
* review: the 375px phone this fix had not been measured on
Answers the five findings on #660.
* Recurring's four fixed columns wanted 416px between them, so at 375px the URL
column got exactly 0 and the table overflowed again -- the same bug as before,
moved down from 540px. Widths are phone-first now, in col-* classes, widening
at the tablet breakpoint.
* The in-progress button in Completed At carried "5 MBps / 3m44s" -- 169-188px
measured, against 124px of column. It shows the percent; the modal it opens
already lists speed, ETA and bytes, and the full text is on the button's title.
* The select column was 2.5em: 16.5px of content for a 22px checkbox, which hung
into the URL column. 3em holds the checkbox and its padding.
* The Cypress fixtures were a raw <table>, so tableLayout would read 'fixed' even
with the selector that loses to Mantine -- the test could not fail for the reason
it claimed. They mount SortableTable, Table.HeaderCell onSort and a real
Checkbox now, and the header check measures the text rather than skipping cells
with no element child.
* Comments trimmed to what cannot be recovered from the code.
Fixing the first surfaced one more: once URL became the small column, its own header
overran -- "URL" and its arrow need 72px against 58 -- and a single word cannot wrap.
A header is clipped by its column as a last resort.
Measured at 375px: once 46/58/123/146, recurring 44/100/104/57/69, nothing
overflowing. 69 suites / 1214 jest tests pass.
The Cypress tests here are NOT verified: the component runner does not start in this
repo (see the separate cypress investigation), so every layout claim above rests on
direct measurement in Chrome, not on that suite.
* review: the 375px phone this fix had not been measured on
Answers the five findings on #660, and now runs against a component suite that can
actually execute (#661).
* Recurring's four fixed columns wanted 416px between them, so at 375px the URL got
exactly 0 and the table overflowed -- the same bug as before, moved down from 540px.
Widths are phone-first now, in col-* classes, widening at the tablet breakpoint.
* The in-progress button in Completed At carried "5 MBps / 3m44s", 169-188px against
124px of column. It shows the percent; the modal it opens already lists speed, ETA
and bytes, and the full text is on the button's title.
* The select column was 2.5em: 16.5px of content for a 22px checkbox, which hung into
the URL column. 3em holds the checkbox and its padding.
* The fixtures were a raw <table>, so tableLayout read 'fixed' even with the selector
that loses to Mantine -- the test could not fail for the reason it claimed. They
mount SortableTable, Table.HeaderCell onSort and a real Checkbox now.
* Comments trimmed to what cannot be recovered from the code.
Two defects the restored suite caught, neither of them findable before it ran:
* The elastic URL column ends up narrower than its own header -- "URL" and its arrow
need 72px against 58 -- and one word cannot wrap, so the column clips. A sized column
must fit its label; only the elastic one may clip. That is what the header test
asserts, per kind of column, rather than measuring text and calling it the label: the
first version measured the text alone, missed the sort arrow, and passed while the
button overran by 14px.
* Recurring's Control column was 4.5em, 70px of content for a 78px "Control". Sized
for the word rather than the button it holds: 5.2em.
At 375px: once 46/58/123/146, recurring 33/100/104/57/80, nothing overflowing and no
sized header clipped. The URL column there is 33px -- the accepted trade for keeping
every column on a phone.
336 component tests pass, 69 suites / 1214 jest tests pass, npm run build is clean.
* review: a header assertion that stopped at the padding box
Three findings on #662.
* The sized-header check compared scrollWidth to clientWidth, which is the padding box,
so a label could overflow its content box into the padding and pass -- the same class
of miss the 4.5em Control failure was meant to close. It measures the label's longest
word (plus the sort arrow, where there is one) against the content box now.
That found two more columns tuned to their exact limit and clipping by a pixel:
Download Frequency wanted 79.4px of a 78.1px box, Completed At 82.3 of 81.9. Every
sized column now has margin over its own label.
* SortableTable stopped applying `width`, leaving dead keys at the Tags and Zim call
sites. Removed, so a column is sized one way. CollectionTable's `width` is NOT dead
-- it converts sixteenths to percentages -- so Channels, Docs, Playlists and Archive
keep theirs.
* Comments trimmed again: the percent button, the header clip, the ButtonGroup rule and
the Cypress helper.
336 component tests pass, 69 suites / 1214 jest tests pass.
* ui: a phone header short enough to leave room for the URL
Five full-length headers do not fit a phone. Sized so each held its own label, the
recurring URL column came out 20px -- narrower than one character and its ellipsis, a
column present but carrying nothing.
A long header carries both forms and `.label-*` shows one: Freq and Done below the tablet
breakpoint, Download Frequency and Completed At above it, where there is room. The mobile
widths are sized for the short word.
At 375px the URL column goes from 20px to 105px on the recurring table and 54px to 97px on
the once table. Nothing overflows, nothing clips, and every column stays.
336 component tests pass, 69 suites / 1214 jest tests pass, npm run build is clean.
* test: the component suite has not run since cypress 15, on any machine
Six component spec files -- 334 tests, including the 4,300-line layout suite -- could not
start. Cypress exited before printing anything, and under its default Electron browser it
did not even do that: it hung at 0% CPU until killed. Chrome prints the error in seconds,
which is the only reason this was findable.
Three things broke, all of them from the 13.17.0 -> 15.20.0 bump.
`baseUrl: "src"`. Cypress 15 loads its config with tsx, and tsx honours tsconfig path
resolution. Cypress requires Node's core `events` module; with that baseUrl the bare
specifier resolves against `src`, and on a case-insensitive filesystem it matches
`src/Events.js`. That file imports the app, so the whole component tree was required in
Node: JSX in `.js` files, which esbuild will not parse.
This is why CI stayed green while every local run died. Linux is case-sensitive, `events`
never matched `Events.js`, and it resolved to the built-in. The baseUrl buys nothing --
there is not one absolute import in the codebase -- so it goes.
`framework: "create-react-app"`. Cypress 14 removed that dev server; the handler is in the
13.17.0 binary and absent from 15. The webpack config is handed over explicitly now, taken
from react-scripts so components compile exactly as `npm start` compiles them. Two things
follow from that: `webpack-dev-server` 5 is a devDependency because Cypress requires 5 and
react-scripts pins 4 (npm nests react-scripts' copy, and the production build is unchanged),
and babel-loader is widened to `cypress/`, which react-scripts does not compile -- the
support file is JSX and was reaching babel with no preset.
The suite ran nowhere. `cypress run` means e2e -- its specPattern is `cypress/e2e/**` --
and these specs sit beside the components in `src`, so no job has ever executed them. A
`cypress-component-tests` job runs them now; without it this breaks again in silence.
One test fails the moment the suite can run, and it is mine, from #660: it asserted the URL
column is the widest in the row. It is not, and never was -- at that width the fixed columns
take ~307px of 400, leaving the URL ~93 against Control's 146. It shipped green because
nothing could run it. It asserts a usable remainder instead.
334 component tests pass, 69 suites / 1214 jest tests pass, `npm run build` is clean.
* review: an assertion with a floor, and comments that told the story instead of the rule
The leftover-width test had a lower bound and nothing else, so it passed on any column
wide enough to have padding -- including one that had grown, which is what losing
table-layout: fixed does. It measures the leftover now: the sized columns are held to
the widths they declare, and the truncating column to exactly what they did not take.
Comments in the config, the CI job and the spec cut back to the rule a reader needs. The
version history and the measurements are in the PR.
The two downloads tables came out 676px and 727px in a 540px window, so Completed At and
Control sat off the right edge and their buttons could not be reached without scrolling
sideways.
`.table-ellipsis` was trying to truncate under `table-layout: auto`, where a table is as
wide as its columns want to be and the container simply overflows. The rule leaned on
`max-width: 0` to force an ellipsis and `min-width: 25em` to keep the text column readable
-- but 25em is 385px, and a floor wider than the screen makes the overflow certain.
Fixed layout inverts it: the columns divide the table's width and the column marked
`column-ellipsis` gives up what it cannot show. Neither the max-width trick nor the floor
is needed then. Columns that must fit something exact -- a checkbox, a group of buttons --
state a width; the rest is the truncating column's.
`table.table-ellipsis`, not the bare class: Mantine sets `table-layout` on its own
`.mantine-Table-table`, the same specificity and later in the cascade, so the first version
of this lost silently. Every other property in the rule applied, which made the table look
fixed while it stayed auto -- the recurring table got *wider* than before, not narrower.
Three things the rework then had to answer for, each of them measured in the browser:
* A header overran its column: "Completed At" printed on top of "Control", because
`.wrolpi-th-sort` is nowrap so a label never splits from its arrow. It wraps here, and
needs `min-width: 0` as well -- the label is an anonymous flex item whose automatic
minimum is its longest word, so without it the label still refuses to go below
"Completed" and still overruns.
* A grouped button was squeezed: `ButtonGroup` is a nowrap flex row whose children shrink,
and the Stop button on a pending download came out 23px against its neighbour's 40 --
clipped, and too small to hit. The columns are sized for the widest group a row can
have, so the buttons no longer give.
* Status's Top Processes table had the same bug. It already asked for `.table-ellipsis`
and already marked its Command cell, so it overflowed a phone too (548px into 503px).
The same rule fixes it, with widths for the three short readings.
A wrapped header is the fallback, not the layout, so every column is sized to hold its own
label on one line -- including on a phone. "Download Frequency" is the exception at 11.7em,
which a phone cannot spare beside four other columns, so it takes a class that widens at the
tablet breakpoint and wraps below it.
Measured, nothing overflowing, no header overrun, no clipped cell, no shrunken button:
540px once 40 / 208 / 146 / 146 recurring 123 / 108 / 131 / 62 / 116
1400px once 40 / 1033 / 146 / 146 recurring 871 / 185 / 131 / 62 / 116
Desktop gains from it as well: the checkbox column drops from 90px to 40 and Completed At
from 291 to 146, so the URL column picks up about 255px.
Covered by five tests in the Cypress layout suite. None of this is checkable in jsdom --
it is all resolved widths, and the central question is whether our `table-layout` beats the
one Mantine sets, which is exactly the part that failed silently.
69 suites / 1214 jest tests pass.
* ui: a preview map that waited for a container it could never see
Both map preview modals -- the per-region eye button and "View All Regions"
-- opened to an empty box. Each renders a div into a Modal and hands it to
MapLibre from an effect gated on `open`, reading the node out of a `useRef`:
if (!open || !mapContainer.current) return;
Mantine's Modal mounts its body one commit after `opened` flips true, so on
the render that opens the modal the ref is still null. The effect returns,
nothing it depends on changes again, and it never runs a second time. No
console error, no rejected promise, no canvas -- just an empty div.
Semantic UI's Portal mounted the body in the same commit that opened it, so
the ref was attached by the time effects ran. The effect bodies are
byte-identical across that migration; the modal library is the only variable,
confirmed against a device still running the older build where both previews
render.
A callback ref instead, so the effect re-runs when the node arrives and this
stops depending on any modal's mounting schedule. The async source lookup
also gets a cancelled flag: it could resolve after the cleanup ran, building
a map nothing would ever remove and leaking a WebGL context per open.
The tests drive the real page and assert a map was built against a node that
is in the document -- the outcome, rather than the mechanism that quietly
changed underneath it. Seven of the eight fail on the code before this.
* build: docker had no world underneath its map regions
Caddy serves /blobs/map-overview.pmtiles out of /opt/wrolpi-blobs, and that
blob is the world at zoom 0-6 -- what a regional extract sits on top of.
Without it a user who subscribes to a few regions gets those regions floating
on an empty background everywhere their data does not reach. Only a full
planet file, at 134GB, makes it unnecessary.
Raspberry Pi and Debian installs get the blob from pi-gen, the live-build
config, and upgrade.sh. Docker got nothing: the compose file bind-mounted a
git-ignored host directory over /opt/wrolpi-blobs, and a bind mount replaces
the image's directory rather than merging with it, so the path existed and was
empty on every Docker WROLPi. Verified against a running deployment, where
that URL returned 404 while the same URL on hardware returned 206.
So the web image fetches the blob at build time, as the other three builders
do, and the mount that would hide it is gone. `curl -f` so an HTTP error
fails the layer -- an image that builds without the blob is this same bug,
except silent at build time too.
The tests read the deployment files rather than a built container: building
to check costs a CDN download and several minutes, and both things that broke
-- nothing fetches it, a mount hides it -- are visible in the source.
* ui: cancel the load handler too, and make the cancel test falsifiable
Review follow-ups on the preview modals.
The `cancelled` flag was checked before constructing the map but not inside
the `load` handler it installs, so cleanup landing between the two left that
body drawing into a map `remove()` had already torn down. Guarded, mirroring
the `destroyed` pattern in MapViewer.js. The source promise also gets a
`.catch`: the helper cannot reject today, but nothing says it never will, and
an unhandled rejection here would be as quiet as the bug this all started as.
Two test corrections, both cases of an assertion claiming more than it read:
The region-drawing test spied on `addSource` AFTER construction and then
re-fired `load` by hand, so its count came entirely from that manual call and
said nothing about the production path. FakeMap now records the sources it is
given, and the assertion reads what the component's own `load` drew -- each
region once, in order, and none for the entry with no bbox.
The new cancel test passed with the guard deleted, which is worse than not
having it: Mantine had not mounted the modal body when it closed, so the
effect had never run and there was nothing to cancel. It now waits for the
effect to reach the source lookup before closing, and ships beside a control
that releases the same held promise WITHOUT closing and requires a map. The
control is what makes the zero meaningful; verified that removing the guard
fails the cancel test while the control still passes.
`IconButton` resolved `variant={props.variant ?? fromRole?.variant ?? 'default'}`, so any
call site that named a color without also naming a role or a variant landed on Mantine's
`default` variant. That variant draws its background, hover, text and border from
`--mantine-color-default*` alone and never reads `color`, so the button rendered
byte-identical to one with no props at all. The color was accepted, stored, and dropped.
`Button` was never affected: it leaves `variant` undefined when there is no role and
Mantine fills in `filled`, which is why the same `color='violet'` works there. Match it --
`filled` when a color is named, `default` when one is not, which is most of them.
Eleven call sites were passing a color that could not reach a pixel:
DonatePage:19, Files:421 the color the USER picked, thrown away
Downloads:241,404,427,460 danger and blue on the download controls
ControllerPage:326,408,1591 the service Open buttons
FilePreview:71,541
The fallback is last, behind both `variant` and `role`, so `role='danger'` keeps its
dashed night outline even when a color is also named; there is a test for that.
Verified in the browser across all four themes: each icon button now matches the labelled
button of the same color exactly, and an icon button with no color keeps the neutral
default. In night and amber the hues collapse to brightnesses, as those themes intend.
The gallery grows a color pair per hue, so a regression here is visible rather than silent.
68 suites / 1204 tests pass.
On the Control tab, each service's actions -- start/stop, restart, logs -- sit in a
`.wrolpi-button-row`, a wrapping flex row. Open sat outside it, after the logs modal,
so it was a block-level sibling of the row rather than a member of it: it landed on its
own line under the others, on both the mobile and the desktop row.
Move it inside the row, after logs. Nothing else changes -- same icon, same link, same
order left to right -- and the row already wraps, so a narrow Actions column breaks it
where it needs to instead of hard-coding a break at Open.
The two row components are now exported so the test can mount them: `Media` renders
neither branch under jsdom, where no media query matches, so `ServicesSection` yields no
rows there at all and the buttons were untestable through the page. The test asserts
that Open shares a `.wrolpi-button-row` with stop, restart and logs, and it fails on the
old layout.
68 suites / 1202 tests pass; `npm run build` is clean.
* ui: a PDF asks before it lights up a night-adapted eye
Night and amber pass media through an SVG color matrix so nothing a user opens costs
them their dark adaptation. It reaches everything drawn in our own document -- images,
video, canvases, and same-origin iframes like the EPUB viewer.
It does not reach a PDF, and the miss is not subtle. Chrome draws PDFs in its own
out-of-process viewer, which the compositor paints outside the parent's filter: the
computed style on the iframe really does say `filter: url(#wrolpi-night-red)` and the
page still arrives in full color. Measured in night mode on a device, a PDF covered
55% of the doc page in non-red pixels -- a full-brightness photograph of a blue sky --
while the chrome around it was correctly red. The preview modal measured 39.8%.
Tinting from outside was tried and does not work. A red `mix-blend-mode: multiply`
overlay above the iframe cannot see the plugin's pixels, so it stops blending and paints
flat opaque red: zero non-red pixels by measurement, document entirely obliterated.
So the PDF is withheld and the reader chooses. Nothing renders until they do; hiding it
with CSS would still load and paint the viewer, which is the flash being avoided.
Gated on whether the theme is CURRENTLY filtering, not on its name. A reader who turns
night's filter off has said they do not want this, and amber with its filter on has the
same unreachable PDF and does. Only PDFs: text and the EPUB viewer are documents of our
own and tint correctly, so gating them would hide something for no reason.
The gallery paragraph claiming the matrix remaps PDFs is corrected -- it was the one
place we told users something this measurement disproves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* review: the reveal that outlived its document, and a test that proved nothing
Five review comments, all correct.
The bug: MediaGate was not keyed in FilePreview. The effect that rebuilds a preview sets
the modal to null and then to the new content in one pass, so React 18 batches both and
reconciles instead of unmounting -- the fiber survives, and a reader who revealed PDF A
would get PDF B in full color with no prompt. Keyed by url, which is what Docs already
did with activePath.
The test named "re-gates when filtering is turned back on" never revealed anything first,
so it went from gated=false to gated=true with `revealed` already false. That path never
reaches the reset effect: the test passed with the effect deleted. It now clicks Show
first, and fails without the effect.
The hide control moved above the media. A preview fills `calc(100dvh - 18.75rem)`, a
height calibrated with no room for a trailing button row, so a control after it lands
below the fold of a full-screen modal -- leaving the reader who just lit up their eyes
scrolling to find the way back.
Docs' embed -- the surface measured at 55% non-red -- had no contract test where
FilePreview had one. It has one now, asserting the gate wraps the iframe, keys per
document, and reads mediaFilterEnabled rather than the theme's name.
Gallery copy said "night and amber withhold it"; amber's filter is off unless asked for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test: do not leave videos_destination as an invalid template
test_format_videos_destination deliberately sets %(channel)s to prove
unknown placeholders fail, then never restored the global WROLPi config.
On xdist workers that later ran channel_factory without async_client
re-init, the polluted template caused KeyError/FileNotFoundError in
test_format_video_filename_uses_channel_name_over_info_json.
* test: stop search_suggestions from flaking empty domains on CI
A different failure than videos_destination pollution: under xdist load,
search_suggestions sometimes returned no domains for a valid query while
channels still matched. Harden the path by:
- running channel/domain/author/subject searches sequentially on the shared
Session (asyncio.gather is not safe with one SQLAlchemy session)
- giving archive_factory a URL when only a domain is set so validate() does
not clear collection_id via apply_domain()
- rewriting test_search_suggestions without hardcoded Collection ids and with
unique names that will not collide with other tests
Every Python job started failing in "Install Requirements" with
Error: [Errno 2] No such file or directory: '/home/circleci/project/venv/bin/python3'
on branches whose only changes were under app/. The same config passed two days
earlier, so this is the environment moving, not the code.
The cache key names the minor version (`deps-3.13-...`) but not the patch, and the
cached paths include the whole `venv`. A venv's bin/python3 points at the exact
interpreter that built it, so once cimg/python:3.13 takes a patch bump the restored
venv refers to an interpreter the image no longer has, and creating a venv over that
directory fails instead of repairing it.
Two changes, both needed:
- Rebuild the venv when its interpreter cannot run. This heals the restored cache
whatever left it unusable, and costs a reinstall only on the first run after a bump.
- Move the cache key (-5 -> -6). CircleCI keys are immutable and `save_cache` will
not overwrite one, so without this the poisoned entry is restored and discarded on
every run forever and is never replaced.
Not reproduced locally: macOS python recreates the symlink and exits 0, so the exact
crash is CI's Linux python. The guard does not depend on the mechanism -- it removes
any venv whose bin/python3 will not run, which is the state the error names. Verified
that the guard heals a venv with a dangling interpreter and keeps a healthy one, so the
cache still pays off.
Unrelated and still open: api-tests fails on master itself, in
test_format_video_filename_uses_channel_name_over_info_json, which flakes under -nauto
and passes alone.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
