`loading` only became true once the debounce timer fired, so for the
first 300-500ms after every sendRequest the hook reported
data=null, loading=false. Consumers branch on `loading` to choose
between a spinner and an empty state, so the global search dropdown,
the Zim search tab, and the map search rendered "No results" (or
"No Zims have been indexed") for a search that had not run yet.
Invariants now enforced and tested:
* loading is true from sendRequest until the latest request settles.
* The latest-request id is claimed synchronously in sendRequest, so a
fetch already in flight is stale the moment a newer request is sent
and cannot write data or clear loading during the new debounce.
* A stale request that rejects late no longer nulls newer data.
* A pending debounce is cancelled on unmount, and any in-flight fetch
is treated as stale.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* startup: run database config imports off the worker event loop
import_all_db_configs runs in a Sanic worker's after_server_start
listener, after that worker's socket is already accepting connections.
Every import in it was a synchronous call on the event loop, so a large
tags/channels/domains/downloads config froze that worker for the whole
import and every request the kernel routed to it hung. With the default
worker count equal to CPU count, a one- or two-core host lost all or
half of the API for minutes on startup, and the frontend's 60s request
limit turned that into repeated silent retries.
Each synchronous import now runs in asyncio.to_thread. The SQLite engine
already allows cross-thread connections (check_same_thread off, NullPool)
and other startup work already uses to_thread. Imports still run one at
a time in the documented order; only the loop is no longer blocked.
Test: a ticker task must keep ticking while two deliberately slow
synchronous imports run. It scored zero ticks before this change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* startup: thread the downloads import too; test covers every import
import_downloads_config is async but never awaited anything: its body
called the synchronous DownloadManagerConfig.import_config on the event
loop, so a large download_manager.yaml still froze the worker. The
db_up check stays on the loop and only the import moves to a thread.
The responsiveness test now patches every import the startup path
performs, including that sync downloads import, with the same blocking
sleep, and requires ~90% of the total sleep budget in ticks so any one
import returning to the loop fails it. The ticker is awaited after
cancel so the task does not outlive the loop. Docstring trimmed to the
one constraint not visible at the call site.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* files: shared modeler batch loop for video, doc, zim, archive
Paging, skip-ids, progress, and cancel yields live in
wrolpi.files.modeler.run_modeler_loop. Each module still owns select /
prepare / apply and its transaction style: video probes outside the write
lock, docs use one write per file with lock retries, zim and archive use
one write per page. Archive HTML that is not a SingleFile raises
SkipModeler so apply_indexers can claim it.
* files: quieter batch-modeler errors; drop unused imports
Break out of a batch page on InvalidRequestError after a flush failure
so the rest of the page does not each log the same poisoned session.
Re-raise PYTEST apply failures without a misleading commit-failure
message. Remove asyncio/List/OperationalError/PYTEST/Iterable leftovers
from docs, archive, and zim.
FileWorker stays the queue, status, and refresh apply path. Move planning
and execute/revert live in wrolpi.files.move; collection reorganize jobs
live in wrolpi.files.reorganize_job. Perpetual-process dispatch is
unchanged.
Existing `from wrolpi.files.worker import build_move_plan_bulk` still works.
* api: run the perpetual loops in one Sanic-managed process
The singleton background loops (file worker, downloads, switches, jobs,
bulk tag, flag checks, update check) now run in a dedicated process that
Sanic's Worker Manager owns: `run_perpetual_process` is registered with
`app.manager.manage()` from `main_process_ready`. The manager guarantees
a single instance and SIGTERMs it on shutdown. No server worker runs the
loops, so the owner-pid / heartbeat / watchdog claim scheme is deleted
rather than hardened.
`perpetual_signal` keeps its name and is now a plain registrar. A new
`per_worker_task` registrar covers loops every server worker must run for
itself: syncing its log level (previously only the owner did, so the other
workers never picked up level changes) and a health check that asks the
manager to restart the perpetual process when it is FAILED (Sanic marks a
dead managed process but does not restart it).
Downloads are started as background tasks instead of Sanic signals; the
perpetual process has no server, so no finalized signal router. The
update check and the upgrade-check endpoint run git in a thread via
`refresh_update_status`. /api/status lists the perpetual process next
to the servers.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perpetual: survive a predecessor's death, exit cleanly, coordinate restarts
Review fixes for the managed perpetual process.
Shutdown: cancel_background_tasks / cancel_refresh_tasks gather with
return_exceptions, so a cancelled child's CancelledError no longer aborts
the shutdown that called them (the process exited 1 with a download in
flight).
Restart after a crash: reconcile_after_predecessor heals shared locks a
dead process left held, fails RUNNING Jobs owned by a dead pid, clears
processing_domains, renews pending Downloads, and resets the file-worker
status before the loops start. N server workers issue one restart
request per cooldown through a shared timestamp in shared_ctx.
The log-level sync now also runs in the perpetual process. Manage() is
registered through register_perpetual_process so it is testable; the
entry point refuses to run with no loops registered (spawn).
Tests exercise the real entry point in a forked process with a live
background task under SIGINT and SIGTERM, Sanic's real WorkerManager
marking a crashed managed process FAILED and reviving it on restart,
Sanic's process naming and ProcessState enum, the listener registration,
and the reconcile step.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perpetual: import the jobs helper inside its reconcile step
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Discover/compare (find, TEMP table, SQL diff) now lives in
wrolpi.files.refresh so FileWorker is the queue plus apply/move/reorganize.
Existing `from wrolpi.files.worker import compare_file_groups` still works.
file_group.idempotency is unused since SQL compare. DROP COLUMN on SQLite
3.40 does not rebuild the table, so file_group_fts stays valid.
directory.idempotency is unchanged.
* files: pass find -printf \0 as an escape, not a Python NUL (#716)
* files: pass find -printf \\0 as an escape, not a Python NUL
'%h\\0%f\\0%T@\\0' in a Python string is an embedded null byte. subprocess
cannot pass that as an argument, so every GNU-find refresh on Linux CI
raised ValueError. find wants the two-character escape \\0 in the format
string and emits NULs itself.
* files: serialize stem backfill so a cancelled compare cannot lock SQLite
compare_file_groups runs ensure_file_group_stems in a thread. Cancelling
the task does not stop that thread, and a second compare then writes
wrolpi_kv on the same connection (tests share one session). A lock
plus a version re-check makes the second call a no-op.
* files: unique (directory, stem) after collapsing duplicate FileGroups
713 added stem and a non-unique index. This release backfills any NULL
stems, merges leftover duplicates (tags and collection membership move
onto the winner), and creates a UNIQUE index.
Upsert deletes sidecar FileGroups before INSERT so a new primary cannot
collide. Refresh deletes before upsert for the same reason.
from_paths looks up by stem so adding a sidecar does not insert a second
row. Same-stem files without a modeled primary stay one FileGroup.
* files: do not use the test SQLite connection from a worker thread
get_db_curs in pytest shares the test session connection. Running
ensure_file_group_stems and _upsert_files via asyncio.to_thread then
segfaults when the event loop commits that same connection (CI xdist).
Keep to_thread in production, where get_db_curs opens a dedicated
raw connection.
* files: keep tags when a sidecar joins an unmodeled group
choose_primary_file is the single fallback for unmodeled stem groups so
_upsert_files and _upsert_file_groups cannot pick different primaries
and DELETE the tagged row. FileGroup.move merges an existing
(directory, stem) group and flushes the delete before taking that stem.
refresh_files_directly deletes before upsert, matching the other two
callers.
* files: group by (directory, stem), not stem alone
group_files_by_stem mixed marauder/s3.bin and other/s3.bin into one
FileGroup because it ignored directory. Unique (directory, stem) then
kept only one of them, so flasher search missed a firmware file.
run_command_streaming awaits the stdout reader instead of cancelling it
when the process exits, so the last lines are not lost on a fast CI run.
* jobs: write the exception into the Job log on failure
test_jobs_failure only saw 'boom' in the log when JobLogHandler captured
logger.error(..., exc_info=e). Sanic/pytest dictConfig replaces root
handlers (especially under xdist on CI), so that path is empty even
though status and error are set. Log the exception on the Job itself
and store it in the same _update_job as FAILED.
* files: pass find -printf \\0 as an escape, not a Python NUL
'%h\\0%f\\0%T@\\0' in a Python string is an embedded null byte. subprocess
cannot pass that as an argument, so every GNU-find refresh on Linux CI
raised ValueError. find wants the two-character escape \\0 in the format
string and emits NULs itself.
* files: serialize stem backfill so a cancelled compare cannot lock SQLite
compare_file_groups runs ensure_file_group_stems in a thread. Cancelling
the task does not stop that thread, and a second compare then writes
wrolpi_kv on the same connection (tests share one session). A lock
plus a version re-check makes the second call a no-op.
* files: SQL FileGroup compare, stem column, drop unused discovery path
Refresh walked the tree twice (find, then stat every unchanged group),
and a two-directory refresh scanned the whole media directory. Compare
now diffs in SQL on a TEMP table that already has mtime, accepts multiple
roots, and groups by a stored stem that is versioned so a suffix-rule
change cannot delete-and-reinsert tagged files.
Deletes refresh_discover_paths and /api/files/refresh_progress, which
nothing in production called. Video/doc/zim modelers use the same
(no domain row OR indexed != True) gate.
* files: keep stem current on reorganize; NUL-safe find parse
Reorganize wrote the new primary_path but not stem, so the next refresh
treated the FileGroup as deleted+new and dropped tags. GNU find now
emits NUL-delimited records so a tab or newline in a filename cannot
abort a global refresh. Stem backfill skips the table scan once the
algorithm version matches, and runs off the event loop. An empty roots
list is a global refresh, not invalid SQL.
POST /api/videos/channels failed with "database is locked" on
`INSERT INTO collection` on the docker WROLPi while background workers
(config saves, refreshes) were writing. The handler wrote through the
deferred request session: `create_channel` reads (the URL/name/directory
conflict check) and then INSERTs, and SQLite refuses that mid-transaction
lock upgrade instantly when another connection wrote in between --
busy_timeout is skipped for upgrades.
Same shape and same fix as POST /api/download (daf2ba39): the handler
now uses `get_db_session(commit=True)`, which begins IMMEDIATE so the
concurrent writer waits its turn, and which commits before the refresh
is fired so the refresh does not wait on a lock the request still holds.
The regression test commits a write from a second connection between
the handler's conflict check and its INSERT. The existing
`write_lock_held_briefly` helper cannot reproduce this: the Sanic test
client takes over a second to start the app, so its lock is released
before the handler runs. Before the fix the test fails with the
production error verbatim.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Jobs queue and video transcode from the video page
Add an in-memory FIFO Job queue (wrolpi/jobs.py): functions register by
name, callers enqueue by name with JSON kwargs, and the perpetual-tasks
owner runs them one at a time. Logging emitted inside a Job and the
output of its subprocesses are captured into a per-Job log; Jobs report
progress and can be cancelled (a running Job's subprocess is killed).
Jobs are not persisted; an API restart forgets them.
Add a transcode Job and POST /api/videos/<id>/transcode. ffmpeg reports
progress via -progress pipe:1. The existing FileGroup is pointed at the
new file before the refresh so tags and history survive a container
change.
Replace the video page Refresh button with an Edit dropdown (Refresh,
Transcode). The Transcode modal pre-fills from the preferred codecs in
the Videos settings and shows the Job's progress, log tail and Cancel.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Video page: move Delete into the Edit dropdown
The Edit dropdown now holds Refresh, Transcode and Delete, so the action
row is Download / Edit / Add to Playlist. Delete keeps its confirmation
(a Confirm dialog) and the tagged-files prompt, and is disabled in WROL
Mode like the other items.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Video page: Edit > Edit to change a video's title
PUT /api/videos/<file_group_id> writes the title (and fulltitle) into the
Video's info json, creating the file when the video has none, then
validates the Video again so its title and search text follow. Files
stay the source of truth: the edit survives a refresh, and a re-download
of the metadata replaces it (the modal says so).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Video page: edit the description in Edit > Edit
PUT /api/videos/<file_group_id> now takes optional title and description
and changes only the fields given; an empty description clears it. The
modal sends only what changed and refetches the description after saving.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Transcode modal: allow a container-only remux
Keeping both streams is no longer refused: ffmpeg copies them into the
chosen container with fast start. The container select starts at the
file's own container, the hint explains what a remux does, and the button
reads Remux when nothing is re-encoded.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* WROL Mode: refuse video edits and transcodes past the API
The routes already returned 403, but a transcode Job queued before WROL
Mode was enabled still ran, update_video could be called from anywhere,
and the Edit modal's Save button stayed live. The Job now fails before
ffmpeg starts, update_video carries wrol_mode_check, and the modal
disables Save with a notice.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Transcode: verify the output before replacing the original
After ffmpeg exits, the output is probed and compared with the source
(container, presence of video/audio streams, codec per stream, duration
within 1s or 1%) and its last 30 seconds are decoded to a null output.
Any failure removes the output and keeps the original.
The tail decode replaces the download-time completeness check, which
grabbed a single frame near the end: one ffmpeg_video_complete (sync for
the downloader, async for Jobs) now serves both, and is stricter, since
every frame of the tail plus the audio must decode.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Jobs: claim atomically, cap pending, own-session subprocesses; retarget before delete
Review fixes. The worker read a Job's status and wrote RUNNING as two
steps, so a cancel landing between them was overwritten and the Job ran;
_claim_job now does both under the lock. enqueue_job refuses past
JOB_PENDING_LIMIT pending Jobs. JobContext.run_command starts the
subprocess in its own session and raises CancelledError when the command
was killed, so a handler cannot mistake it for a normal exit.
The transcode Job now retargets the FileGroup (committed) while the
original file still exists, and deletes the original afterwards
(transcode_video_file gained keep_original). A crash in between leaves
both files with the database pointing at a real one.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Video page: the Edit menu owns the transcode Job; polling survives Close and errors
Review fixes. The transcode Job's id and polling (useJob) move from the
modal to VideoEditMenu, so closing the modal no longer stops the page
from learning the Job finished (and refetching the video, whose path may
have changed). While a Job runs the menu shows Transcoding..., reopening
shows its progress instead of the form, and Delete waits. A failed poll
request is retried instead of ending the polling. getJobs checks the
response like its siblings. Design narration trimmed from comments.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Transcode: audio files, and Remove video to turn a video into an audio file
video_codec 'none' (REMOVE_VIDEO) drops the video stream (-vn, audio
required); an audio-only source is audio-only output whatever the video
target says, and a video target on it is refused. Audio output gets an
audio container (m4a/ogg/mp3) checked against the codec it will hold, and
a video container asked for audio output is replaced by the codec's
natural one. faststart only for mp4/m4a. Verification learned the
audio-only shape and ffprobe's container names (mkv reports matroska).
The modal offers 'Remove video (audio only)', switches to audio
containers and follows the audio codec, reads Extract Audio, and opens
for audio files (no video codec select).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Video page: yellow Edit button
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Transcode: remove the temp output on cancel; tell the user when a closed-modal Job fails or is lost
Review fixes. The temp output was only removed on Exception, but a
cancelled Job's run_command raises CancelledError (a BaseException), so a
cancel left the partial file for a refresh to index. Closing the modal
during a Job that then failed showed nothing; it now toasts the error
(and a cancellation). Polling gives up on a 404 or five failures in a
row (the API restarted and forgot its Jobs), warning the user and
unlocking Transcode and Delete instead of reading Transcoding... forever.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Jobs: _claim_job docstring states the invariant, not the old race
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Transcode: fragmented mp4 option for very long videos
A whole-file mp4 index grows with the frame count; for a many-hour video
it reaches tens of MB, which a phone must fetch and parse before the
first frame, and every seek walks it. `fragmented` writes the file with
-movflags +frag_keyframe+empty_moov+default_base_moof+global_sidx and
10s fragments: a ~1 KB moov plus a global seek index, so playback starts
and seeks quickly. mp4/m4a only; a remux, so it takes seconds per GB.
Measured on an 11-hour h264/aac file: moov 40 MB -> 1.4 KB, seek index
2 x 120 KB, 10k fragments, duration and codecs unchanged, remux in 25s.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Transcode modal: the Fragmented label says audio for audio output
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Video edits keep the original: custom_title/custom_description in the info json's wrolpi section
Editing a title or description no longer overwrites yt-dlp's keys. The
values go into the info json's `wrolpi` section (custom_title,
custom_description); the modeler prefers them when deriving the Video, so
a refresh keeps the edit while the original title and description stay in
the file. An empty custom description means none.
Both re-download paths carry the wrolpi section over: the metadata-only
refresh merges it into the fresh info json, and a full re-download
snapshots it before yt-dlp overwrites the file and restores it (then
re-validates) in finalize_download. FileGroup.get_wrolpi_json reads it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Refreshing a specific file (the Files page Refresh button) bypassed the
ignored_directories check that directory and global refreshes apply.
Refreshing a hard link inside the playlists directory therefore created a
second Video for the same file.
The direct-file path now drops existing files that live in an ignored
directory before comparing against the database. Deleted paths are kept
so stale records are still removed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Hotspot captive portal: welcome page with the WROLPi's address
Devices that join the hotspot probe a fixed plain-HTTP URL to check for
Internet access. NetworkManager's shared-mode dnsmasq now resolves those
probe hostnames (and `wrolpi`) to the hotspot address via an include the
Controller writes when the hotspot starts, the probes land on the Controller
(port 80), and are redirected to /portal: "This is your WROLPi, its address is
10.42.0.1, open WROLPi / open the Controller".
Only the probe hostnames are hijacked so the hotspot keeps forwarding
everything else when the WROLPi has an uplink. The include is corrected and
the connection re-activated once if NetworkManager picked a non-default
subnet. Writing the include is best-effort; the hotspot starts regardless.
`hotspot.captive_portal` (controller.yaml, default true) toggles the probe
redirects and the include; like the other hotspot settings it applies on the
next hotspot start, so saving while the hotspot is up reports
restart_required and both Controller UIs say so. The status endpoint gains
the hotspot address and portal URL, shown in both UIs. repair.sh only
creates the include directory so a disabled setting survives upgrades.
Native only: Docker has no hotspot.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: address review
* React: re-read the hotspot address whenever the HotspotToggle starts or
stops the hotspot, matching the fallback UI's refresh-on-status.
* A WPA3 start that reverts to WPA2 leaves the hotspot up, so it now gets
the captive portal address correction too.
* The corrective `nmcli connection up` logs a warning on a non-zero exit;
dnsmasq would otherwise keep the old address silently.
* HotspotStatusResponse.captive_portal defaults to true like everywhere else.
* Trim design narrative from the module docstring and React comments.
* The include-before-activate test now observes the include at the moment
of the `nmcli ... hotspot` call instead of inferring from call order.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: never forward any record type for probe hosts
`address=` only synthesizes A/AAAA answers. iOS also asks for the HTTPS
record type; dnsmasq forwarded that upstream, Apple's answer aliased to
captive.g.aaplimg.com, and the phone fetched the genuine "Success" from
Apple instead of the portal. Each probe host now also gets `local=`, so
dnsmasq answers every type itself, and the CDN alias is a probe host too.
Observed with dnsmasq query logging on a hotspot with an eth0 uplink.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: let acknowledged clients finish so the OS offers "Done"
A phone's sign-in sheet only shows "Done" once the OS re-probes and gets
the genuine success reply. The Controller now remembers each client (by
hotspot IP) that has loaded the portal page and answers its later probes
with the reply each OS expects (Apple "Success" HTML, Android 204, Windows
"Microsoft Connect Test", ...). The set is cleared when the hotspot
starts so a new session shows the page again.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: acknowledge on Continue, not on view
iOS fetches the portal page in the background before opening the sheet,
then re-probes after the sheet's own navigation. Acknowledging on view
made that re-probe return "Success" as the sheet's content. Phones only
re-check the network (and offer "Done") after a full-page navigation, so
the welcome page now has a Continue button leading to /portal/continue,
which is the only place a client is acknowledged. The Continue page says
to tap Done and links to WROLPi and the Controller.
Also hijack clients3.google.com (older Android probe host).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* help: bump wrolpi-help to the captive portal and Speed Test docs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: comments describe the Continue design; clear acks on stop
Three comments still described acknowledging on view, which c72a7f7b
removed. Acknowledged clients are now also cleared when the hotspot
stops, so a lease reused in a later session sees the portal page.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Speed Test calculator: ping, download, and upload against the API
Users asking whether the hotspot is fast enough, or how many people can
stream video at once, had no objective tool that works offline. This
adds one under Calculators > Tools & Reference.
Backend (wrolpi/speedtest.py, /api/speedtest):
* ping: tiny no-store JSON the browser times for round trip latency.
* download: streams slices of a pre-built random buffer for a clamped
number of seconds (0.1-15). Random so no proxy or compression can
flatter the result; slices so no bytes are generated per request.
* upload: streaming-body handler that reads and discards the body and
reports bytes and elapsed time. Requests over 8 MiB are refused.
* info: the client address Caddy forwarded plus how many tests are in
flight. Sanic now trusts one proxy hop (PROXIES_COUNT = 1) so
request.remote_addr is the browser, not Caddy. Nothing else read
that field before.
There is deliberately no one-test-at-a-time lock. Several people
testing at once is how a user learns how the link shares between
streamers, so the server only counts in-flight tests (a shared_ctx
Value created pre-fork so every worker sees the same number) and the
UI reports how many others ran alongside.
Both Caddyfiles route /api/speedtest/* unbuffered, as the SSE chat
route already is, so throughput is measured live rather than in bursts.
Frontend:
* speedtest.js holds the logic with no React in it (bucketed
throughput, steady-state average that drops the first second, ping
stats, stream-capacity tiers, hotspot address check) so Jest drives
it with fake fetch and streams.
* SpeedTestCalculator runs 20 pings, a 10 s download, and a 10 s upload
with a live figure and rolling trace, then shows Mbps and MB/s, ping
verdict, the client address with a hotspot badge, and a table of how
many 480p..4K streams the download speed supports. Upload is
sequential POSTs of one 4 MiB Blob because Safari cannot stream a
request body. Nothing is saved.
* Sparkline is a new inline-SVG ui component (no chart dependency),
added to /theme-sample and checked in all four themes. A missing
sample is a gap, not a plunge to zero.
The page states that it measures the network path, not the drive that
actually serves video files.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* speedtest: bound uploads in time, close the Start race, address review
Review on #708 found three bugs and some polish:
* Start could be clicked again while the first fetchInfo() was still
pending, because the phase only left `idle` after that await. A
second click made a second AbortController and a second concurrent
run, and Cancel only reached the later one. start() now refuses when
a controller already exists and enters a `starting` phase
synchronously, so the button is Cancel before the first await.
* The upload phase could overrun the advertised 10 s by however long
the in-flight 4 MiB POST took: on a slow hotspot, the very link this
tool exists for, that is tens of seconds. runUpload() now aborts the
in-flight request at the phase deadline (counting only completed
requests) and uses 1 MiB bodies so a few-Mbps link still completes
several inside the phase.
* The server upload sink was capped by size only, so a slow or paused
body could hold a worker until 8 MiB eventually arrived. It now also
enforces MAX_UPLOAD_SECONDS (15 s, matching download) and answers
408 when exceeded. Both caps together are what "cannot hold a
worker" actually requires.
Also: a NaN `seconds` now gets a NaN message instead of "must not be
negative"; the module, Sparkline, and gallery docstrings are trimmed
to purpose; and the upload counter is covered for the mid-stream 413,
the 408, and a broken body.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* docs: browse walks the file_group date index instead of probing every doc
The unfiltered /api/docs/search drove its join from `doc`, so SQLite probed
every doc's file_group row for the sort key before returning the first page.
With 20k docs scattered through a 10 GB database on a spinning disk, the first
request took about a minute cold (0.07s warm).
Use the same CROSS JOIN shape as the archive browse: file_group is the outer
table, SQLite walks file_group_published_datetime_idx and probes doc's covering
unique index on file_group_id. Both sides stay index-only and the scan stops at
the page limit. Measured cold: ~60s -> 0.6s, identical rows.
Filtered searches and the size/title orderings keep the ORM query.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: trim browse comments, stabler plan assertion
Review feedback: drop the benchmark narrative from docstrings, state only the
non-obvious constraint at the call site, and assert the doc covering-index probe
by shape rather than by autoindex name.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* zim: recompute the outdated-Zims flag after startup and on Delete
The Dashboard shows the "Outdated Zim Files" banner from the persisted
`outdated_zims` flag, but the modal lists files from a live scan. The two
disagreed: the flag was restored from the DB on startup and never
recomputed once the outdated file had been removed, so the banner showed
with nothing to delete, and the modal displayed an endless placeholder.
- The startup recompute now runs after the DB is up and configs are
imported, so it overrides the value `init_flags` restored from the DB.
Previously it ran before the DB was up, and the restored value won.
- DELETE /api/zim/outdated always recomputes the flag, even when nothing
was deleted, so clicking Delete clears a stale banner.
- The modal re-scans when opened and says when nothing was found instead
of showing a placeholder.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* zim: enforce restore-then-scan ordering; no Kiwix restart on empty delete
- Startup calls flags.init_flags() then flag_outdated_zim_files() right
after db_up, before the DB config import, so the disk scan always
overwrites the DB-restored value in one local sequence.
- DELETE /api/zim/outdated recomputes the flag in a finally.
- remove_outdated_zim_files skips refresh and the Kiwix restart when no
file was deleted.
- Trim narrative comments to the constraint they enforce.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
COUNT(*) OVER() forced SQLite to visit every matching file_group
before LIMIT, which is a full table scan of the (large) file_group
table — ~22s cold on a Pi for videos ordered by download date.
Count separately and cache for 60s by filters so indexed orders can
stop after one page. Past-the-end pages now report the true total.
Videos/archives/docs pagers keep totalPages across page changes.
* videos: codec download preferences with transcode and strict-fail options
Add video_codecs/audio_codecs ordered preference lists to the video,
channel, and RSS downloaders, alongside the existing resolution
selector. Codec preference is codec-major: a preferred codec at a
lower resolution beats another codec at a higher resolution.
When 'transcode' is enabled, a downloaded file whose streams do not
match the preferences is re-encoded with ffmpeg after download (only
the mismatching stream; targets are extensible maps, currently h264
and aac/opus/mp3). When 'strict_codecs' is enabled and transcode is
not, a download with no matching format fails permanently instead of
falling back.
Empty codec lists produce a byte-identical yt-dlp command to before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* videos: honor strict_codecs when transcode cannot produce the preference
Review fixes:
- strict_codecs is now suppressed only when transcoding can produce a
preferred codec for every non-empty preference list (new
transcode_can_satisfy_codecs); av1-only preferences with transcode
enabled fail under strict instead of silently keeping the file.
- No partial transcode: an unfixable video mismatch blocks an
audio-only conversion (and vice versa); the mismatch falls through
to strict/keep.
- fsync the transcode output before rename, and the directory after,
so power loss on a Pi cannot leave a truncated final file.
- mp3 yt-dlp pattern matches mp4a.6B as well as mp4a.6b.
- ffmpeg maps use 0:V:0 (excludes attached pictures) and 0🅰️0?
(tolerates missing audio).
- Strict toggle is no longer disabled by transcode in the UI; popup
wording corrected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* videos: serialize transcodes machine-wide via shared_ctx lock
Sanic runs multiple worker processes, so an asyncio primitive cannot
serialize transcodes across download workers. A multiprocessing.Lock
in shared_ctx now guarantees only one ffmpeg transcode runs at a time
on the machine; waiting workers poll the lock asynchronously so the
event loop stays responsive while a transcode holds it for hours.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* videos: transcode-lock review fixes
- Document that a crashed lock holder requires an API restart; this is
accepted rather than adding owner-death recovery.
- Move the 'Transcoding' user notification to after the lock is
acquired, so it fires when work begins rather than before a
potentially hours-long wait.
- Soften the lock test docstring to what it measures (in-process
wait/release); trim the acquire docstring to the polling rationale.
- Give transcode_lock its own section header in contexts.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
test_search_videos_does_not_block_event_loop asserted a wall-clock
threshold (echo < 0.4s), which fails on loaded CI machines from
scheduler noise alone. Replace the timing with ordering: the mocked
search parks its thread on an Event, the echo is sent only once the
handler is provably inside the query, and the test asserts the echo
completes while the query is still held open. Events are only waited
on off the event loop, so a regression fails the ordering assertion
instead of deadlocking.
Verified both directions: passes repeatedly against the real endpoint,
and fails when the endpoint is changed to run the search on the loop.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* files: left-align the preview toolbar, Open joins the button row
The modal actions row packs its children to the right, which suits a
confirming button but stranded the preview's toolbar. The toolbar now
fills the row so its buttons read from the left, with Open first in
the same row as Download. On phones the tags selector always takes its
own line instead of squeezing in beside the buttons.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* tags: the add-tag button is violet like every other tag button
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* share: a real button everywhere but the navbar corner
The share trigger was a bare anchor-wrapped glyph, which matched the
navbar's other corner icons but looked out of place beside real buttons
in the file preview toolbar. It is an IconButton by default now; the
navbar keeps the bare glyph via asIcon.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* files: readable text previews in dark themes
A same-origin text document inherits the page's color-scheme, so dark
themes rendered the browser's default light text on the preview's
forced white background. Force a light color-scheme on the preview
iframe so text is always black on white.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* files: text previews follow the theme
Instead of pinning the preview iframe to a light scheme, follow the
theme: dark, night, and amber render the document light-on-dark against
the theme's panel color; light stays black-on-white.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Uploaded singlefiles are base64-encoded into JSON before being sent to the
archive container, so an 88MB upload becomes a ~118MB request. Sanic's default
REQUEST_MAX_SIZE of 100MB rejected it with a 413 before the handler ran, and the
API surfaced this as KeyError: 'singlefile'.
Only the Docker deployment is affected; Debian/RPi extract in-process.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Completed once-downloads are never written to the downloads config, and
importing the config (which runs on every startup) deleted any DB Download
the config did not list. That erased the last_download_attempt rows the
per-domain and global daily limits count from, so a restart reset the count.
Keep completed once-downloads on import; pending/failed and recurring
downloads still follow the config as the source of truth.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Fix API stalls: non-blocking switch worker, threaded searches, indexed Videos page
Three response-time fixes found by benchmarking the read-only API endpoints
hot and cold on real libraries:
* `switch_worker` runs on the Sanic worker's event loop every 0.1s and called
the blocking `multiprocessing.Event.wait(timeout=1)`. With no switch pending
(the normal state) this froze the loop for a second at a time; about a third
of all requests, including trivial ones, took ~1,000 ms. It now polls
`is_set()` and returns immediately.
* The search handlers (videos, files, archives, docs, zims) and `/api/statistics`
ran synchronous SQLite queries on the event loop, so one slow search stalled
every other request on that worker. They now run via `asyncio.to_thread`.
* The unfiltered Videos page (`POST /api/videos/search` with only an order)
used `mimetype LIKE 'video/%' OR ...` plus `COUNT(*) OVER()`, which visits
every video's index entry before LIMIT applies: seconds cold on a large
library on a spinning disk, and ~10 s hot for a deep page. It now seeks each
exact mimetype on the (mimetype, effective_datetime) index for `limit + offset`
rows and merges them, with the total from a separate count cached for 60 s.
Results are identical to the generic query.
Also adds `scripts/api_perf.py` (times every read-only endpoint against a host,
optionally restarting the API over SSH for cold numbers) and
`scripts/api_perf_compare.py` (merges per-host results). Host profiles live in a
gitignored JSON file; an example is included.
Measured on an 8-core docker host with a 10 GB library: median hot request
1,007 ms -> 22 ms; Videos page 1 hot 287 -> 108 ms, page 50 1,345 -> 120 ms.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Serve the video description from its own endpoint instead of Video.__json__
`Video.__json__` computed `description` as `c_text or get_video_description()`,
where the fallback opens and parses the video's info json from disk. The list
endpoints serialize every result with `__json__`, so any page containing videos
without an indexed description (common: many videos have none) read their info
json files on every request, cold, for a field the list never displays. The
description is not part of search either: search runs against the FTS copy of
`c_text`, so the fallback never fed the index.
Drop `description` from `__json__` and serve it from
`GET /api/videos/<id>/description`, like comments and captions. The endpoint
returns the indexed `c_text`, falling back to the info json only there, for a
row indexed before descriptions were stored. The player page fetches it through
`useVideoExtras` alongside the comments.
Tests assert the list and detail responses never touch the info json, and cover
the endpoint's indexed, fallback, and not-found cases.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address review: thread-local zim sessions, fast-path equivalence test, comments
* Zim search, single-zim search, and search estimates run libzim off the event
loop, each opening its own session inside the worker thread rather than
passing the request's SQLAlchemy Session across threads.
* The Videos fast-path test now runs the same requests through the generic
query and asserts identical rows and paging for both date orders. The one
intentional difference is documented: past the end, the fast path still
reports the true total.
* switch_worker returns before its try block so the empty path does not clear
the event; its comment states the invariant only. The copied "synchronous
SQLite" comments are reduced to one line in one place.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* videos: a collaboration video's channel directory conflict killed the download
YouTube collaboration videos appear in every collaborator's feed but their
metadata names only the primary uploader's channel. When that channel's
computed directory already belonged to an existing Channel that no longer
matched by source_id/url/name, channel creation raised
ChannelDirectoryConflict and the download failed unrecoverably. Now the
channel that owns the directory is used instead of creating a conflicting one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: trim comments that restated the code
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
NAME_PARSER's suffix whitelist had no audio suffixes (and no mkv/m4v),
so a correctly-named audio file got no published date and sorted to the
very end of its Channel -- which reads as missing. The whitelist now
carries the media suffixes a Video FileGroup can lead with.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* videos: an uploaded audio file never appeared in its Channel
The upload flow asks Video.can_model, which only accepted video/*, so an
uploaded .flac was indexed with no Video row -- and the batch modeler's
`indexed != True` gate then excluded it on every later refresh, leaving
the file permanently invisible in its Channel.
can_model now accepts audio/* like the batch modeler always has, and the
modeler gates on the Video row being missing (mirroring the doc_modeler
fix for the same class of bug) so already-stuck files heal on the next
refresh. Audio playlists (.m3u detects as audio/x-mpegurl) are excluded
from both paths and unmodeled by video_cleanup.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: widen the playlist denylist, prove modeler termination
The denylist gains audio/x-scpls and audio/scpls -- .pls detection
varies by libmagic version, and any audio/* playlist type must be
refused wherever it comes back. Modeling one file is extracted as
_model_video so a test can force persistent failure and prove the loop
terminates; failed_ids is recorded before the PYTEST re-raise so no
failure path can re-select an id forever. Playlist mimetypes are bound
through json_each like the rest of video_cleanup, and the comments
state the invariants instead of the history.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: the playlist test must not depend on libmagic's m3u opinion
CI's libmagic detects .m3u as text/plain where a Mac's says
audio/x-mpegurl. The invariant is that a playlist mimetype is never
modeled by either path, so the test forces the mimetype like the other
denylist tests instead of asserting what detection returned.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* common: config writes must be atomic and version-checked under the save lock
A config file must never be observable in a truncated state: writes go to a
temporary file in the same directory and are renamed into place. The version
check in ConfigFile.save() reads the config file, so it must hold
config_save_lock like the write does.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* collections: item writes must begin as SQLite writers
The add/remove/reorder item endpoints read before their first write; a
deferred transaction upgrading at that write fails instantly with "database
is locked" when any other connection wrote in between, bypassing
busy_timeout. Open their sessions with get_db_session(commit=True) so the
transaction takes the write lock at BEGIN, matching post_download.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: error responses must not commit the write session; trim comments
Map exceptions to 4xx outside the get_db_session(commit=True) block so an
error response never commits, and serialize inside the session. Close the
mkstemp descriptor when fdopen never took ownership. Trim comments and test
docstrings to what the code and assertions actually cover.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* app: the navbar corner wrapped onto a second row on small phones
Every status-icon slot carried a 1.5em separator margin -- ~80px across
three slots -- which wrapped the mobile bar's whole corner onto a second
row on any screen narrower than about 380px. The margin is now 0.25em on
both bars: the corner's gap and the slots' own padding still space the
icons, and on desktop the freed room just goes back to the links that
useOverflowNav folds into "More".
The mobile bar is extracted as MobileNav (mirroring DesktopNav) so a
Cypress suite can mount it and hold it to one row at phone widths.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: prove 320px with a faithful fixture, trim the CSS narration
The suite claimed 360px covered narrower devices because the fixture was
wider than production, but the fixture also lacked the real theme
toggle's 0.8em right margin, so the safety margin was overstated. The
fixture now matches a phone -- 32px logo home link, the toggle's margin,
and a coarse pointer so the search link drops its keyboard hint, with a
test asserting the hint is really absent -- and 320px is measured
directly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The channel downloader refreshes a Channel's files before downloading,
already passing send_events=False, but the parameter was never plumbed
into the file worker, so every pre-download refresh notified the user.
FileTask now carries send_events and handle_refresh skips the
Refreshing/Refreshed events when it is False. User-requested refreshes
and global refreshes still send Events.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* e2e: exercise the FileBrowser footer buttons against real files
Sixteen Cypress tests covering Rename, Move, Delete (including the
tagged-file force-delete flow), Ignore/Unignore, Upload, and scoped
Refresh, with no API mocking: fb:* tasks seed a cypress-fb/ sandbox
inside the media directory, the UI mutates it through the real API, and
assertions check the disk as well as the table. Skipped in CI, which
runs only the React dev server.
Background jobs (move, refresh) only queue work and worker_status reads
idle while jobs wait, so cy.refreshFiles / cy.waitForEvent poll the
events feed for completion events newer than the server's clock, and
fail loudly when one never arrives. Every suite cleans up
unconditionally: sandbox files, DB rows via a final refresh, test tags
and their tags/ directories, and any leaked ignored-directory config.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* e2e: act on review — drain stragglers, teardown that cannot be skipped
Events carry no job id, so waitForEvent cannot prove WHICH job finished.
The rename tests now drain the parent-directory refresh their rename
queues before ending, so the next test's refresh wait cannot be
satisfied by a straggler while the DB still reflects the old tree; move
waits match the destination in the completion message.
Teardown must always reach the disk wipe: a failed command aborts the
rest of its hook, so the tag cleanup and the DB-purge refresh are now
best-effort (failOnStatusCode: false, no assertions), and the unignore
runs after every test instead of once per spec. History and
forward-looking product notes trimmed from comments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
perpetual_shutdown is set by the real before_server_stop listener, so any
earlier async_client teardown in the same pytest-xdist worker leaves it
True on the module-global app and the watchdog returns before reclaiming.
Failed under -nauto, passed alone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Starting the file-worker pump in every Sanic worker made 5 processes
race the shared status dict, the busy flag, and the same directories.
Non-owners now run only a reclaim watchdog: they refresh nothing unless
the owner PID is gone or its heartbeat is stale.
Production wait_for_job polls every 0.1s instead of 100 Hz. A failed
rename includes the recorded job error in the 409 body.
A directory rename waits inline on a move job. When that move failed
(stale FileGroup rows colliding on primary_path), handle_move reverted
the files and sent a failure event but left the job pending. wait_for_job
only understands 'complete', so POST /api/files/rename spun until its
300s timeout and the Rename modal never closed.
The same sequence could kill file processing for the life of the process.
perpetual_signal treated any CancelledError as terminal and logged
nothing. Separately, perpetual_tasks_started is a shared Event that
survives the worker that set it: docker auto-reloads and worker crashes
left every later after_server_start as a no-op. Queued refreshes sat
untouched while worker_status reported idle. Restarting the API was the
only recovery.
Mark failed jobs failed and raise immediately from wait_for_job. Catch
unexpected cancellation and reschedule unless the app is actually
stopping. Track the perpetual-loop owner PID and retake the claim when
that process is gone; every Sanic worker still runs the file-worker
pump. Harden transfer_queue against the cross-process Empty race, and
stop request processes stealing from the shared public queue.
* upgrade: restart services when the upgrade fails before code changes
The upgrade script stops wrolpi-api and wrolpi-app before fetching, so a
failed fetch, signature verification, or checkout left the WROLPi down
with unchanged code on disk. The EXIT trap now restarts both services
for failures that occur before scripts/upgrade.sh begins modifying the
system. The post-checkout verification failure rolls the branch back to
the previously-running commit first, so services never restart on an
unverified commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* upgrade: only restart services when rollback lands on the known-good commit
Review fixes: a failed rollback no longer leaves the restart flag set, so
services cannot start on an unverified or mixed tree; the non-atomic
checkout failure path now rolls back too; trimmed narrative comments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* downloads: a batch of failed downloads that retried in lockstep
`calculate_next_download` gives every deferred download the same 3^attempts
backoff, measured from the moment it failed. Downloads that fail together
therefore retry together, and a synchronized burst against one host is what
gets a whole batch rejected -- which defers them all again, on the same
schedule. The backoff grows, but the batch never breaks up.
Jitter each retry forward by up to half its backoff. The backoff stays a
floor, so nothing is retried early; a recurring download is still capped at
its own frequency, so nothing is retried later than it was before.
Deferred scheduling is no longer an exact value, so the tests that pinned it
assert a floor and a ceiling instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* review: comments that narrated the design instead of the code
Trim the jitter comment to a single WHY and the test docstring to its summary
line. The reasoning that led here belongs in the PR, not beside the
arithmetic.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* downloads: a deferred once-download that nothing ever retried
Deferring a download only schedules it: `defer()` sets the status and
`calculate_next_download` sets `next_download`. Acting on that schedule was
`renew_recurring_downloads`, which filters on `frequency != None` -- so only
recurring downloads were ever marked "new" again.
A once-download has no frequency. One transient error (an HTTP 403 from the
host, a dropped connection) deferred it with a `next_download` that nothing
read, and it stayed deferred until someone retried it by hand.
`delete_old_once_downloads` did not clear it either -- that requires a
`last_successful_download`, which a download that never succeeded does not
have. So every transient failure added one more permanently-stuck row to the
downloads list.
Add `renew_deferred_once_downloads`, called from `do_downloads` alongside the
recurring renewal. `attempts` is preserved, so `calculate_next_download`'s
3^attempts backoff keeps growing across retries: a URL that never succeeds
backs off toward never instead of retrying at a fixed interval.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* review: comments that narrated the bug instead of the contract
The new docstrings restated why the bug existed and how the fix was reasoned
about. That history belongs in the PR, not in code that has to stay true.
Keep the invariant that a reader needs: attempts are not reset, so the backoff
grows. In the test, the assertions already document the contract.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ui: a download form that emptied itself after every submit
A successful submit reset the whole form, so adding a second download meant
re-choosing the tags, the destination and the video settings that the first
one had already been given.
It now empties only the URLs. Everything else stays for the next download,
and a Clear button beside Download puts the form back to its defaults when
the next one is unrelated.
`useForm` gains `clearPathsOnSuccess`; `clearOnSuccess` is untouched for the
forms that want the whole reset.
* review: the one download form that kept its URLs
Scrape took a `clearOnSuccess` prop that no caller passed, so mapping it to
the URL paths left create-mode scrape clearing nothing. It follows the same
rule as Videos and Archive now: the edit form supplies `onSuccess`, and only
that keeps the form as it was.
`clearPaths` also wrote every path it was given, so a `urls` form gained a
`url: ''` it never declared -- which the comment beside it and the test named
for it both said it would not do. It skips a path neither the defaults nor
the form has, and the test asserts the key is absent.
* ui: two upload dialogs that had drifted apart
The dashboard and the file browser each had their own upload component: two
dropzones, two overwrite toggles, two copies of the progress-bar rendering,
against the same `useUploadFile` state. The file browser's dialog was made
fullscreen with a drop target that grows; the dashboard's stayed a `large`
dialog with a strip to drop onto, because it was a different file.
They differ in one thing that is real: the file browser knows the destination
because the user selected it, and the dashboard has to ask. That is now the
`destination` prop -- `undefined` asks, a string uploads there, and '' is the
media directory rather than a missing destination.
Everything else is one component, so the next change to either entry point
lands on both.
* review: an upload dialog that looped on the media directory
Pressing Upload in the file browser with nothing selected passes `''` -- the
media directory. Three checks along that path read it as "no destination":
* The clearing effect ran, and `doClear` is a new function every render whose
own setState scheduled the next one. React ended it with "Maximum update
depth exceeded", 16 warnings deep, on the default Upload path. The effect
now clears only while asking, and `doClear` is stable.
* The title read `Upload to: /media/wrolpi/`.
* `doUpload` bailed, so a drop on the media root uploaded nothing.
All three predate this branch; moving the dialog is what put them behind one
component. Tests cover the title, what the hook is told, and that a fixed
destination is never the thing that empties the queue.
