Caddy is PID 1 in the web container and reaps nothing. The healthcheck's
busybox `wget --spider https://...` forks an `ssl_client` helper that is
orphaned to caddy and stays a zombie, one per check. `init: true` runs
tini as PID 1 so they are reaped.
The zim healthcheck calls curl, which the debian:13 kiwix image does not
install, so every check exited 127. The help healthcheck probed port
8086, which is Caddy's HTTPS proxy on the web container; mkdocs serves
on port 80 inside its own container.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
A Sanic worker is forked from the manager and carries the manager's
Python SIGTERM/SIGINT handlers until `_setup_system_signals` installs its
own, which happens after the startup listeners (about two seconds with
sixteen workers starting at once). When auto-reload fires a second time
inside that window the manager terminates the half-started generation,
each child runs `WorkerManager.shutdown_signal` instead of exiting, marks
itself TERMINATED, and then finishes starting. The manager has already
spawned replacements, so the old generation lives on: untracked, holding
the listening socket, serving stale code, and holding ~220 MB each.
Register an at-fork hook in the main process that resets both signals to
their default action in the child. The perpetual process already did
this by hand for itself; server workers now get the same treatment.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The listing still read every channel row (for the summary columns) and every download
row of the listed Collections (for the minimum frequency). Both tables store yt-dlp
info_json, so rows are tens of KB and each sits on its own scattered page; selecting
fewer columns did not help because the seek per row is the cost. From a cold disk
cache that was 5 s per table, 10+ s total, on a large library.
Two covering indexes, channel (collection_id, video_count, total_size) and
download (collection_id, frequency), answer both queries without touching a row. The
listing no longer loads Download objects at all; it computes min_download_frequency
with one grouped query and drops the unused `downloads` array from listing rows (the
detail endpoints keep it; the list views only read min_download_frequency).
Adds a models-vs-migrations test (alembic check) so an index or column added to a
model without a migration, or vice versa, fails the suite.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* collections: channel listing reads trigger columns and batch-loads tag/downloads
Listing channel collections summed FileGroup sizes through a video join, which reads a
random page of the file_group table per video, and lazy-loaded every collection's tag
and downloads one query at a time. On a large library with a cold disk cache the
listing took over two minutes, past the browser's 60 s timeout, so the Channels page
showed an error.
channel.video_count and channel.total_size are already maintained by triggers
(wrolpi/schema_ddl.py), so read those. Tag and downloads are selectin-loaded, so the
query count no longer grows with the number of collections.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* collections: domain listing reads trigger-maintained summary columns
Domain Collections summed file_group.size through an archive join on every listing,
one random page read per archive (about 9 s from a cold disk cache on a large
library). New triggers keep collection.item_count and collection.total_size current
for Domain Collections, as the channel triggers already do for Channels, and the
listing reads those columns. A migration installs the triggers and backfills
existing Domain Collections.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The Protomaps basemap keeps a few attributes per POI beyond the name the
map already labels: kind, kind_detail and elevation. Clicking a POI now
opens a popup with those, plus the feature's coordinates (or the click
location when the feature has no point). The same POI appears at several
min_zoom levels within one tile, so features are deduped by name and kind.
Map popups (this one and the existing pin popup) now follow the active
theme's tokens instead of MapLibre's fixed white, covering light, dark,
night and amber.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
A video's audio stream can be dropped the same way its video stream could
already be: a new REMOVE_AUDIO audio target maps only the video stream with
-an, needs a video container, and is refused for audio files. Output
verification insists the result has no audio stream.
Removing both streams is refused by the validator, the API, resolve_output,
and the modal: choosing Remove for one stream resets the other to Keep and
withdraws the other's Remove option.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Six helpers turned a non-OK response into an empty value, so their
consumers rendered a server error as "nothing here":
* getConfigBackups -> {dates: []}: "No backups available for this
config." A thrown request also left "Loading backups..." forever.
* getInventoryBackups -> []: "No backups yet."
* getDownloaders -> {downloaders: []} on a throw, the error body on a
non-OK response (which then crashed the .map): the RSS form's
Downloader select sat empty.
* searchMap -> {results: []}: "No map locations found".
* getMapSearchStatus -> {indexed: [], missing: []}: "No maps are
installed".
* flasherSearch -> [null, null], then `|| []`: "No .bin firmware
found".
Invariants now enforced and tested:
* On a non-OK response each helper returns undefined; an empty value
only ever comes from the server.
* Each consumer shows an error for a failed or thrown fetch and keeps
its "none" message for a confirmed empty result. The Downloader
select is disabled and says it is loading or could not fetch. A
failed map status is unknown, never "no maps are installed".
* The map search tab loads from the first render, and a query shorter
than the two characters the request needs now says so instead of
"No map locations found".
Archive browsers and recent tags are left as they are: a failed fetch
there hides an optional picker or suggestion list without claiming
anything.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`useForm`'s input handlers called `e.preventDefault()` unconditionally, but the
Mantine-backed `NumberField` (Sleep between requests, Depth, Maximum Pages)
hands them a bare `{target}` object because NumberInput emits a value, not an
event. Every keystroke in those fields threw.
`handleInputEvent` now only calls `preventDefault` when the argument actually
has one, and `getInputProps`' wrapper no longer repeats the call. Audited every
other `preventDefault` call site: all are bound to real DOM/React events or a
guarded no-argument call.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Both contexts start as {} and nothing distinguished that from a
server that reported nothing, so for up to one poll interval after a
page load consumers rendered the negative case as fact: the Settings
form appeared with blank values and an "undefined/config/wrolpi.yaml"
path (the ready check `settings ? true : undefined` was always true),
the Downloads toggle was clickable and read "Downloading Disabled",
the upgrade panel said "up to date, vundefined", and the five hardware
toggles on the Controller page claimed "not supported on this server".
Invariants now enforced and tested:
* StatusContext.loaded is true once a poll returned a real status;
the API being down does not count. SettingsContext.loaded is true
once settings were fetched, and `failed` when a fetch returned
nothing before that; a failed refresh after a load keeps the last
known values and `loaded`.
* The Settings page shows its loader until loaded and its error only
when failed. The upgrade panel shows a loader until status loaded.
The Downloads toggle is disabled with a neutral label until then.
* A subsystem toggle is disabled without claiming "unsupported" while
status is unknown (undefined); "unsupported" (null) is reserved for
a loaded status that lacks the field. A call in flight is unknown,
not unsupported.
The context fixtures declare loaded so existing specs render as
before.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The Flasher's saved configurations started as [] with no loading
flag, and the Saved Firmwares tab is the default tab, so every load
painted "No saved configurations yet." until the fetch returned. The
api helper returned [] on a non-OK response too, so a failed fetch
read as an empty list.
useStatistics used useState() with no initial value, so the
Statistics page's first render saw undefined and flashed "Failed to
fetch statistics" for one frame before the effect set {} and the
spinner appeared.
Invariants now enforced and tested:
* null = pending, undefined = fetch failed, [] / object = loaded. The
api helper returns undefined (and toasts) on a non-OK response.
* The Saved Firmwares tab shows a placeholder while pending, an error
on failure, and "No saved configurations yet." only once the server
confirmed there are none. Only the newest configs request may write
state, since save and delete refetch.
* The Statistics page shows its loader while pending and its error
only for a failed fetch.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* app: map pins and subscriptions tables distinguish pending from empty
Both lists were initialised to [], which SortableTable renders as
"loaded and empty". The Pins tab therefore showed "No pins.
Right-click on the map to add one." until the pins arrived, and the
Manage tab's subscriptions table showed "No results" until the catalog
did. A non-OK response (the api helpers return undefined) or a thrown
request left the same messages up permanently.
Invariants now enforced and tested:
* null = pending, undefined = fetch failed, [] = confirmed empty, so
SortableTable shows its placeholder while pending and each empty
message appears only once the server confirmed the list is empty.
* A failed fetch renders an error in place of the table.
* Only the newest request for each list may write state, since edits,
deletes, subscribes and unsubscribes refetch while the mount request
may still be in flight.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* app: the regions preview waits for the catalog
"View All Regions" stayed enabled while the catalog was pending or
failed, and the modal received a fresh `catalog || []` on every
render. Its map effect depends on that array, and ManageMap
re-renders on every status poll, so an open preview tore its WebGL
map down and rebuilt it every three seconds, and a failed catalog was
previewed as zero regions over the error message.
The button is disabled until the catalog is an array, and the modal
is mounted only then, so it receives the state array itself.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
useSuggestions started with channels: [] and reset to that on every
keystroke, and the normalizer treats an empty list as "the server
found none". So the global search dropdown showed a "Channels: No
results" row on focus before anything was typed, and on every
keystroke through the debounce and the request. Because that row
counts as a result, the SearchBox's own "Searching…" state never
appeared.
The channel, domain, author and subject groups now start as null,
meaning not fetched; an empty list only ever comes from the server.
The count summary tolerates the null. Tests render the hook and
check no Channels group exists before a search or while the request
is pending, and that the "No results" row and real rows still appear
after the response.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
ffmpeg wrote `<stem>.transcode.<container>` next to the source. A refresh
that overlapped the transcode discovered that file, probed the half-written
output (errors in the log), and created a FileGroup and Video for a path the
final rename removed moments later.
The temporary file is now `.<stem>.transcode.<container>`: every discovery
path already skips hidden files, and staying in the same directory keeps
the rename over the final path atomic. `transcode_tmp_path` is the single
place the name is built.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* bookmarks: user bookmarks in the navigation bar, stored in bookmarks.yaml
A "Bookmarks" tab between Calculators and Flasher opens on hover or click and
lists the user's bookmarks, with directories as flyout submenus (Menu.Sub).
The mobile hamburger shows directories as labelled groups instead. The tab
always shows, with an "Edit bookmarks" item, so the feature can be found
before any bookmark exists.
Bookmarks live in a new config-only bookmarks.yaml: a nested tree where a
node with `children` is a directory and a node with `url` is a bookmark.
Every node carries an integer id so the API can rename, move and delete
without relying on names. A URL may be a path on this WROLPi, a port on this
host (`:8096/` or `http://:8096/`, resolved against whatever hostname the
browser used, so it works over LAN IP, mDNS and the hotspot alike), or an
absolute URL. Each bookmark chooses whether to open in a new tab.
The editor at /more/bookmarks follows the playlist items table: numbered rows,
up/down/edit/delete, and inline "Add a bookmark" / "Add a directory" forms
with a Location picker. Deleting a directory asks first, since its contents
go with it. The menu and the editor are in the component gallery.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* bookmarks: mark More current when the editor is folded inside it; rem indents in the mobile menu
Found on the QA Pi at a width where Bookmarks overflows into More: the More
button carried no mark on /more/bookmarks, because the bookmarks link has no
`to` for isLinkActive to match. isLinkActive now knows the bookmarks link owns
its editor page, and the Bookmarks tab uses the same rule.
The mobile menu's directory labels sat a few pixels left of the labels around
them: the indent was in em, and a Menu.Label is set smaller than a Menu.Item.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* bookmarks: parse URLs instead of matching a prefix; validate the whole tree on every write
The URL contract was checked by prefix on both sides, and a prefix is not a
form: `//evil.com` starts like a path and `:8096@evil.com` starts like a port,
and a browser takes both off this host. A path is now a single leading slash
that resolves to this origin, a port form is an optional scheme, `:`, digits
and then only a path, query or fragment (no `@`, no backslash, no control
characters), and anything else must parse as http(s). The same rules live in
wrolpi/bookmarks.py and BookmarksContext.js, with the same bypass strings in
both test suites.
The config's dataclass validator only saw that `bookmarks` was a list, so a
hand-edited file or the generic config API could store a `javascript:` URL
that the bookmarks API would refuse, and the nav rendered it. The config now
validates the tree itself on every write and import: URL forms, exactly one of
url/children, unique integer ids, boolean new_tab, and no cycles (a YAML
anchor can build one). An invalid file is not imported and, since it was never
successfully imported, is never overwritten.
On the page, a stored URL that does not resolve is rendered as text, never as
an href. A refused add keeps what was typed instead of clearing the form, and
an edit whose move is refused still refreshes the nav (the rename was
committed) and keeps the modal open. An IPv6 host is bracketed before a port.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* help: document Bookmarks
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* app: search Other tab distinguishes pending channels from none
useSearchChannels initialised channels to [] and loading to false, and
only set loading inside its mount effect, so the Other tab's first
render painted "No Channels" with a count of 0 before the search had
run. A failed search was rendered the same way, since the hook had no
catch and the API helper returns undefined on a non-OK response.
Invariants now enforced and tested:
* null = pending, undefined = fetch failed, [] = no channels; loading
is true from the first render because the search starts on mount.
* Changing tags resets to pending, and only the newest request may
write state, so a slow response for previous tags is dropped.
* The tab shows a loader while pending, an error on failure, and
"No Channels" only once the server confirmed there are none.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* app: useSearchChannels follows its caller's tags
The hook read its tag argument only as the initial state, so the
Other tab never refetched when the filter modal changed the URL's tags
while the tab stayed mounted; the list, including a stale "No
Channels", was whatever the mount fetched.
The internal tag state now follows the argument's contents, keyed on
the serialized list because the caller builds a new array every
render. Tests change the hook argument after a loaded result and
expect a second search and a reset to pending, check that a new array
with the same tags does not refetch, and keep the slow-response drop.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* perpetual: clear in-progress flags after a crash; downloads only in the perpetual process
Found by a kill -9 stress test on a device:
A SIGKILL inside `with flags.file_worker_counting` (or any of the file
worker's progress flags) left the shared flag set, so every later refresh
raised "flag is already set" until the whole API was restarted.
reconcile_after_predecessor now clears the perpetual process's in-progress
flags (file_worker_*, global_refresh_active) and reports it.
create_downloads started a dispatch in whatever process called it, so an
API worker executed downloads (and could take transcode_lock) despite the
single-process design. It now dispatches only inside the perpetual
process (or under pytest); the download loop picks new rows up within
seconds otherwise.
The shared restart cooldown drops from 60 s to 15 s: it only has to
outlast the manager acting on a request, and a replacement that dies
right away was left dead for the rest of the minute.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* systemd: KillMode=mixed for the API so main shuts its own children down
With control-group kill, SIGTERM reaches the multiprocessing Manager at
the same moment as Sanic's main process; main's cleanup then writes
worker state to a dead Manager (broken pipe) and exits 1, so a plain
`systemctl stop` is recorded as a failure whenever a child takes a
moment to stop (a transcode being killed, for instance). Signal only
main; it stops the workers and the perpetual process itself.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perpetual: clear an in-progress flag only when its holder is dead
`with flag:` now records the holder pid in shared_ctx.flag_holders and
drops it on exit. The reconcile step clears the flags only the perpetual
process ever enters (file_worker_counting, file_worker_discovery,
global_refresh_active) whenever they are set, and the flags API workers
also enter (file_worker_modeling / indexing / cleanup via refresh_sync,
map_search_building via the map rebuild endpoints) only when the
recorded holder is dead. file_worker_busy is no longer touched: a
collection move in an API worker holds it as a mutex, and
process_queue drops a stale one itself.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* app: video comments and description distinguish pending from empty
useVideoExtras initialised both values to null and the page treated
null the same as "none", so every video open (and every prev/next)
showed "No comments have been downloaded. Refresh the video?" with a
live Refresh button until the comments arrived. The Description tab
showed "No description available." the same way. A failed fetch was
also rendered as "none", and the previous video's data lingered until
the next video's responses arrived.
Invariants now enforced and tested:
* null = pending, undefined = fetch failed, [] / '' = loaded and empty.
The API serialises a missing info_json field as null, so the hook
normalises that to empty; consumers never see null for "none".
* Switching video resets both to pending, and a slow response for the
previous video is dropped.
* The page renders a placeholder while pending, an error message on
failure, and offers Refresh only once the server confirmed there are
no comments.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* app: useVideoExtras drops any request that is not the newest
Matching on the video id let a stale request land when the user left
and returned to the same video, or saved an edit while the page's
first description request was still in flight. Each fetch now takes a
generation number and only the newest generation may write state,
per value, so a description refetch never invalidates a comments
request.
The edit dialog adopts a description that arrives while it is open
only if the user has not typed, so a draft written over the
placeholder is not wiped by the empty or failed value that follows.
Tests: 7 -> none -> 8 -> 7 with the first request rejecting last; a
save refetch beating the initial request; the draft surviving; the
pending placeholder asserted; the console spy restored in finally.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
`loading` only became true once the debounce timer fired, so for the
first 300-500ms after every sendRequest the hook reported
data=null, loading=false. Consumers branch on `loading` to choose
between a spinner and an empty state, so the global search dropdown,
the Zim search tab, and the map search rendered "No results" (or
"No Zims have been indexed") for a search that had not run yet.
Invariants now enforced and tested:
* loading is true from sendRequest until the latest request settles.
* The latest-request id is claimed synchronously in sendRequest, so a
fetch already in flight is stale the moment a newer request is sent
and cannot write data or clear loading during the new debounce.
* A stale request that rejects late no longer nulls newer data.
* A pending debounce is cancelled on unmount, and any in-flight fetch
is treated as stale.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* startup: run database config imports off the worker event loop
import_all_db_configs runs in a Sanic worker's after_server_start
listener, after that worker's socket is already accepting connections.
Every import in it was a synchronous call on the event loop, so a large
tags/channels/domains/downloads config froze that worker for the whole
import and every request the kernel routed to it hung. With the default
worker count equal to CPU count, a one- or two-core host lost all or
half of the API for minutes on startup, and the frontend's 60s request
limit turned that into repeated silent retries.
Each synchronous import now runs in asyncio.to_thread. The SQLite engine
already allows cross-thread connections (check_same_thread off, NullPool)
and other startup work already uses to_thread. Imports still run one at
a time in the documented order; only the loop is no longer blocked.
Test: a ticker task must keep ticking while two deliberately slow
synchronous imports run. It scored zero ticks before this change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* startup: thread the downloads import too; test covers every import
import_downloads_config is async but never awaited anything: its body
called the synchronous DownloadManagerConfig.import_config on the event
loop, so a large download_manager.yaml still froze the worker. The
db_up check stays on the loop and only the import moves to a thread.
The responsiveness test now patches every import the startup path
performs, including that sync downloads import, with the same blocking
sleep, and requires ~90% of the total sleep budget in ticks so any one
import returning to the loop fails it. The ticker is awaited after
cancel so the task does not outlive the loop. Docstring trimmed to the
one constraint not visible at the call site.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* files: shared modeler batch loop for video, doc, zim, archive
Paging, skip-ids, progress, and cancel yields live in
wrolpi.files.modeler.run_modeler_loop. Each module still owns select /
prepare / apply and its transaction style: video probes outside the write
lock, docs use one write per file with lock retries, zim and archive use
one write per page. Archive HTML that is not a SingleFile raises
SkipModeler so apply_indexers can claim it.
* files: quieter batch-modeler errors; drop unused imports
Break out of a batch page on InvalidRequestError after a flush failure
so the rest of the page does not each log the same poisoned session.
Re-raise PYTEST apply failures without a misleading commit-failure
message. Remove asyncio/List/OperationalError/PYTEST/Iterable leftovers
from docs, archive, and zim.
FileWorker stays the queue, status, and refresh apply path. Move planning
and execute/revert live in wrolpi.files.move; collection reorganize jobs
live in wrolpi.files.reorganize_job. Perpetual-process dispatch is
unchanged.
Existing `from wrolpi.files.worker import build_move_plan_bulk` still works.
* api: run the perpetual loops in one Sanic-managed process
The singleton background loops (file worker, downloads, switches, jobs,
bulk tag, flag checks, update check) now run in a dedicated process that
Sanic's Worker Manager owns: `run_perpetual_process` is registered with
`app.manager.manage()` from `main_process_ready`. The manager guarantees
a single instance and SIGTERMs it on shutdown. No server worker runs the
loops, so the owner-pid / heartbeat / watchdog claim scheme is deleted
rather than hardened.
`perpetual_signal` keeps its name and is now a plain registrar. A new
`per_worker_task` registrar covers loops every server worker must run for
itself: syncing its log level (previously only the owner did, so the other
workers never picked up level changes) and a health check that asks the
manager to restart the perpetual process when it is FAILED (Sanic marks a
dead managed process but does not restart it).
Downloads are started as background tasks instead of Sanic signals; the
perpetual process has no server, so no finalized signal router. The
update check and the upgrade-check endpoint run git in a thread via
`refresh_update_status`. /api/status lists the perpetual process next
to the servers.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perpetual: survive a predecessor's death, exit cleanly, coordinate restarts
Review fixes for the managed perpetual process.
Shutdown: cancel_background_tasks / cancel_refresh_tasks gather with
return_exceptions, so a cancelled child's CancelledError no longer aborts
the shutdown that called them (the process exited 1 with a download in
flight).
Restart after a crash: reconcile_after_predecessor heals shared locks a
dead process left held, fails RUNNING Jobs owned by a dead pid, clears
processing_domains, renews pending Downloads, and resets the file-worker
status before the loops start. N server workers issue one restart
request per cooldown through a shared timestamp in shared_ctx.
The log-level sync now also runs in the perpetual process. Manage() is
registered through register_perpetual_process so it is testable; the
entry point refuses to run with no loops registered (spawn).
Tests exercise the real entry point in a forked process with a live
background task under SIGINT and SIGTERM, Sanic's real WorkerManager
marking a crashed managed process FAILED and reviving it on restart,
Sanic's process naming and ProcessState enum, the listener registration,
and the reconcile step.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perpetual: import the jobs helper inside its reconcile step
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Discover/compare (find, TEMP table, SQL diff) now lives in
wrolpi.files.refresh so FileWorker is the queue plus apply/move/reorganize.
Existing `from wrolpi.files.worker import compare_file_groups` still works.
file_group.idempotency is unused since SQL compare. DROP COLUMN on SQLite
3.40 does not rebuild the table, so file_group_fts stays valid.
directory.idempotency is unchanged.
* files: pass find -printf \0 as an escape, not a Python NUL (#716)
* files: pass find -printf \\0 as an escape, not a Python NUL
'%h\\0%f\\0%T@\\0' in a Python string is an embedded null byte. subprocess
cannot pass that as an argument, so every GNU-find refresh on Linux CI
raised ValueError. find wants the two-character escape \\0 in the format
string and emits NULs itself.
* files: serialize stem backfill so a cancelled compare cannot lock SQLite
compare_file_groups runs ensure_file_group_stems in a thread. Cancelling
the task does not stop that thread, and a second compare then writes
wrolpi_kv on the same connection (tests share one session). A lock
plus a version re-check makes the second call a no-op.
* files: unique (directory, stem) after collapsing duplicate FileGroups
713 added stem and a non-unique index. This release backfills any NULL
stems, merges leftover duplicates (tags and collection membership move
onto the winner), and creates a UNIQUE index.
Upsert deletes sidecar FileGroups before INSERT so a new primary cannot
collide. Refresh deletes before upsert for the same reason.
from_paths looks up by stem so adding a sidecar does not insert a second
row. Same-stem files without a modeled primary stay one FileGroup.
* files: do not use the test SQLite connection from a worker thread
get_db_curs in pytest shares the test session connection. Running
ensure_file_group_stems and _upsert_files via asyncio.to_thread then
segfaults when the event loop commits that same connection (CI xdist).
Keep to_thread in production, where get_db_curs opens a dedicated
raw connection.
* files: keep tags when a sidecar joins an unmodeled group
choose_primary_file is the single fallback for unmodeled stem groups so
_upsert_files and _upsert_file_groups cannot pick different primaries
and DELETE the tagged row. FileGroup.move merges an existing
(directory, stem) group and flushes the delete before taking that stem.
refresh_files_directly deletes before upsert, matching the other two
callers.
* files: group by (directory, stem), not stem alone
group_files_by_stem mixed marauder/s3.bin and other/s3.bin into one
FileGroup because it ignored directory. Unique (directory, stem) then
kept only one of them, so flasher search missed a firmware file.
run_command_streaming awaits the stdout reader instead of cancelling it
when the process exits, so the last lines are not lost on a fast CI run.
* jobs: write the exception into the Job log on failure
test_jobs_failure only saw 'boom' in the log when JobLogHandler captured
logger.error(..., exc_info=e). Sanic/pytest dictConfig replaces root
handlers (especially under xdist on CI), so that path is empty even
though status and error are set. Log the exception on the Job itself
and store it in the same _update_job as FAILED.
* files: pass find -printf \\0 as an escape, not a Python NUL
'%h\\0%f\\0%T@\\0' in a Python string is an embedded null byte. subprocess
cannot pass that as an argument, so every GNU-find refresh on Linux CI
raised ValueError. find wants the two-character escape \\0 in the format
string and emits NULs itself.
* files: serialize stem backfill so a cancelled compare cannot lock SQLite
compare_file_groups runs ensure_file_group_stems in a thread. Cancelling
the task does not stop that thread, and a second compare then writes
wrolpi_kv on the same connection (tests share one session). A lock
plus a version re-check makes the second call a no-op.
* files: SQL FileGroup compare, stem column, drop unused discovery path
Refresh walked the tree twice (find, then stat every unchanged group),
and a two-directory refresh scanned the whole media directory. Compare
now diffs in SQL on a TEMP table that already has mtime, accepts multiple
roots, and groups by a stored stem that is versioned so a suffix-rule
change cannot delete-and-reinsert tagged files.
Deletes refresh_discover_paths and /api/files/refresh_progress, which
nothing in production called. Video/doc/zim modelers use the same
(no domain row OR indexed != True) gate.
* files: keep stem current on reorganize; NUL-safe find parse
Reorganize wrote the new primary_path but not stem, so the next refresh
treated the FileGroup as deleted+new and dropped tags. GNU find now
emits NUL-delimited records so a tab or newline in a filename cannot
abort a global refresh. Stem backfill skips the table scan once the
algorithm version matches, and runs off the event loop. An empty roots
list is a global refresh, not invalid SQL.
POST /api/videos/channels failed with "database is locked" on
`INSERT INTO collection` on the docker WROLPi while background workers
(config saves, refreshes) were writing. The handler wrote through the
deferred request session: `create_channel` reads (the URL/name/directory
conflict check) and then INSERTs, and SQLite refuses that mid-transaction
lock upgrade instantly when another connection wrote in between --
busy_timeout is skipped for upgrades.
Same shape and same fix as POST /api/download (daf2ba39): the handler
now uses `get_db_session(commit=True)`, which begins IMMEDIATE so the
concurrent writer waits its turn, and which commits before the refresh
is fired so the refresh does not wait on a lock the request still holds.
The regression test commits a write from a second connection between
the handler's conflict check and its INSERT. The existing
`write_lock_held_briefly` helper cannot reproduce this: the Sanic test
client takes over a second to start the app, so its lock is released
before the handler runs. Before the fix the test fails with the
production error verbatim.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Jobs queue and video transcode from the video page
Add an in-memory FIFO Job queue (wrolpi/jobs.py): functions register by
name, callers enqueue by name with JSON kwargs, and the perpetual-tasks
owner runs them one at a time. Logging emitted inside a Job and the
output of its subprocesses are captured into a per-Job log; Jobs report
progress and can be cancelled (a running Job's subprocess is killed).
Jobs are not persisted; an API restart forgets them.
Add a transcode Job and POST /api/videos/<id>/transcode. ffmpeg reports
progress via -progress pipe:1. The existing FileGroup is pointed at the
new file before the refresh so tags and history survive a container
change.
Replace the video page Refresh button with an Edit dropdown (Refresh,
Transcode). The Transcode modal pre-fills from the preferred codecs in
the Videos settings and shows the Job's progress, log tail and Cancel.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Video page: move Delete into the Edit dropdown
The Edit dropdown now holds Refresh, Transcode and Delete, so the action
row is Download / Edit / Add to Playlist. Delete keeps its confirmation
(a Confirm dialog) and the tagged-files prompt, and is disabled in WROL
Mode like the other items.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Video page: Edit > Edit to change a video's title
PUT /api/videos/<file_group_id> writes the title (and fulltitle) into the
Video's info json, creating the file when the video has none, then
validates the Video again so its title and search text follow. Files
stay the source of truth: the edit survives a refresh, and a re-download
of the metadata replaces it (the modal says so).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Video page: edit the description in Edit > Edit
PUT /api/videos/<file_group_id> now takes optional title and description
and changes only the fields given; an empty description clears it. The
modal sends only what changed and refetches the description after saving.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Transcode modal: allow a container-only remux
Keeping both streams is no longer refused: ffmpeg copies them into the
chosen container with fast start. The container select starts at the
file's own container, the hint explains what a remux does, and the button
reads Remux when nothing is re-encoded.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* WROL Mode: refuse video edits and transcodes past the API
The routes already returned 403, but a transcode Job queued before WROL
Mode was enabled still ran, update_video could be called from anywhere,
and the Edit modal's Save button stayed live. The Job now fails before
ffmpeg starts, update_video carries wrol_mode_check, and the modal
disables Save with a notice.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Transcode: verify the output before replacing the original
After ffmpeg exits, the output is probed and compared with the source
(container, presence of video/audio streams, codec per stream, duration
within 1s or 1%) and its last 30 seconds are decoded to a null output.
Any failure removes the output and keeps the original.
The tail decode replaces the download-time completeness check, which
grabbed a single frame near the end: one ffmpeg_video_complete (sync for
the downloader, async for Jobs) now serves both, and is stricter, since
every frame of the tail plus the audio must decode.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Jobs: claim atomically, cap pending, own-session subprocesses; retarget before delete
Review fixes. The worker read a Job's status and wrote RUNNING as two
steps, so a cancel landing between them was overwritten and the Job ran;
_claim_job now does both under the lock. enqueue_job refuses past
JOB_PENDING_LIMIT pending Jobs. JobContext.run_command starts the
subprocess in its own session and raises CancelledError when the command
was killed, so a handler cannot mistake it for a normal exit.
The transcode Job now retargets the FileGroup (committed) while the
original file still exists, and deletes the original afterwards
(transcode_video_file gained keep_original). A crash in between leaves
both files with the database pointing at a real one.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Video page: the Edit menu owns the transcode Job; polling survives Close and errors
Review fixes. The transcode Job's id and polling (useJob) move from the
modal to VideoEditMenu, so closing the modal no longer stops the page
from learning the Job finished (and refetching the video, whose path may
have changed). While a Job runs the menu shows Transcoding..., reopening
shows its progress instead of the form, and Delete waits. A failed poll
request is retried instead of ending the polling. getJobs checks the
response like its siblings. Design narration trimmed from comments.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Transcode: audio files, and Remove video to turn a video into an audio file
video_codec 'none' (REMOVE_VIDEO) drops the video stream (-vn, audio
required); an audio-only source is audio-only output whatever the video
target says, and a video target on it is refused. Audio output gets an
audio container (m4a/ogg/mp3) checked against the codec it will hold, and
a video container asked for audio output is replaced by the codec's
natural one. faststart only for mp4/m4a. Verification learned the
audio-only shape and ffprobe's container names (mkv reports matroska).
The modal offers 'Remove video (audio only)', switches to audio
containers and follows the audio codec, reads Extract Audio, and opens
for audio files (no video codec select).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Video page: yellow Edit button
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Transcode: remove the temp output on cancel; tell the user when a closed-modal Job fails or is lost
Review fixes. The temp output was only removed on Exception, but a
cancelled Job's run_command raises CancelledError (a BaseException), so a
cancel left the partial file for a refresh to index. Closing the modal
during a Job that then failed showed nothing; it now toasts the error
(and a cancellation). Polling gives up on a 404 or five failures in a
row (the API restarted and forgot its Jobs), warning the user and
unlocking Transcode and Delete instead of reading Transcoding... forever.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Jobs: _claim_job docstring states the invariant, not the old race
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Transcode: fragmented mp4 option for very long videos
A whole-file mp4 index grows with the frame count; for a many-hour video
it reaches tens of MB, which a phone must fetch and parse before the
first frame, and every seek walks it. `fragmented` writes the file with
-movflags +frag_keyframe+empty_moov+default_base_moof+global_sidx and
10s fragments: a ~1 KB moov plus a global seek index, so playback starts
and seeks quickly. mp4/m4a only; a remux, so it takes seconds per GB.
Measured on an 11-hour h264/aac file: moov 40 MB -> 1.4 KB, seek index
2 x 120 KB, 10k fragments, duration and codecs unchanged, remux in 25s.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Transcode modal: the Fragmented label says audio for audio output
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Video edits keep the original: custom_title/custom_description in the info json's wrolpi section
Editing a title or description no longer overwrites yt-dlp's keys. The
values go into the info json's `wrolpi` section (custom_title,
custom_description); the modeler prefers them when deriving the Video, so
a refresh keeps the edit while the original title and description stay in
the file. An empty custom description means none.
Both re-download paths carry the wrolpi section over: the metadata-only
refresh merges it into the fresh info json, and a full re-download
snapshots it before yt-dlp overwrites the file and restores it (then
re-validates) in finalize_download. FileGroup.get_wrolpi_json reads it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Refreshing a specific file (the Files page Refresh button) bypassed the
ignored_directories check that directory and global refreshes apply.
Refreshing a hard link inside the playlists directory therefore created a
second Video for the same file.
The direct-file path now drops existing files that live in an ignored
directory before comparing against the database. Deleted paths are kept
so stale records are still removed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Hotspot captive portal: welcome page with the WROLPi's address
Devices that join the hotspot probe a fixed plain-HTTP URL to check for
Internet access. NetworkManager's shared-mode dnsmasq now resolves those
probe hostnames (and `wrolpi`) to the hotspot address via an include the
Controller writes when the hotspot starts, the probes land on the Controller
(port 80), and are redirected to /portal: "This is your WROLPi, its address is
10.42.0.1, open WROLPi / open the Controller".
Only the probe hostnames are hijacked so the hotspot keeps forwarding
everything else when the WROLPi has an uplink. The include is corrected and
the connection re-activated once if NetworkManager picked a non-default
subnet. Writing the include is best-effort; the hotspot starts regardless.
`hotspot.captive_portal` (controller.yaml, default true) toggles the probe
redirects and the include; like the other hotspot settings it applies on the
next hotspot start, so saving while the hotspot is up reports
restart_required and both Controller UIs say so. The status endpoint gains
the hotspot address and portal URL, shown in both UIs. repair.sh only
creates the include directory so a disabled setting survives upgrades.
Native only: Docker has no hotspot.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: address review
* React: re-read the hotspot address whenever the HotspotToggle starts or
stops the hotspot, matching the fallback UI's refresh-on-status.
* A WPA3 start that reverts to WPA2 leaves the hotspot up, so it now gets
the captive portal address correction too.
* The corrective `nmcli connection up` logs a warning on a non-zero exit;
dnsmasq would otherwise keep the old address silently.
* HotspotStatusResponse.captive_portal defaults to true like everywhere else.
* Trim design narrative from the module docstring and React comments.
* The include-before-activate test now observes the include at the moment
of the `nmcli ... hotspot` call instead of inferring from call order.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: never forward any record type for probe hosts
`address=` only synthesizes A/AAAA answers. iOS also asks for the HTTPS
record type; dnsmasq forwarded that upstream, Apple's answer aliased to
captive.g.aaplimg.com, and the phone fetched the genuine "Success" from
Apple instead of the portal. Each probe host now also gets `local=`, so
dnsmasq answers every type itself, and the CDN alias is a probe host too.
Observed with dnsmasq query logging on a hotspot with an eth0 uplink.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: let acknowledged clients finish so the OS offers "Done"
A phone's sign-in sheet only shows "Done" once the OS re-probes and gets
the genuine success reply. The Controller now remembers each client (by
hotspot IP) that has loaded the portal page and answers its later probes
with the reply each OS expects (Apple "Success" HTML, Android 204, Windows
"Microsoft Connect Test", ...). The set is cleared when the hotspot
starts so a new session shows the page again.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: acknowledge on Continue, not on view
iOS fetches the portal page in the background before opening the sheet,
then re-probes after the sheet's own navigation. Acknowledging on view
made that re-probe return "Success" as the sheet's content. Phones only
re-check the network (and offer "Done") after a full-page navigation, so
the welcome page now has a Continue button leading to /portal/continue,
which is the only place a client is acknowledged. The Continue page says
to tap Done and links to WROLPi and the Controller.
Also hijack clients3.google.com (older Android probe host).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* help: bump wrolpi-help to the captive portal and Speed Test docs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* captive portal: comments describe the Continue design; clear acks on stop
Three comments still described acknowledging on view, which c72a7f7b
removed. Acknowledged clients are now also cleared when the hotspot
stops, so a lease reused in a later session sees the portal page.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Speed Test calculator: ping, download, and upload against the API
Users asking whether the hotspot is fast enough, or how many people can
stream video at once, had no objective tool that works offline. This
adds one under Calculators > Tools & Reference.
Backend (wrolpi/speedtest.py, /api/speedtest):
* ping: tiny no-store JSON the browser times for round trip latency.
* download: streams slices of a pre-built random buffer for a clamped
number of seconds (0.1-15). Random so no proxy or compression can
flatter the result; slices so no bytes are generated per request.
* upload: streaming-body handler that reads and discards the body and
reports bytes and elapsed time. Requests over 8 MiB are refused.
* info: the client address Caddy forwarded plus how many tests are in
flight. Sanic now trusts one proxy hop (PROXIES_COUNT = 1) so
request.remote_addr is the browser, not Caddy. Nothing else read
that field before.
There is deliberately no one-test-at-a-time lock. Several people
testing at once is how a user learns how the link shares between
streamers, so the server only counts in-flight tests (a shared_ctx
Value created pre-fork so every worker sees the same number) and the
UI reports how many others ran alongside.
Both Caddyfiles route /api/speedtest/* unbuffered, as the SSE chat
route already is, so throughput is measured live rather than in bursts.
Frontend:
* speedtest.js holds the logic with no React in it (bucketed
throughput, steady-state average that drops the first second, ping
stats, stream-capacity tiers, hotspot address check) so Jest drives
it with fake fetch and streams.
* SpeedTestCalculator runs 20 pings, a 10 s download, and a 10 s upload
with a live figure and rolling trace, then shows Mbps and MB/s, ping
verdict, the client address with a hotspot badge, and a table of how
many 480p..4K streams the download speed supports. Upload is
sequential POSTs of one 4 MiB Blob because Safari cannot stream a
request body. Nothing is saved.
* Sparkline is a new inline-SVG ui component (no chart dependency),
added to /theme-sample and checked in all four themes. A missing
sample is a gap, not a plunge to zero.
The page states that it measures the network path, not the drive that
actually serves video files.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* speedtest: bound uploads in time, close the Start race, address review
Review on #708 found three bugs and some polish:
* Start could be clicked again while the first fetchInfo() was still
pending, because the phase only left `idle` after that await. A
second click made a second AbortController and a second concurrent
run, and Cancel only reached the later one. start() now refuses when
a controller already exists and enters a `starting` phase
synchronously, so the button is Cancel before the first await.
* The upload phase could overrun the advertised 10 s by however long
the in-flight 4 MiB POST took: on a slow hotspot, the very link this
tool exists for, that is tens of seconds. runUpload() now aborts the
in-flight request at the phase deadline (counting only completed
requests) and uses 1 MiB bodies so a few-Mbps link still completes
several inside the phase.
* The server upload sink was capped by size only, so a slow or paused
body could hold a worker until 8 MiB eventually arrived. It now also
enforces MAX_UPLOAD_SECONDS (15 s, matching download) and answers
408 when exceeded. Both caps together are what "cannot hold a
worker" actually requires.
Also: a NaN `seconds` now gets a NaN message instead of "must not be
negative"; the module, Sparkline, and gallery docstrings are trimmed
to purpose; and the upload counter is covered for the mid-stream 413,
the 408, and a broken body.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* docs: browse walks the file_group date index instead of probing every doc
The unfiltered /api/docs/search drove its join from `doc`, so SQLite probed
every doc's file_group row for the sort key before returning the first page.
With 20k docs scattered through a 10 GB database on a spinning disk, the first
request took about a minute cold (0.07s warm).
Use the same CROSS JOIN shape as the archive browse: file_group is the outer
table, SQLite walks file_group_published_datetime_idx and probes doc's covering
unique index on file_group_id. Both sides stay index-only and the scan stops at
the page limit. Measured cold: ~60s -> 0.6s, identical rows.
Filtered searches and the size/title orderings keep the ORM query.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: trim browse comments, stabler plan assertion
Review feedback: drop the benchmark narrative from docstrings, state only the
non-obvious constraint at the call site, and assert the doc covering-index probe
by shape rather than by autoindex name.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* zim: recompute the outdated-Zims flag after startup and on Delete
The Dashboard shows the "Outdated Zim Files" banner from the persisted
`outdated_zims` flag, but the modal lists files from a live scan. The two
disagreed: the flag was restored from the DB on startup and never
recomputed once the outdated file had been removed, so the banner showed
with nothing to delete, and the modal displayed an endless placeholder.
- The startup recompute now runs after the DB is up and configs are
imported, so it overrides the value `init_flags` restored from the DB.
Previously it ran before the DB was up, and the restored value won.
- DELETE /api/zim/outdated always recomputes the flag, even when nothing
was deleted, so clicking Delete clears a stale banner.
- The modal re-scans when opened and says when nothing was found instead
of showing a placeholder.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* zim: enforce restore-then-scan ordering; no Kiwix restart on empty delete
- Startup calls flags.init_flags() then flag_outdated_zim_files() right
after db_up, before the DB config import, so the disk scan always
overwrites the DB-restored value in one local sequence.
- DELETE /api/zim/outdated recomputes the flag in a finally.
- remove_outdated_zim_files skips refresh and the Kiwix restart when no
file was deleted.
- Trim narrative comments to the constraint they enforce.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
COUNT(*) OVER() forced SQLite to visit every matching file_group
before LIMIT, which is a full table scan of the (large) file_group
table — ~22s cold on a Pi for videos ordered by download date.
Count separately and cache for 60s by filters so indexed orders can
stop after one page. Past-the-end pages now report the true total.
Videos/archives/docs pagers keep totalPages across page changes.
* videos: codec download preferences with transcode and strict-fail options
Add video_codecs/audio_codecs ordered preference lists to the video,
channel, and RSS downloaders, alongside the existing resolution
selector. Codec preference is codec-major: a preferred codec at a
lower resolution beats another codec at a higher resolution.
When 'transcode' is enabled, a downloaded file whose streams do not
match the preferences is re-encoded with ffmpeg after download (only
the mismatching stream; targets are extensible maps, currently h264
and aac/opus/mp3). When 'strict_codecs' is enabled and transcode is
not, a download with no matching format fails permanently instead of
falling back.
Empty codec lists produce a byte-identical yt-dlp command to before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* videos: honor strict_codecs when transcode cannot produce the preference
Review fixes:
- strict_codecs is now suppressed only when transcoding can produce a
preferred codec for every non-empty preference list (new
transcode_can_satisfy_codecs); av1-only preferences with transcode
enabled fail under strict instead of silently keeping the file.
- No partial transcode: an unfixable video mismatch blocks an
audio-only conversion (and vice versa); the mismatch falls through
to strict/keep.
- fsync the transcode output before rename, and the directory after,
so power loss on a Pi cannot leave a truncated final file.
- mp3 yt-dlp pattern matches mp4a.6B as well as mp4a.6b.
- ffmpeg maps use 0:V:0 (excludes attached pictures) and 0🅰️0?
(tolerates missing audio).
- Strict toggle is no longer disabled by transcode in the UI; popup
wording corrected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* videos: serialize transcodes machine-wide via shared_ctx lock
Sanic runs multiple worker processes, so an asyncio primitive cannot
serialize transcodes across download workers. A multiprocessing.Lock
in shared_ctx now guarantees only one ffmpeg transcode runs at a time
on the machine; waiting workers poll the lock asynchronously so the
event loop stays responsive while a transcode holds it for hours.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* videos: transcode-lock review fixes
- Document that a crashed lock holder requires an API restart; this is
accepted rather than adding owner-death recovery.
- Move the 'Transcoding' user notification to after the lock is
acquired, so it fires when work begins rather than before a
potentially hours-long wait.
- Soften the lock test docstring to what it measures (in-process
wait/release); trim the acquire docstring to the polling rationale.
- Give transcode_lock its own section header in contexts.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
test_search_videos_does_not_block_event_loop asserted a wall-clock
threshold (echo < 0.4s), which fails on loaded CI machines from
scheduler noise alone. Replace the timing with ordering: the mocked
search parks its thread on an Event, the echo is sent only once the
handler is provably inside the query, and the test asserts the echo
completes while the query is still held open. Events are only waited
on off the event loop, so a regression fails the ordering assertion
instead of deadlocking.
Verified both directions: passes repeatedly against the real endpoint,
and fails when the endpoint is changed to run the search on the loop.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* files: left-align the preview toolbar, Open joins the button row
The modal actions row packs its children to the right, which suits a
confirming button but stranded the preview's toolbar. The toolbar now
fills the row so its buttons read from the left, with Open first in
the same row as Download. On phones the tags selector always takes its
own line instead of squeezing in beside the buttons.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* tags: the add-tag button is violet like every other tag button
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* share: a real button everywhere but the navbar corner
The share trigger was a bare anchor-wrapped glyph, which matched the
navbar's other corner icons but looked out of place beside real buttons
in the file preview toolbar. It is an IconButton by default now; the
navbar keeps the bare glyph via asIcon.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* files: readable text previews in dark themes
A same-origin text document inherits the page's color-scheme, so dark
themes rendered the browser's default light text on the preview's
forced white background. Force a light color-scheme on the preview
iframe so text is always black on white.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* files: text previews follow the theme
Instead of pinning the preview iframe to a light scheme, follow the
theme: dark, night, and amber render the document light-on-dark against
the theme's panel color; light stays black-on-white.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Uploaded singlefiles are base64-encoded into JSON before being sent to the
archive container, so an 88MB upload becomes a ~118MB request. Sanic's default
REQUEST_MAX_SIZE of 100MB rejected it with a 413 before the handler ran, and the
API surfaced this as KeyError: 'singlefile'.
Only the Docker deployment is affected; Debian/RPi extract in-process.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
