Commit graph

2276 commits

Author SHA1 Message Date
Roland
dc811d08e3
ui: put the card mimetype accent on the bottom edge
Roland's call, and the better one: along the top the accent sat immediately
against the poster's own top border, so the two edges competed.  On the bottom it
closes the card and reads as a property of the whole thing rather than a header
rule.
2026-08-05 18:36:28 -06:00
Roland
31d20c04da
ui: bring back the card mimetype accent
Two agents independently dropped it, each because our Card had no `color` prop
and each citing the other's precedent -- which is how a deliberate feature
quietly becomes a convention.  Roland wanted it back.

Card takes a token colour name and draws it as a 3px edge along the top rather
than tinting the surface: surfaces stay flat per the design rules, and an edge
survives night mode without becoming a bright patch.  Because it is a token, the
accent remaps per theme -- four hues in light and dark, a single one in night and
amber, which is the point of those themes.

Restored on every card that shows a file: the video, image and generic file cards
in Files.js, and the archive card.  All of them take it from
`mimetypeColor(mimetype, path)`, the same helper that colours the file icons, so
a card and its icon agree.
2026-08-05 18:36:27 -06:00
Roland
825e5ac768
ui: the last Semantic UI file, and the stylesheet with it
ControllerPage was the final call site; Theme.tsx then had nothing left to wrap,
so it drops from 861 lines to 338 -- the provider that resolves the theme, stamps
`data-theme` and `data-media-filter`, persists both and hands Mantine our tokens.
Everything below that was Semantic wrappers.

The dark-mode compatibility props went with them.  `i`, `s`, `t` and `inverted`
existed so unmigrated components could borrow Semantic's dark treatment, and
they carried hardcoded greys -- #1B1C1D, #dddddd, #eeeeee -- which is exactly
what the token themes replaced.  Three call sites in Common.js were still
injecting the #eeeeee one into links.

Then `semantic-ui-offline/semantic.min.css` came out of App.js.  That took more
with it than expected, and looking at the running app is the only reason it was
caught:

  Every link lost its colour and gained the browser's underline and visited
  purple.  The dashboard's load averages and section headings were underlined
  and purple.  Links now take `--blue`, which remaps per theme, and underline
  only on hover -- many of them wrap cards, posters and icons.

  `box-sizing: border-box` disappeared, which silently changes the size of every
  padded element in the app.  Semantic's reset had been providing it.

  The navigation bar overflowed a 1000px viewport with `nowrap` and cut "Admin"
  off the right edge -- the one link a user needs to reach the settings that fix
  a broken WROLPi.  Semantic's menu wrapped; ours does now too.

The gzipped CSS bundle is 47.5 kB, and one call site was still leaning on
Semantic's grid classes -- now a flex column, since all three of its callers
pass library components.

The ratchet stays rather than being deleted.  At zero it asserts four things: no
file imports Semantic UI, none calls its toast library, none leans on its
stylesheet, and no file reaches back through Theme.tsx.  It can go when the
packages are uninstalled.

Theme.test.js is deleted, but not before porting the two behaviours it
protected: that Button forwards a ref to the real DOM element (a class-component
ref once crashed the Control page, because portals call `node.contains`), and
that the progress percentage stays readable on every bar colour.
2026-08-05 18:36:26 -06:00
Roland
7272c9c552
ui: migrate ControllerPage.js off Semantic UI onto the token component library
Replaces semantic-ui-react and Theme.tsx imports with components from
components/ui (Panel, Table, Modal, Confirm, Status, Select, TextInput,
Checkbox, IconButton, Message). Preserves every confirmation, disabled
state, and polling interval; service status now maps onto Status'
complete/active/pending/failed kinds instead of a raw color. Repairs
ControllerPage.test.js to render inside MantineProvider and to find the
unmount Confirm button by dialog role instead of DOM-order position,
since Mantine portals the modal ahead of the page content. Drops
ControllerPage.js from semantic-baseline.json now that it's migrated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 18:36:23 -06:00
Roland
f3dbe20be5
ui: Nav off Semantic, and the memory warning starts working
The navigation bar was the last visibly-Semantic surface, so this finishes the
visual migration.  Its two fragile features survive intact: the navbar colour
still comes from the user's `nav_color` setting, now as `var(--<name>)` since
our token tables define all fourteen Semantic colour names, with text taking
`--btn-text` so every theme stays legible; and the warning-icon contrast swap
is unchanged, including the case where `lowWarningColor` is null and must mean
"no override" rather than `var(--null)`.

Fixed a pre-existing bug the migration exposed: the memory indicator passed its
colour to `Link`, which ignores it, so that icon never changed colour -- 81% and
95% RAM looked identical.  Every sibling indicator colours the Icon; now this
one does too.

The agent that migrated this file also gave names to the warning indicators,
which were unlabelled icons inside links, so a screen reader now announces what
each one is.  The three data-testid hooks are unchanged.

Baseline 15 -> 2.  Only ControllerPage and Theme.tsx remain; Theme.tsx is last
by design, since it can only go once nothing imports its Semantic wrappers.
2026-08-05 18:35:48 -06:00
Roland
c8b27b625d
ui: Nav.js off Semantic UI onto the token-driven component library
Replaces semantic-ui-react's Menu/Dropdown/Icon/Popup with plain flexbox
elements plus the Mantine-based Menu (for the mobile hamburger and desktop
"More" overflow), Tooltip, and Icon from ./ui. Nav color and the
warning-icon contrast swap keep their exact logic, now resolved through
CSS custom properties (var(--{navColor}), var(--btn-text)) instead of
Semantic color props. Adds accessible labels to previously-unlabelled
warning icons while preserving their data-testid hooks and threshold/
priority logic. New nav CSS lives in App.css (wrolpi-navbar*).
2026-08-05 18:35:47 -06:00
Roland
fa1715523e
ui: migrate Flasher.js off Semantic UI React
Replace semantic-ui-react/Theme.js imports with the token-driven ui/
library: Tab -> Tabs, Segment -> Panel, Message/Label/Progress/Loader ->
their ui equivalents, Table.* compound API, Confirm-free flows preserved
exactly. Remove obsolete ThemeContext i/s/t spreads. Drop Flasher.js from
semantic-baseline.json now that it no longer imports Semantic UI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 18:35:45 -06:00
Roland
7de1bead98
ui: three silent library defects the agents' reports surfaced
Each of these was reported by an agent working around it rather than patched in
one file, which is why all three are fixed once, for every call site.

Button accepted an `href` and dropped it.  Mantine renders an anchor only when
it also gets `component='a'`; Semantic spelled that `as='a'`.  So a migrated
call site that kept just the href produced something that looks like a download
link and navigates nowhere -- silently, with no warning.  Button and IconButton
now infer the anchor from the href.

The media filter ran twice on nested media.  A `.media` wrapper filters its
subtree as one unit -- the audio player needs that, because its control glyphs
are SVG and the element-type rule does not reach them -- but the poster image
inside also matched that rule.  Two passes of a luminance-to-red matrix is not
one: the second takes 0.2126 of an already-red pixel, so the poster came out
markedly darker than everything around it.  Leaf media inside a filtered
wrapper is now left alone.

Progress had no indeterminate state.  Semantic's `indicating` animated a stripe
when the size was unknown; without it an upload that has not reported yet sits
at 0% and reads as stalled -- and `aria-valuenow="0"` tells a screen reader the
same wrong thing.  It now animates, reports no value, and falls back to a
static fill under prefers-reduced-motion.

Also: night and amber now override an inline `--label-color`.  A call site may
legitimately need many distinguishable colours -- the Flasher's chip badges use
a twelve-colour palette no four tokens can express -- but on a night-adapted
screen a green or blue chip badge defeats the entire mode.  The labels lose
their distinctness there, which is the right trade.
2026-08-05 18:35:43 -06:00
Roland
93f38b1c4c
ui: migrate Archive.js off Semantic UI React
Replaces every semantic-ui-react and ./Theme import with the token-driven
./ui library: Segment->Panel, Tab/Tab.Pane->Tabs (Tabs.List/Tabs.Tab/
Tabs.Panel), Loader active inline='centered'->Loading, Message/Message.Header
->Message kind/title, Statistic/StatisticValue/StatisticLabel->Statistic
value/label, Card/CardContent/CardHeader/CardMeta/CardDescription->Card
media/title/meta/children, Input label={...}->ActionInput action={...},
Form.Dropdown->Select, Form/Form.Field/TextArea->plain markup + Textarea.

Converts all three Semantic grid spellings (Grid.Row/Grid.Column, bare
GridRow/GridColumn) to flexbox for the two-up rows in ArchivePage and
DomainsPage, and to plain "row"/"column" divs (CollectionEditForm's existing
CSS-class convention, shared with still-unmigrated Channels.js/Playlists.js)
for the fields passed into CollectionEditForm in DomainEditPage.

ArchiveFileNameForm no longer goes through hooks/useForm's InputForm helper,
which still renders Semantic's Form.Input internally (hooks/useForm.js is
off-limits for this migration) -- it now builds its field directly against
form.getInputProps, mirroring the pattern Download.js already established.

Drops ThemeContext's obsolete i/s/t/inverted spreads throughout, and the
mimetype-color accent on Card (the new Card has no color prop; Videos.js's
already-migrated VideoCard dropped the same accent for the same reason).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 18:35:40 -06:00
Roland
21eda2950a
ui: FileBrowser.js and FilePreview.js off Semantic
Migrated both files onto the token-driven ./ui component library: Table,
Modal, Button/IconButton, Checkbox, Header, Panel, Progress, Placeholder,
Skeleton, ActionInput/TextInput, IconStack, Menu/ButtonGroup for the slicer
split-button, and native <img>/<form> in place of Semantic's Image/Form.

Notable adaptations:
- Semantic's per-row `disabled` styling (no visual equivalent in the new
  Table) is reproduced with inline opacity/pointer-events instead.
- DirectorySearch's autoFocus now passes straight through as a prop instead
  of a nested `input={{autoFocus}}` object, matching SearchBox's API.
- The responsive icon+label footer buttons keep a single Button and toggle
  children instead of swapping component type.
- FileBrowser.test.js: two upload-button assertions keyed off a literal
  Semantic `.green` class; broadened to also match on the icon, mirroring
  the sibling assertion already using that fallback.

Preserved as-is (reported, not fixed): getVideoPreviewModal's
`<Header as='h5'>{path['path']}</Header>` indexes a string with `['path']`,
which was already dead/undefined before this migration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 18:35:36 -06:00
Roland
3eeaa951ce
ui: migrate Downloads, Upload, and BulkTagModal off Semantic
Ports admin/Downloads.js, Upload.js, and BulkTagModal.js to the token-driven
ui library: Table/Modal/Progress/Message/Button/IconButton replace their
Semantic and Theme.tsx equivalents. Failed download rows take the dashed
danger border via Table.Row's `failed` prop; retry/restart actions use
role='retry' and delete/stop actions use role='danger' instead of hardcoded
colors. RecurringDownloadsTable keeps its export name and props for
Archive.js. Updates BulkTagModal.test.js to render through the real
Mantine-backed components instead of mocking the retired ./Theme module, and
drops the three files from semantic-baseline.json now that they no longer
import Semantic UI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 18:35:32 -06:00
Roland
bb61a3a6a3
ui: migrate Docs.js and Zim.js off Semantic UI React
Replace semantic-ui-react and ./Theme imports with ./ui across both
files: Grid -> Group/flexbox (two-up rows), Loader -> Loading, Segment
-> Panel, Statistic/StatisticLabel/StatisticValue -> Statistic +
StatisticGroup, Tab -> Tabs, Accordion -> Mantine Accordion.Item with
controlled string value, Dropdown -> Select, Message/Label/Placeholder
-> ui equivalents. Buttons take role (primary/save/retry/danger/cancel)
instead of Semantic color names where a role fits.

Zim unsubscribe now goes through a destructive Confirm (via APIButton's
confirmContent) instead of firing immediately, matching the
Confirm+destructive rule for irreversible actions; the outdated-Zims
delete flow keeps its rich custom Modal (dynamic file lists) but its
trigger and confirm buttons are styled destructive and labeled "Delete".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 18:35:27 -06:00
Roland
d70a8733b9
ui: migrate VideoPlayer.js off Semantic UI React
Replaces semantic-ui-react and ./Theme imports with the token-driven ./ui
library: Segment->Panel, Tab->Tabs (Tabs.List/Tab/Panel, keepMounted=false
to preserve per-tab lazy render/fetch), Loader active->Loading, Input->
TextInput, Grid/Grid.Column->Mantine Grid/Grid.Col, Container->plain div,
color=/APIButton->role=. Comment/CommentGroup and Transition have no ui
equivalent, so the comment thread and audio-poster fade are rebuilt from
plain elements/tokens. The audio poster wrapper (custom play/pause/volume
glyphs drawn over the poster image) gets .media so night mode tints the
composite as one unit; the video/audio elements and poster img stay
untagged since they are already filtered leaves. Added ARIA button
semantics to the poster click target since it has no native href/button.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 18:35:17 -06:00
Roland
f1056d1409
ui: Channels.js and Playlists.js off Semantic
Migrates both files onto the token-driven component library: Grid ->
Mantine Grid/Grid.Col, Segment -> Panel, Message/Header/Modal/Statistic
compounds -> ui equivalents, Semantic Form/Input -> plain form +
TextInput, and button colors -> role. Confirm/delete flows keep their
guards and now report through Button role='danger'/APIButton role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 18:35:10 -06:00
Roland
826b37527b
ui: fix the cypress failures, and two real defects behind them
All 43 cypress specs pass now (was 36 of 43).  Five of the seven failures were
stale selectors, two were real.

Real, and both mine:

  The search modal stopped focusing its input.  Mantine mounts modal content on
  a later tick, so the effect that focused it ran while the ref was still null;
  Semantic's modal mounted synchronously, which is why focusing there had been
  enough.  Fixed declaratively -- the field asks for focus itself, and SearchBox
  also sets `data-autofocus`, which is what Mantine's focus trap honours when it
  takes focus away from a plain autoFocus.

  Escape closed the suggestion list and the enclosing modal at once, so a user
  correcting a mistyped query lost the whole modal.  Escape now dismisses the
  suggestions and stops there; a second Escape, with the list closed, still
  closes the modal.

Stale selectors, all in the specs: Semantic's icon font (`i.search.icon`) is
SVG now, its `.ui.modal`/`.ui.dimmer` are the library's modal and overlay, and
one test asserted a button carried the classes `ui button secondary`.  That last
one is rewritten to check what a user can actually perceive -- that Edit is a
real link pointing at the edit route -- because framework class names say
nothing and break on every library change.

Two accessibility gaps the tests walked into:

  The nav's search control was an anchor with an onClick, no name and no
  keyboard path.  It now has a name, a role and Enter/Space activation, which is
  also what let the spec stop matching on icon-font classes.

  The search dialog had no accessible name at all; it has one now.

One `{force: true}` remains, with its reason recorded: Cypress judges that
field covered because it sits in a fixed modal under a sticky header, and I
verified by hand at Cypress's own viewport that it is the topmost element at
its centre, unobscured, and focused on open.

Also closed a hole in the ratchet that this uncovered.  A file could drop its
`semantic-ui-react` imports, leave the baseline, and still render Semantic
through Theme.tsx's wrappers -- which is exactly how a Semantic modal survived
in the search shortcut long after that file looked migrated.  A test now
forbids it, and the three files doing it (Calculators, KeyboardShortcutsProvider,
and Common.js reaching for theme names) import from the library or from
themes/names instead.
2026-08-05 18:35:07 -06:00
Roland
4349ac6301
ui: keep the unmigrated pages' notifications working
Greptile P1, and a real one: App.js migrated and took `<SemanticToastContainer/>`
with it, but seven pages -- Archive, Channels, Docs, FilePreview, Playlists,
Zim, ControllerPage -- still raised their toasts through the Semantic library.
With no container mounted those notifications went nowhere.  No error, no
warning; a user deletes a document and is simply never told whether it worked.

Fixed by pointing those seven at the library's `toast`, which takes the same
options, rather than by mounting a second toast system.  Nothing calls the
Semantic toast library now, so a test asserts that at zero: a file that goes
back to it would raise notifications that silently never appear, and that is
not something a reviewer would spot.

Also reverted Archive.js to its last good state.  An interrupted agent had
migrated part of it -- Panel, Tabs, and Mantine's Grid.Col -- while the rest of
the file still used Semantic's Segment, Tab and Grid.Row, so a single `Grid`
identifier had to be two different components at once.  A half-migrated
1400-line file is worse than an unmigrated one; it goes back on the list and
gets done in one piece.

That break was masked: react-scripts caches ESLint results per file, so the
earlier build passed while Archive.js sat on a stale pass.  Editing its import
invalidated the entry and the errors appeared.  Verified with the cache cleared
this time, which is the only way that check means anything after a wide change.
2026-08-05 18:35:03 -06:00
Roland
929534e455
ui: thirty-five files off Semantic, leaving fifteen
Phase 2, part 4.  Baseline 50 -> 15.

Migrated: all eight calculators, the rest of inventory and collections, the
Map/Zim/Docs cluster, Videos, Files, Search, Archive, DashboardPage,
DatesSelector, HelpModal, Events, Tags, Flasher's neighbours, admin Settings
and Status, Download, and App.

Also the three shared modules everything imports: api.js and customHooks.js
swap the Semantic toast library for the library's shim (same call shape), and
useForm.js renders the library's Message.  useForm accepts both the old
`{content, header, positive|negative}` shape and the new `{kind, title}`,
because the pages building those objects migrate separately.

Two library gaps closed, both found by call sites needing them:

  ActionInput is a forwardRef now.  Its ref reaches the input rather than the
  wrapper, because callers use it to focus or select the field -- a ref to the
  surrounding div would do neither, which is why one call site had resorted to
  hand-copying the component's markup.

  Header takes a `color` token name.  Three separate call sites had reached for
  an inline style to colour a heading that carries meaning ("3 items to add"
  against "2 to remove"), which is how a hex ends up in markup.

Two files needed repair after being interrupted mid-edit: MapViewer had a
Segment left in its error branch, and VideoPlayer had lost the useContext
import its own code still used.  Neither jest nor tsc noticed; only
`npm run build` did, which is the whole reason that check exists.

Fifteen files remain: Archive, BulkTagModal, Channels, Docs, FileBrowser,
FilePreview, Flasher, Nav, Playlists, Theme.tsx, Upload, VideoPlayer, Zim,
ControllerPage, admin/Downloads.  Theme.tsx is deliberately last -- the
unmigrated files still import its Semantic-backed wrappers, so it can only go
once they have.
2026-08-05 18:34:58 -06:00
Roland
737056acf5
ui: fix search suggestion selection, rendering, and clipping
Three findings from review and CI on the Common.js migration, all real.

Selecting a suggestion navigated nowhere.  Semantic handed its callers
`{result}` and every caller destructures that shape, so passing the bare result
left `result.location` undefined -- the search suggestions and the Ctrl+K modal
both stopped navigating.  The shape stays until the call sites can change
together.  (Greptile P1.)

Suggestions lost their custom markup.  Callers pass `resultRenderer` to draw a
tag as a chip rather than a line of text, and the migrated wrapper forwarded it
into a div's props instead of to SearchBox, so every suggestion fell back to the
generic title and description.  SearchBox takes it now.  (Greptile P2.)

The suggestion list clipped its last group.  A cypress spec caught it: with
directories, channels and domains all present, the content runs past the 320px
cap and the final row was cut off.  Eight grouped suggestions is an ordinary
result set, so the cap was simply too low; it is viewport-relative now and
still bounded.

Both regressions are pinned by tests -- one asserting the callback shape a
caller actually destructures, one asserting the custom renderer replaces the
default markup rather than wrapping it.
2026-08-05 18:34:54 -06:00
Roland
6ea9d3645f
ui: Common.js and thirteen more files off Semantic
Phase 2, part 3.  Baseline 63 -> 50.

Common.js is the one that mattered: 51 files import it, so its Semantic usage
was propping up the rest of the migration.  Everything in it now comes from the
library -- Paginator, the two search inputs, DirectorySearch, TabLinks, the
toggles, the messages, APIButton, Breadcrumbs, SimpleAccordion, IframeViewer,
CardPoster, FileIcon, LoadStatistic, the theme menu, and the hotspot icons.

Also migrated: Placeholder, Apps, ArchivePreview, CbzViewer, AddToPlaylist,
TaggedDeleteConfirmModal, the collection edit/tag forms, and four inventory
components.

New in the library, all in the gallery and tested:

  SearchBox     Semantic's `<Search category>` with its grouped results shape
                kept, because four call sites already build it.  A real
                combobox rather than a div that listens for clicks.
  Pagination    Semantic's prop names; onPageChange gets the page number.
  TabBar        the strip only -- routing stays with the caller.
  IconStack     Semantic's IconGroup, named once for assistive technology.
  Table.HeaderCell sorted/onSort -- a real button carrying aria-sort.
  Button/IconButton translate Semantic's size names.

That last one is defensive: unmigrated callers still pass size='tiny', and
Mantine drops a size it does not recognise without a word, so the button
quietly renders at the wrong size.  Mapping here keeps the mixed state honest.

Three real bugs found and fixed on the way, none of them cosmetic:

  * `fileSuffixIconName` returned fourteen icon names the map did not have, so
    opening a folder with a PDF, a Word document or an archive in it logged an
    error and drew a fallback glyph.  All fourteen mapped, with a test that
    walks the mapper's own return values, and another asserting every mapped
    name resolves to a Tabler component that exists.
  * `Statistic` relied on Semantic's `Statistic.Group` for its spacing, so on
    the dashboard the load averages ran together into "1 MIN. LOAD5 MIN.
    LOAD15 MIN. LOAD".  It carries its own spacing now.
  * Semantic's inverted stylesheet colors every descendant of an inverted
    segment, so Toggle labels inherited #dddddd -- white pixels in night mode,
    on the Settings page, which is exactly where someone adjusts their WROLPi
    in the dark.  Restated from tokens.

The baseline ratchet gained a third assertion: a migrated file may still lean
on Semantic's CSS classes without importing the library, which the import check
cannot see and which would collapse silently when Phase 4 deletes the
stylesheet.  There is one such bridge today and it is now declared.
2026-08-05 18:34:50 -06:00
Roland
d359eea2d3
ui: nine leaf files off Semantic, and the components they needed
Phase 2, part 2.  Baseline 72 -> 63.

Migrated: Tables, SortableTable, DonatePage, Share, Headline, ErrorBoundary,
and the QR code, VIN decoder, and temperature calculators.

New in the library, all in the gallery and tested:

  SearchBox     Semantic's `<Search category>`, keeping its grouped `results`
                shape because four call sites already build it.  A real
                combobox: arrows move, Enter takes the highlighted suggestion
                or submits the raw text, Escape closes without clearing.
  Pagination    Semantic's prop names, but onPageChange gets the page number
                rather than an (event, data) pair.
  TabBar        the strip and its house style only; routing stays at the call
                site, which is what decides what is active.
  IconStack     two glyphs composed into one symbol, named once for assistive
                technology (Semantic's IconGroup).
  Table.HeaderCell gains `sorted` and `onSort`.

That last one is a fix, not just a port.  Semantic's `sorted` prop set
`aria-sort`; migrating a table by hand dropped it, leaving a clickable <th>
that no keyboard could reach and no screen reader could describe.  Putting it
in the library means the next table gets it right without anyone remembering.

Night mode holes closed along the way: the QR codes on the donate, share, and
calculator pages are white by necessity — a scanner needs the contrast — and
they render as SVG, which the media filter does not target.  In night mode that
was a lit panel.  They now carry `.media`, so the filter remaps the block as a
unit and the contrast survives the mapping.

Also: `paypal` joins the icon map rather than being imported ad hoc in a page,
and FileBrowser stops passing Semantic-only table props into a DOM element.
2026-08-05 18:34:49 -06:00
Roland
2ee81fae14
ui: media filtering is a per-theme setting, not a night-mode hardcode
Filtering media was keyed on `data-theme="night"`, which made it a property of
the theme that no user could turn off.  Some people want their media tinted to
match the interface; some want to see the actual image.

A theme now declares a filter and whether it starts on, and the user decides:

  night   night-red    on by default.  An unfiltered thumbnail undoes dark
                       adaptation, which is the whole point of the mode.
  amber   amber-mono   off by default.  Amber is a look, not a night-vision
                       aid, so media keeps its real colors until asked for.
  light   -            no filter offered, so no toggle is shown.
  dark    -

The choice is stored per theme, so wanting amber's tint says nothing about
wanting night's.  The toggle lives in the theme picker, which puts it on the
Settings page and in the gallery at once.

Mechanically the rules now key on a `data-media-filter` attribute rather than
`data-theme`: a rule keyed on the theme could not be turned off.  index.html
stamps that attribute before first paint alongside data-theme -- without it,
night mode paints a screen of unfiltered thumbnails while the bundle arrives
over a hotspot, costing the user the adaptation the theme exists to protect.
Tests cover both the attribute rule and the fact that the inline script's
duplicated ids and defaults still agree with names.ts.

The amber matrix is the same luminance collapse tinted to #ff9500 (red full,
green 0.584, blue zero) with a gentler gamma than night's: amber carries far
more luminance and clips to a flat wash otherwise.

The gallery gains a Media filtering section showing the WROLPi logo filtered,
beside an unfiltered copy that opts out with `night-unfiltered`.  That copy is
behind a click on purpose -- an unfiltered image is a bright patch, and night
mode should not spring one on someone reading in the dark.

ui-design.md gains the media filtering contract it never got in Phase 0,
including the note that a Phase 3 custom theme names an existing filter id and
cannot ship its own matrix, since that would be code rather than data.
2026-08-05 18:34:47 -06:00
Roland
528a5702d0
ui: migrate the extension pages and Configs; five components they needed
Phase 2, part 1.  Three files leave Semantic UI (baseline 75 -> 72), and the
components they needed join the library and the gallery:

  Header       Semantic's shape (as/icon/subheader/dividing).  47 files use
               <Header>, and the level now picks the size, so the type scale
               stays a scale instead of each call site choosing a font size.
  Message      gains an icon and a dismiss button.
  ActionInput  input and button sharing one outline, for <Input action={...}/>.
  Loading      centered loader with a caption, replacing the
               <Loader active inline='centered'> + wrapper div in 19 places.
  Modal        becomes compound - Modal.Header/.Content/.Actions, plus
               Semantic's open, closeIcon, and size names - because 34 files
               are written against that shape.

test-utils mounts MantineProvider with ThemeProvider's exact configuration:
migrated components render Mantine internals and every one of their test files
would otherwise need its own provider.

One Semantic bleed found and fixed: semantic.min.css sets
`h1,h2,h3,h4,h5 { font-family: Lato }` and loads after Mantine's element rule,
so headings ignored the theme's body face.  Only visible in amber, where the
body is mono and the headings were not.  A class-level restatement outranks it;
the rule comes out when Semantic does.

Verified: 925 jest tests, clean production build, and a night-mode audit of
every painted pixel in the gallery (red-only, one known false positive).  The
Configs backup and restore flow was exercised against the real API.
2026-08-05 18:34:46 -06:00
Roland
a5d2bd7084
ui: night mode actually filters media, and fix three picker icons
Addresses the automated review on #583; all three were real.

The night filter only applied to elements tagged `.media`, and nothing in the
app carried that class - so selecting Night filtered the interface but left
every video, thumbnail, PDF and map full-colour, which is precisely the thing
the mode exists to prevent. The Settings copy promised otherwise, so it was
also wrong.

It now targets the element types instead of an opt-in class, because night mode
has to fail safe: a library of thousands of files reaches the screen through
many components, and one that forgot to opt in would undo half an hour of dark
adaptation. `.media` still works for wrappers that must be filtered as a unit,
and `.night-unfiltered` opts out. The filter is restated on `:fullscreen`, which
a filter on an ancestor stops reaching. Verified on the videos page: all 25
media elements filtered.

Three of the five theme choices rendered a fallback glyph and logged errors,
because the icon map was built by grepping literal name='...' strings and the
navigation picker passes its icon through a variable. Adds sun, moon and
lightbulb, plus a test asserting every theme choice resolves to a real icon.

The pre-paint script wrapped the `matchMedia` fallback in the same try as the
storage read, so a device with storage blocked painted light even when it
prefers dark - recreating the flash the script exists to prevent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 18:34:46 -06:00
Roland
95c089edc4
ui: theme picker in Settings, and ship the component gallery
The gallery at /theme-sample is now a normal route rather than a development
one, and Settings links to it, so a user can see what a theme actually looks
like before committing to it. Every new component belongs in the gallery: it is
how the design gets reviewed, and how a component that only looks wrong in one
of the four themes gets caught.

Settings gains a Theme section with the new ThemePicker, which explains what
each theme is for - notably that night mode filters videos, documents and maps
to red as well, not just the interface.

Theme names and picker options move to themes/names.ts so the component library
can read them without importing components/Theme.tsx, which still pulls in
Semantic UI. Theme.tsx re-exports them, so existing call sites are unchanged.
The navigation bar's compact picker now reads the same list, so the two cannot
drift apart.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 18:34:45 -06:00
Roland
fb4c03d7a0
ui: token-driven component library on Mantine
Adds src/components/ui: buttons, messages, labels, progress, status, tables,
form controls, panels, cards, statistics, modals, tooltips, tabs, and icons.
Everything resolves its colors through theme tokens, so no component branches
on the current theme, and Semantic UI keeps rendering the app meanwhile.

Mantine is bridged rather than restyled: its own CSS variables are pointed at
our tokens. Four details were not obvious, and each caused a visible defect
in night mode before being fixed:

  - Structural colors must go in the resolver's light/dark blocks. Mantine
    emits its values under [data-mantine-color-scheme], which outranks the
    :root block that `variables` becomes, so its grays were winning.
  - The dark and gray palettes are mapped shade by shade to tokens. Mantine
    reads them directly for table borders, zebra stripes and input chrome;
    they can be neither flattened nor left alone.
  - Mantine writes --button-* as inline styles, which no stylesheet can
    outrank, so the dashed danger treatment sets real properties instead.
  - Switch colors are written onto inner elements, so they need scoped CSS
    rather than a variable on the root. Adds a --knob token for control
    handles, because Mantine's default thumb follows theme.white, which is
    near-black in the dark-based themes.

An audit of every painted pixel in the /theme-sample gallery now finds no
non-red channel in night mode.

Icons come from @tabler/icons-react with the 102 Semantic names the app uses
mapped to components; they stroke with currentColor, so they follow the theme
without any rule of their own. Toasts keep the react-semantic-toasts-2
signature so those call sites migrate by changing an import.

TypeScript 5 is required: Mantine's declarations use `const` type parameters,
which 4.9 cannot parse (Mantine 7 is the same, so downgrading is no escape).
That needs two npm overrides to keep clean installs working, both verified
against a Docker build: react-scripts declares peerOptional typescript ^3||^4,
and fork-ts-checker 6.5.2 assigns typescript.performance.mark, which TS 5
made getter-only - crashing the dev server's type checker on every rebuild.

semantic-baseline.test.js ratchets the removal: it fails if a file not already
listed imports Semantic UI. An ESLint rule would have printed 75 warnings per
rebuild, and CI unsets CI= for its build step, so it would guard nothing there.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 18:34:44 -06:00
Roland
e6d34b1be1
ui: theme tokens and a four-theme runtime
Every color now resolves through CSS custom properties on
html[data-theme=...], so a theme is a table of values rather than code.
Four themes ship: light, dark, night (red hues only, to preserve dark
adaptation off-grid) and amber (monochrome amber terminal).

Fonts are exposed as two roles, --font-body and --font-mono, which a theme
assigns a bundled face to; amber uses the mono face. Faces must be
self-hosted, never fetched from a CDN, so the app keeps working offline.

Night mode also filters media that cannot consume tokens - video, images,
PDFs, embedded pages, canvases - through an SVG color matrix that maps
luminance onto the red channel. Tag such an element with `.media`; a filter
on an app-level ancestor would create a containing block and break fixed
positioning.

ThemeProvider persists the choice and index.html stamps data-theme from
localStorage before the bundle loads, so the first paint already matches the
saved theme on a slow connection. The OS preference is followed only while
"system" is selected; night and amber are explicit choices. The nav toggle
becomes a five-item picker.

Semantic components still render themselves, so only the shell is
token-driven for now; the inverted props stay in place until each component
is migrated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 18:34:41 -06:00
Roland
4b9593580d
fix: a video download requests one format, not every matching format
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
The yt-dlp `-f` selector was built by joining every selector of every
requested resolution with a comma.  In yt-dlp a comma means "download all
of these formats"; `/` is the fallback operator.  The resolution map is
ordered best-to-worst and its entries already use `/` internally, so the
comma was never the intent.

With the default resolutions this handed yt-dlp ~45 comma-separated
selectors and asked it to download every one that matched.  A 720p video
matched 8 of them (including duplicates, which yt-dlp does not dedupe), so
the whole per-download pipeline ran 8 times against the same output path:
subtitles downloaded, thumbnail downloaded, info.json written, subtitles
converted, thumbnail converted, existing subtitles deleted, repeat.  The
duplicate work was invisible because each pass overwrites the last.

Join with `/` so the resolution list behaves as the preference chain it
reads as.
2026-08-05 17:25:19 -06:00
Roland
b9a892b203
fix: act on review — don't hold the write lock across the source-tree walk
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
Two review points, both correct.

The planning session was opened before the filesystem collection that runs inside it, so
BEGIN IMMEDIATE held the write lock across every glob and directory walk - trading an
instant abort for minutes of global write starvation on a large channel.  The collection
now finishes before the session opens; only the temp-table load, the FileGroup reads and
inserts, and the commit run under the lock.  `file_source_set` is built during that same
pass instead of re-globbing every file source a second time.

`test_file_worker_move_survives_concurrent_writer` held the lock across the whole move, so
planning waited it out and the execute phase usually met no contention - the test would
have stayed green if only the execute site regressed.  It now takes the lock after planning
returns.  Verified by mutation: reverting either site alone fails its own test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 19:53:44 -06:00
Roland
dbbcab4d90 fix: a concurrent writer no longer aborts a file move with "database is locked"
Tagging a Channel with "move files" checked failed the whole move: the Collection's
directory was updated, but its files stayed put.

`build_move_plan_bulk` reads FileGroups and then INSERTs the ones missing from the DB,
inside a transaction that begins *deferred*.  SQLite refuses a mid-transaction lock
upgrade immediately — busy_timeout is skipped for upgrades to avoid deadlock — so any
concurrent writer (a refresh, a download, a config save) aborted planning outright.
The same pattern in `_execute_move_chunks` fails the same way on its Directory INSERT.

Both transactions now begin as writers, so busy_timeout absorbs the contention.  That
requires the writes nested inside the move to share its transaction rather than open a
second connection, which would deadlock against the lock the move already holds:
`_bulk_update_file_groups_db` now runs on the caller's session, and the old-directory
cleanup deletes its Directory record through that session instead of via
`delete_directory`.  As a side effect, a failed move no longer leaves earlier chunks'
path updates committed after their files have been moved back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 19:52:23 -06:00
Brian S.
adce235a7e
Strip whitespace from inventory item string values on save
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
modules/inventory/common.py: _normalize_items now strips leading/trailing whitespace from any string-valued item field before persisting an inventory (name, category, subcategory, location, etc.). Non-string values (numbers, None) are left untouched.

Closes #52
2026-07-30 17:34:46 -06:00
Roland
540081203f
controller: restore useConfigs and useDownloads
The toggle-hook de-duplication replaced a range of customHooks.js that also
contained these two unrelated hooks, breaking the production build. Restored
byte-identical from master; exports now match master exactly.

Jest missed this because it only compiles what tests import — `npm run build`
is what catches a missing export.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-30 13:42:34 -06:00
Roland
07af0c9f28 controller: de-duplicate the hardware toggles
The Hotspot, Bluetooth, Desktop, VNC and Throttle toggles had each been written
out longhand at four layers. Collapse the shared parts:

- useSubsystemToggle() replaces five near-identical hooks. This also fixes a
  latent bug: only useDesktop/useVnc restored state when a request failed, so a
  failed Bluetooth/Throttle/Hotspot toggle left  at null and — since the
  status poll sees no change — the toggle stayed disabled and read as
  unsupported until it remounted.
- SubsystemToggle renders all five, with the Desktop's stop confirmation and
  VNC's desktop gating as declared options rather than bespoke copies.
- The fallback UI's four update*/toggle* pairs become one config-driven
  implementation (-100 lines).
- _raise_for_action() centralises the 13 endpoint failure checks, including
  VNC's 409-vs-500 split.

Deliberately not shared: the six get_*_status_dict() builders (payloads
genuinely differ) and the per-subsystem mechanisms (nmcli/rfkill/sysfs/systemctl).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 19:48:08 -06:00
Roland
5313ab57ef controller: report and stop every installed VNC backend
Greptile P1 (security): with both stacks installed, selection keyed only off
wayvnc's unit and config. If RealVNC was the one running, status reported VNC
as off and stop targeted wayvnc — leaving the screen remotely accessible while
the UI claimed VNC was off. Reproduced on 10.0.0.8: RealVNC serving on 5900
reported as running=false.

Now every installed backend is reported with its own active state: status is on
when any backend runs and names the one actually serving, and stop stops all
running backends. wayvnc is still preferred for starting.

Greptile P2 (latency): both units are probed in a single multi-unit
`systemctl show`, and the status dict no longer re-probes, so VNC adds one
subprocess to /api/stats rather than three.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 18:10:05 -06:00
Roland
2328524ab2 controller: VNC toggle, gated on the Desktop running
Add a VNC toggle to the Controller (both UIs) that starts/stops whichever VNC
backend is installed: wayvnc for Wayland (Raspberry Pi OS), RealVNC for X11.

Runtime-only, like the Desktop toggle: the unit is never enabled, so VNC does
not return after a reboot. Users who want it permanently on can enable it with
raspi-config.

VNC serves the desktop session, so starting it without a desktop gives clients
a blank screen. The gate lives in the backend, not just the UI, so curl callers
get it too (409, distinct from a systemctl failure). Stopping is never gated so
a running VNC server can't be stranded on with no way to turn it off.

Backend selection deliberately avoids raspi-config's live-compositor probe
(pgrep labwc), which reports X11 — and would pick the wrong backend — whenever
the desktop is stopped. wayvnc also requires its config to exist; that is
checked directly rather than via the unit's ConditionResult, which reports 'no'
for any unit that has never been started.

Extracts _unit_properties()/_systemctl_unit() so Desktop and VNC share one
systemd probe and start/stop path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 18:10:05 -06:00
Roland
a0fa03e6c5
controller: leave a usable terminal when stopping the desktop
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
Stopping the display manager left the Pi's monitor black with only a blinking
cursor. The console VT is already in the foreground with a getty and a live
shell on it (verified: /dev/vcs1 holds the login banner and prompt) — the
display manager just hands the display back without the console being
repainted, so nothing is visible.

Activate the console VT, then bounce off another VT to force a modeset and
redraw, and print a short message telling the user the Controller stopped the
desktop and how to start it again.

Best-effort throughout: skipped when no getty is on the console VT, a chvt or
console-write failure never fails the stop, and being stranded on the repaint
VT is reported rather than claimed as success. Verified on 10.0.0.8 hardware.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 16:07:36 -06:00
Roland
ab5f416b3f
upgrade: report git branch via --show-current so a stale release tag can't break upgrades
git rev-parse --abbrev-ref HEAD reports "heads/release" when a stale
tag named release exists alongside the release branch (tags outrank
heads in git ref resolution).  check_for_update then looked up the
nonexistent origin/heads/release and permanently reported no updates
available, and the Controller UI pre-filled "heads/release" as the
upgrade branch.

Use git branch --show-current everywhere instead; it never
disambiguates.  A detached HEAD (empty output) now defaults to the
release branch for upgrades; a genuine git failure still returns None.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 16:03:33 -06:00
Roland
9e8050bd48
controller: restore Desktop toggle state when a start/stop request fails
Greptile P1: on failure the status poll never re-syncs (the desktop did
not change), so the toggle stayed disabled until remount.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 15:19:47 -06:00
Roland
3e559e1bf8 controller: Desktop start/stop toggle; rename hotspot/bluetooth actions to what they do
Add a Desktop toggle to the Controller (both UIs) that starts/stops
display-manager.service. Fail open: the unit is never enabled/disabled,
so the desktop always returns on reboot; users who want it disabled
across reboots can set the default target themselves.

Rename the hotspot and bluetooth actions to reflect their real behavior:
- enable/disable_hotspot -> start/stop_hotspot (/api/hotspot/start|stop)
- enable/disable_bluetooth -> unblock/block_bluetooth (rfkill semantics,
  /api/bluetooth/unblock|block)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 15:19:36 -06:00
Roland
a4eeba14ee
downloads: surface daily-limit stall to users
Add daily_limit_reached to the DownloadManager summary (/api/status):
True when due (new, non-recurring) downloads exist and every one of
them is blocked by the global or per-domain daily download limit.
Show a "Max Daily Downloads Reached" banner on the Downloads page when
set (and the manager is neither disabled nor stopped).

Previously a queue stalled by the daily limit looked broken: the UI
reported 0 pending with no explanation, and the skip reason was only
visible at DEBUG log level.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 14:58:37 -06:00
Roland
2baa9cbd2e
controller: address hotspot protocol review findings
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
- Reset a saved protocol the newly probed device does not support
  instead of offering it (both UIs); a successful probe is
  authoritative, the saved value is only kept when probing fails.
- Ignore out-of-order protocol probe responses when switching devices
  quickly (both UIs).
- Verify the WPA2 rollback after a failed WPA3 activation; report that
  the hotspot may be down instead of claiming the revert succeeded.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 21:12:34 -06:00
Roland
8687816b84 controller: selectable hotspot protocol (WPA2/WPA3) with device capability probing
Probe the selected WiFi device with iw to determine which hotspot
protocols it can host: WPA2 requires AP mode; WPA3 requires SAE in AP
mode (SAE_OFFLOAD_AP, or a mac80211 driver with the CMAC cipher for
software SAE).  The Raspberry Pi's brcmfmac supports neither, so Pis
correctly offer WPA2 only (verified against real Pi 4/Pi 5 iw output).

The protocol is saved as hotspot.protocol in controller.yaml.
enable_hotspot() now always sets the Hotspot connection's key-mgmt
explicitly (sae + required PMF for WPA3) so a reused profile cannot
keep a stale protocol, and reverts to WPA2 if the driver rejects SAE.

New GET /api/hotspot/protocols endpoint; protocol dropdowns added to
both the ControllerPage React UI and the fallback controller UI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 21:12:24 -06:00
Roland
2355095391
Bump version to 0.28.6-beta
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-07-23 19:58:16 -06:00
Roland
1a88fb4a5b
controller: report idle (0 RPM) fans; prefer the CPU fan over case fans
The RPi 5 firmware stops the fan entirely when cool, so a 0 RPM
reading is normal and must not be hidden. On multi-fan machines,
prefer a CPU-labeled hwmon group over faster case fans.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 19:52:58 -06:00
Roland
42a017294e controller: report cooling fan RPM in CPU status
The RPi 5 fan connector exposes a tachometer via the pwmfan hwmon
device. Report it as fan_rpm in cpu_stats and show it in both the
Controller fallback UI and the React Status page. Hidden when no fan
is present.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 19:52:47 -06:00
Roland
f635671266
controller: don't strand the API stopped on a failed primary unmount
Review feedback: skip the service stop when the primary mount is already
unmounted (a no-op request must not cause an API outage), and restart
wrolpi-api (best effort) when umount fails since the drive is still
mounted and usable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 19:23:05 -06:00
Roland
43f6ed2b8e controller: allow unmounting the primary drive, stopping the API first
Support swapping in a backup drive: /api/disks/unmount now accepts the
primary mount (previously rejected as reserved). It stops wrolpi-api so
open files are released, then umounts directly since the reconciler
never manages the primary mount. The /etc/fstab entry is left in place
so a reboot restores the mount.

Both UIs now offer Unmount for the primary mount and warn that the API
service will be stopped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 19:22:53 -06:00
Roland
5a11e23a56
controller: expose kind/group in the services API response
The React ControllerPage partitions services on `service.group`, but
ServiceStatusResponse did not declare the new kind/group fields, so
Pydantic silently stripped them from /api/services and the React UI
rendered one flat, ungrouped table. The fallback UI was unaffected
because it is server-rendered without the response model.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 18:44:59 -06:00
Roland
36f84941db
install: always install serve; resolve it via PATH in wrolpi-app
wrolpi-app crash-loops with 203/EXEC when /usr/local/bin/serve is missing.
Two independent causes, both fixed here:

1. The `single-file --version ||` guard in the install scripts only installed
   serve (and readability) when single-file was ABSENT. On a machine that
   already has single-file (a prior install or a baked image), serve was
   skipped, and re-running install.sh could never repair it. Install each npm
   global independently based on its own presence.

2. The service hardcoded /usr/local/bin/serve, which breaks when npm's global
   prefix puts serve in /usr/bin (depends on how node was installed). Resolve
   serve via PATH with `/usr/bin/env serve`.

Verified on a fresh Raspberry Pi OS Trixie install (10.0.0.8): wrolpi-app is
active with 0 restarts and serves 200 with the PATH-resolved ExecStart.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:21:58 -06:00
Roland
cd11b5b7b7
downloader: don't leak processing_domains on failed claim or missing row
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
Addresses two processing_domains leaks raised in review; each could permanently
exclude a domain and eventually stall the scheduler until restart.

1. Claim-transaction failure (Greptile P1): `_add_processing_domain` mutated shared,
   non-transactional state before the claim transaction committed.  A commit failure
   skipped dispatch but left the domain claimed forever.  Fix: defer both the
   processing_domain add AND the signal dispatch to after the transaction commits;
   track claimed domains in a local `claimed` list so the loop's own dedup /
   concurrency cap is unchanged.

2. Row deleted before dispatch (Greptile P1): if a claimed download was deleted
   between commit and the signal running, `Download.find_by_id` raised before
   `download_domain` was set, so the handler's `finally` released nothing.  Fix:
   derive `download_domain` from the URL (`urlparse(url).netloc`, the same value the
   dispatcher claimed) up front, so cleanup always runs.

Regression tests written bug-first (both fail before this change):
- test_signal_releases_processing_domain_when_download_missing
- test_dispatch_releases_claimed_domains_when_transaction_fails

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 10:33:08 -06:00
Roland
f341e386ad downloader: fix self-deadlock that silently stopped all downloads
`_dispatch_new_downloads` claimed downloads inside a `get_immediate_db_session()`
(BEGIN IMMEDIATE) transaction and then `await`ed `api_app.dispatch(...)` while that
write transaction was still open.  Under NullPool (production, one connection per
session) the dispatched `signal_download_download` opens its OWN connection and —
inheriting the thread-local `_immediate_txn` flag — also issues BEGIN IMMEDIATE,
which blocks on the write lock the outer transaction still holds until busy_timeout
(30s) expires: `(sqlite3.OperationalError) database is locked [SQL: BEGIN IMMEDIATE]`.
Every dispatch cycle died on the first download, so nothing downloaded at all.

Fix: claim the downloads inside the transaction, collect (id, url), and dispatch the
signals AFTER the `with` block commits and releases the write lock.

Diagnosed live on 10.0.0.9 (downloads had silently stopped); with this change the
14 stuck recurring downloads drained immediately and no lock errors recurred.

Why tests missed it: the `test_session` fixture returns a single shared
(engine, session) for every `_get_db_session` call, so all sessions in a test ride
one connection -- the second-connection write-lock contention that causes the
deadlock cannot occur.  Added a regression test asserting the invariant instead: the
download signal must be dispatched with no immediate transaction active.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 10:32:57 -06:00