Migrated both files onto the token-driven ./ui component library: Table,
Modal, Button/IconButton, Checkbox, Header, Panel, Progress, Placeholder,
Skeleton, ActionInput/TextInput, IconStack, Menu/ButtonGroup for the slicer
split-button, and native <img>/<form> in place of Semantic's Image/Form.
Notable adaptations:
- Semantic's per-row `disabled` styling (no visual equivalent in the new
Table) is reproduced with inline opacity/pointer-events instead.
- DirectorySearch's autoFocus now passes straight through as a prop instead
of a nested `input={{autoFocus}}` object, matching SearchBox's API.
- The responsive icon+label footer buttons keep a single Button and toggle
children instead of swapping component type.
- FileBrowser.test.js: two upload-button assertions keyed off a literal
Semantic `.green` class; broadened to also match on the icon, mirroring
the sibling assertion already using that fallback.
Preserved as-is (reported, not fixed): getVideoPreviewModal's
`<Header as='h5'>{path['path']}</Header>` indexes a string with `['path']`,
which was already dead/undefined before this migration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All 43 cypress specs pass now (was 36 of 43). Five of the seven failures were
stale selectors, two were real.
Real, and both mine:
The search modal stopped focusing its input. Mantine mounts modal content on
a later tick, so the effect that focused it ran while the ref was still null;
Semantic's modal mounted synchronously, which is why focusing there had been
enough. Fixed declaratively -- the field asks for focus itself, and SearchBox
also sets `data-autofocus`, which is what Mantine's focus trap honours when it
takes focus away from a plain autoFocus.
Escape closed the suggestion list and the enclosing modal at once, so a user
correcting a mistyped query lost the whole modal. Escape now dismisses the
suggestions and stops there; a second Escape, with the list closed, still
closes the modal.
Stale selectors, all in the specs: Semantic's icon font (`i.search.icon`) is
SVG now, its `.ui.modal`/`.ui.dimmer` are the library's modal and overlay, and
one test asserted a button carried the classes `ui button secondary`. That last
one is rewritten to check what a user can actually perceive -- that Edit is a
real link pointing at the edit route -- because framework class names say
nothing and break on every library change.
Two accessibility gaps the tests walked into:
The nav's search control was an anchor with an onClick, no name and no
keyboard path. It now has a name, a role and Enter/Space activation, which is
also what let the spec stop matching on icon-font classes.
The search dialog had no accessible name at all; it has one now.
One `{force: true}` remains, with its reason recorded: Cypress judges that
field covered because it sits in a fixed modal under a sticky header, and I
verified by hand at Cypress's own viewport that it is the topmost element at
its centre, unobscured, and focused on open.
Also closed a hole in the ratchet that this uncovered. A file could drop its
`semantic-ui-react` imports, leave the baseline, and still render Semantic
through Theme.tsx's wrappers -- which is exactly how a Semantic modal survived
in the search shortcut long after that file looked migrated. A test now
forbids it, and the three files doing it (Calculators, KeyboardShortcutsProvider,
and Common.js reaching for theme names) import from the library or from
themes/names instead.
Greptile P1, and a real one: App.js migrated and took `<SemanticToastContainer/>`
with it, but seven pages -- Archive, Channels, Docs, FilePreview, Playlists,
Zim, ControllerPage -- still raised their toasts through the Semantic library.
With no container mounted those notifications went nowhere. No error, no
warning; a user deletes a document and is simply never told whether it worked.
Fixed by pointing those seven at the library's `toast`, which takes the same
options, rather than by mounting a second toast system. Nothing calls the
Semantic toast library now, so a test asserts that at zero: a file that goes
back to it would raise notifications that silently never appear, and that is
not something a reviewer would spot.
Also reverted Archive.js to its last good state. An interrupted agent had
migrated part of it -- Panel, Tabs, and Mantine's Grid.Col -- while the rest of
the file still used Semantic's Segment, Tab and Grid.Row, so a single `Grid`
identifier had to be two different components at once. A half-migrated
1400-line file is worse than an unmigrated one; it goes back on the list and
gets done in one piece.
That break was masked: react-scripts caches ESLint results per file, so the
earlier build passed while Archive.js sat on a stale pass. Editing its import
invalidated the entry and the errors appeared. Verified with the cache cleared
this time, which is the only way that check means anything after a wide change.
Phase 2, part 4. Baseline 50 -> 15.
Migrated: all eight calculators, the rest of inventory and collections, the
Map/Zim/Docs cluster, Videos, Files, Search, Archive, DashboardPage,
DatesSelector, HelpModal, Events, Tags, Flasher's neighbours, admin Settings
and Status, Download, and App.
Also the three shared modules everything imports: api.js and customHooks.js
swap the Semantic toast library for the library's shim (same call shape), and
useForm.js renders the library's Message. useForm accepts both the old
`{content, header, positive|negative}` shape and the new `{kind, title}`,
because the pages building those objects migrate separately.
Two library gaps closed, both found by call sites needing them:
ActionInput is a forwardRef now. Its ref reaches the input rather than the
wrapper, because callers use it to focus or select the field -- a ref to the
surrounding div would do neither, which is why one call site had resorted to
hand-copying the component's markup.
Header takes a `color` token name. Three separate call sites had reached for
an inline style to colour a heading that carries meaning ("3 items to add"
against "2 to remove"), which is how a hex ends up in markup.
Two files needed repair after being interrupted mid-edit: MapViewer had a
Segment left in its error branch, and VideoPlayer had lost the useContext
import its own code still used. Neither jest nor tsc noticed; only
`npm run build` did, which is the whole reason that check exists.
Fifteen files remain: Archive, BulkTagModal, Channels, Docs, FileBrowser,
FilePreview, Flasher, Nav, Playlists, Theme.tsx, Upload, VideoPlayer, Zim,
ControllerPage, admin/Downloads. Theme.tsx is deliberately last -- the
unmigrated files still import its Semantic-backed wrappers, so it can only go
once they have.
Three findings from review and CI on the Common.js migration, all real.
Selecting a suggestion navigated nowhere. Semantic handed its callers
`{result}` and every caller destructures that shape, so passing the bare result
left `result.location` undefined -- the search suggestions and the Ctrl+K modal
both stopped navigating. The shape stays until the call sites can change
together. (Greptile P1.)
Suggestions lost their custom markup. Callers pass `resultRenderer` to draw a
tag as a chip rather than a line of text, and the migrated wrapper forwarded it
into a div's props instead of to SearchBox, so every suggestion fell back to the
generic title and description. SearchBox takes it now. (Greptile P2.)
The suggestion list clipped its last group. A cypress spec caught it: with
directories, channels and domains all present, the content runs past the 320px
cap and the final row was cut off. Eight grouped suggestions is an ordinary
result set, so the cap was simply too low; it is viewport-relative now and
still bounded.
Both regressions are pinned by tests -- one asserting the callback shape a
caller actually destructures, one asserting the custom renderer replaces the
default markup rather than wrapping it.
Phase 2, part 3. Baseline 63 -> 50.
Common.js is the one that mattered: 51 files import it, so its Semantic usage
was propping up the rest of the migration. Everything in it now comes from the
library -- Paginator, the two search inputs, DirectorySearch, TabLinks, the
toggles, the messages, APIButton, Breadcrumbs, SimpleAccordion, IframeViewer,
CardPoster, FileIcon, LoadStatistic, the theme menu, and the hotspot icons.
Also migrated: Placeholder, Apps, ArchivePreview, CbzViewer, AddToPlaylist,
TaggedDeleteConfirmModal, the collection edit/tag forms, and four inventory
components.
New in the library, all in the gallery and tested:
SearchBox Semantic's `<Search category>` with its grouped results shape
kept, because four call sites already build it. A real
combobox rather than a div that listens for clicks.
Pagination Semantic's prop names; onPageChange gets the page number.
TabBar the strip only -- routing stays with the caller.
IconStack Semantic's IconGroup, named once for assistive technology.
Table.HeaderCell sorted/onSort -- a real button carrying aria-sort.
Button/IconButton translate Semantic's size names.
That last one is defensive: unmigrated callers still pass size='tiny', and
Mantine drops a size it does not recognise without a word, so the button
quietly renders at the wrong size. Mapping here keeps the mixed state honest.
Three real bugs found and fixed on the way, none of them cosmetic:
* `fileSuffixIconName` returned fourteen icon names the map did not have, so
opening a folder with a PDF, a Word document or an archive in it logged an
error and drew a fallback glyph. All fourteen mapped, with a test that
walks the mapper's own return values, and another asserting every mapped
name resolves to a Tabler component that exists.
* `Statistic` relied on Semantic's `Statistic.Group` for its spacing, so on
the dashboard the load averages ran together into "1 MIN. LOAD5 MIN.
LOAD15 MIN. LOAD". It carries its own spacing now.
* Semantic's inverted stylesheet colors every descendant of an inverted
segment, so Toggle labels inherited #dddddd -- white pixels in night mode,
on the Settings page, which is exactly where someone adjusts their WROLPi
in the dark. Restated from tokens.
The baseline ratchet gained a third assertion: a migrated file may still lean
on Semantic's CSS classes without importing the library, which the import check
cannot see and which would collapse silently when Phase 4 deletes the
stylesheet. There is one such bridge today and it is now declared.
Phase 2, part 2. Baseline 72 -> 63.
Migrated: Tables, SortableTable, DonatePage, Share, Headline, ErrorBoundary,
and the QR code, VIN decoder, and temperature calculators.
New in the library, all in the gallery and tested:
SearchBox Semantic's `<Search category>`, keeping its grouped `results`
shape because four call sites already build it. A real
combobox: arrows move, Enter takes the highlighted suggestion
or submits the raw text, Escape closes without clearing.
Pagination Semantic's prop names, but onPageChange gets the page number
rather than an (event, data) pair.
TabBar the strip and its house style only; routing stays at the call
site, which is what decides what is active.
IconStack two glyphs composed into one symbol, named once for assistive
technology (Semantic's IconGroup).
Table.HeaderCell gains `sorted` and `onSort`.
That last one is a fix, not just a port. Semantic's `sorted` prop set
`aria-sort`; migrating a table by hand dropped it, leaving a clickable <th>
that no keyboard could reach and no screen reader could describe. Putting it
in the library means the next table gets it right without anyone remembering.
Night mode holes closed along the way: the QR codes on the donate, share, and
calculator pages are white by necessity — a scanner needs the contrast — and
they render as SVG, which the media filter does not target. In night mode that
was a lit panel. They now carry `.media`, so the filter remaps the block as a
unit and the contrast survives the mapping.
Also: `paypal` joins the icon map rather than being imported ad hoc in a page,
and FileBrowser stops passing Semantic-only table props into a DOM element.
Filtering media was keyed on `data-theme="night"`, which made it a property of
the theme that no user could turn off. Some people want their media tinted to
match the interface; some want to see the actual image.
A theme now declares a filter and whether it starts on, and the user decides:
night night-red on by default. An unfiltered thumbnail undoes dark
adaptation, which is the whole point of the mode.
amber amber-mono off by default. Amber is a look, not a night-vision
aid, so media keeps its real colors until asked for.
light - no filter offered, so no toggle is shown.
dark -
The choice is stored per theme, so wanting amber's tint says nothing about
wanting night's. The toggle lives in the theme picker, which puts it on the
Settings page and in the gallery at once.
Mechanically the rules now key on a `data-media-filter` attribute rather than
`data-theme`: a rule keyed on the theme could not be turned off. index.html
stamps that attribute before first paint alongside data-theme -- without it,
night mode paints a screen of unfiltered thumbnails while the bundle arrives
over a hotspot, costing the user the adaptation the theme exists to protect.
Tests cover both the attribute rule and the fact that the inline script's
duplicated ids and defaults still agree with names.ts.
The amber matrix is the same luminance collapse tinted to #ff9500 (red full,
green 0.584, blue zero) with a gentler gamma than night's: amber carries far
more luminance and clips to a flat wash otherwise.
The gallery gains a Media filtering section showing the WROLPi logo filtered,
beside an unfiltered copy that opts out with `night-unfiltered`. That copy is
behind a click on purpose -- an unfiltered image is a bright patch, and night
mode should not spring one on someone reading in the dark.
ui-design.md gains the media filtering contract it never got in Phase 0,
including the note that a Phase 3 custom theme names an existing filter id and
cannot ship its own matrix, since that would be code rather than data.
Phase 2, part 1. Three files leave Semantic UI (baseline 75 -> 72), and the
components they needed join the library and the gallery:
Header Semantic's shape (as/icon/subheader/dividing). 47 files use
<Header>, and the level now picks the size, so the type scale
stays a scale instead of each call site choosing a font size.
Message gains an icon and a dismiss button.
ActionInput input and button sharing one outline, for <Input action={...}/>.
Loading centered loader with a caption, replacing the
<Loader active inline='centered'> + wrapper div in 19 places.
Modal becomes compound - Modal.Header/.Content/.Actions, plus
Semantic's open, closeIcon, and size names - because 34 files
are written against that shape.
test-utils mounts MantineProvider with ThemeProvider's exact configuration:
migrated components render Mantine internals and every one of their test files
would otherwise need its own provider.
One Semantic bleed found and fixed: semantic.min.css sets
`h1,h2,h3,h4,h5 { font-family: Lato }` and loads after Mantine's element rule,
so headings ignored the theme's body face. Only visible in amber, where the
body is mono and the headings were not. A class-level restatement outranks it;
the rule comes out when Semantic does.
Verified: 925 jest tests, clean production build, and a night-mode audit of
every painted pixel in the gallery (red-only, one known false positive). The
Configs backup and restore flow was exercised against the real API.
Addresses the automated review on #583; all three were real.
The night filter only applied to elements tagged `.media`, and nothing in the
app carried that class - so selecting Night filtered the interface but left
every video, thumbnail, PDF and map full-colour, which is precisely the thing
the mode exists to prevent. The Settings copy promised otherwise, so it was
also wrong.
It now targets the element types instead of an opt-in class, because night mode
has to fail safe: a library of thousands of files reaches the screen through
many components, and one that forgot to opt in would undo half an hour of dark
adaptation. `.media` still works for wrappers that must be filtered as a unit,
and `.night-unfiltered` opts out. The filter is restated on `:fullscreen`, which
a filter on an ancestor stops reaching. Verified on the videos page: all 25
media elements filtered.
Three of the five theme choices rendered a fallback glyph and logged errors,
because the icon map was built by grepping literal name='...' strings and the
navigation picker passes its icon through a variable. Adds sun, moon and
lightbulb, plus a test asserting every theme choice resolves to a real icon.
The pre-paint script wrapped the `matchMedia` fallback in the same try as the
storage read, so a device with storage blocked painted light even when it
prefers dark - recreating the flash the script exists to prevent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The gallery at /theme-sample is now a normal route rather than a development
one, and Settings links to it, so a user can see what a theme actually looks
like before committing to it. Every new component belongs in the gallery: it is
how the design gets reviewed, and how a component that only looks wrong in one
of the four themes gets caught.
Settings gains a Theme section with the new ThemePicker, which explains what
each theme is for - notably that night mode filters videos, documents and maps
to red as well, not just the interface.
Theme names and picker options move to themes/names.ts so the component library
can read them without importing components/Theme.tsx, which still pulls in
Semantic UI. Theme.tsx re-exports them, so existing call sites are unchanged.
The navigation bar's compact picker now reads the same list, so the two cannot
drift apart.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds src/components/ui: buttons, messages, labels, progress, status, tables,
form controls, panels, cards, statistics, modals, tooltips, tabs, and icons.
Everything resolves its colors through theme tokens, so no component branches
on the current theme, and Semantic UI keeps rendering the app meanwhile.
Mantine is bridged rather than restyled: its own CSS variables are pointed at
our tokens. Four details were not obvious, and each caused a visible defect
in night mode before being fixed:
- Structural colors must go in the resolver's light/dark blocks. Mantine
emits its values under [data-mantine-color-scheme], which outranks the
:root block that `variables` becomes, so its grays were winning.
- The dark and gray palettes are mapped shade by shade to tokens. Mantine
reads them directly for table borders, zebra stripes and input chrome;
they can be neither flattened nor left alone.
- Mantine writes --button-* as inline styles, which no stylesheet can
outrank, so the dashed danger treatment sets real properties instead.
- Switch colors are written onto inner elements, so they need scoped CSS
rather than a variable on the root. Adds a --knob token for control
handles, because Mantine's default thumb follows theme.white, which is
near-black in the dark-based themes.
An audit of every painted pixel in the /theme-sample gallery now finds no
non-red channel in night mode.
Icons come from @tabler/icons-react with the 102 Semantic names the app uses
mapped to components; they stroke with currentColor, so they follow the theme
without any rule of their own. Toasts keep the react-semantic-toasts-2
signature so those call sites migrate by changing an import.
TypeScript 5 is required: Mantine's declarations use `const` type parameters,
which 4.9 cannot parse (Mantine 7 is the same, so downgrading is no escape).
That needs two npm overrides to keep clean installs working, both verified
against a Docker build: react-scripts declares peerOptional typescript ^3||^4,
and fork-ts-checker 6.5.2 assigns typescript.performance.mark, which TS 5
made getter-only - crashing the dev server's type checker on every rebuild.
semantic-baseline.test.js ratchets the removal: it fails if a file not already
listed imports Semantic UI. An ESLint rule would have printed 75 warnings per
rebuild, and CI unsets CI= for its build step, so it would guard nothing there.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every color now resolves through CSS custom properties on
html[data-theme=...], so a theme is a table of values rather than code.
Four themes ship: light, dark, night (red hues only, to preserve dark
adaptation off-grid) and amber (monochrome amber terminal).
Fonts are exposed as two roles, --font-body and --font-mono, which a theme
assigns a bundled face to; amber uses the mono face. Faces must be
self-hosted, never fetched from a CDN, so the app keeps working offline.
Night mode also filters media that cannot consume tokens - video, images,
PDFs, embedded pages, canvases - through an SVG color matrix that maps
luminance onto the red channel. Tag such an element with `.media`; a filter
on an app-level ancestor would create a containing block and break fixed
positioning.
ThemeProvider persists the choice and index.html stamps data-theme from
localStorage before the bundle loads, so the first paint already matches the
saved theme on a slow connection. The OS preference is followed only while
"system" is selected; night and amber are explicit choices. The nav toggle
becomes a five-item picker.
Semantic components still render themselves, so only the shell is
token-driven for now; the inverted props stay in place until each component
is migrated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The toggle-hook de-duplication replaced a range of customHooks.js that also
contained these two unrelated hooks, breaking the production build. Restored
byte-identical from master; exports now match master exactly.
Jest missed this because it only compiles what tests import — `npm run build`
is what catches a missing export.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Hotspot, Bluetooth, Desktop, VNC and Throttle toggles had each been written
out longhand at four layers. Collapse the shared parts:
- useSubsystemToggle() replaces five near-identical hooks. This also fixes a
latent bug: only useDesktop/useVnc restored state when a request failed, so a
failed Bluetooth/Throttle/Hotspot toggle left at null and — since the
status poll sees no change — the toggle stayed disabled and read as
unsupported until it remounted.
- SubsystemToggle renders all five, with the Desktop's stop confirmation and
VNC's desktop gating as declared options rather than bespoke copies.
- The fallback UI's four update*/toggle* pairs become one config-driven
implementation (-100 lines).
- _raise_for_action() centralises the 13 endpoint failure checks, including
VNC's 409-vs-500 split.
Deliberately not shared: the six get_*_status_dict() builders (payloads
genuinely differ) and the per-subsystem mechanisms (nmcli/rfkill/sysfs/systemctl).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Greptile P1 (security): with both stacks installed, selection keyed only off
wayvnc's unit and config. If RealVNC was the one running, status reported VNC
as off and stop targeted wayvnc — leaving the screen remotely accessible while
the UI claimed VNC was off. Reproduced on 10.0.0.8: RealVNC serving on 5900
reported as running=false.
Now every installed backend is reported with its own active state: status is on
when any backend runs and names the one actually serving, and stop stops all
running backends. wayvnc is still preferred for starting.
Greptile P2 (latency): both units are probed in a single multi-unit
`systemctl show`, and the status dict no longer re-probes, so VNC adds one
subprocess to /api/stats rather than three.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a VNC toggle to the Controller (both UIs) that starts/stops whichever VNC
backend is installed: wayvnc for Wayland (Raspberry Pi OS), RealVNC for X11.
Runtime-only, like the Desktop toggle: the unit is never enabled, so VNC does
not return after a reboot. Users who want it permanently on can enable it with
raspi-config.
VNC serves the desktop session, so starting it without a desktop gives clients
a blank screen. The gate lives in the backend, not just the UI, so curl callers
get it too (409, distinct from a systemctl failure). Stopping is never gated so
a running VNC server can't be stranded on with no way to turn it off.
Backend selection deliberately avoids raspi-config's live-compositor probe
(pgrep labwc), which reports X11 — and would pick the wrong backend — whenever
the desktop is stopped. wayvnc also requires its config to exist; that is
checked directly rather than via the unit's ConditionResult, which reports 'no'
for any unit that has never been started.
Extracts _unit_properties()/_systemctl_unit() so Desktop and VNC share one
systemd probe and start/stop path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stopping the display manager left the Pi's monitor black with only a blinking
cursor. The console VT is already in the foreground with a getty and a live
shell on it (verified: /dev/vcs1 holds the login banner and prompt) — the
display manager just hands the display back without the console being
repainted, so nothing is visible.
Activate the console VT, then bounce off another VT to force a modeset and
redraw, and print a short message telling the user the Controller stopped the
desktop and how to start it again.
Best-effort throughout: skipped when no getty is on the console VT, a chvt or
console-write failure never fails the stop, and being stranded on the repaint
VT is reported rather than claimed as success. Verified on 10.0.0.8 hardware.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
git rev-parse --abbrev-ref HEAD reports "heads/release" when a stale
tag named release exists alongside the release branch (tags outrank
heads in git ref resolution). check_for_update then looked up the
nonexistent origin/heads/release and permanently reported no updates
available, and the Controller UI pre-filled "heads/release" as the
upgrade branch.
Use git branch --show-current everywhere instead; it never
disambiguates. A detached HEAD (empty output) now defaults to the
release branch for upgrades; a genuine git failure still returns None.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Greptile P1: on failure the status poll never re-syncs (the desktop did
not change), so the toggle stayed disabled until remount.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a Desktop toggle to the Controller (both UIs) that starts/stops
display-manager.service. Fail open: the unit is never enabled/disabled,
so the desktop always returns on reboot; users who want it disabled
across reboots can set the default target themselves.
Rename the hotspot and bluetooth actions to reflect their real behavior:
- enable/disable_hotspot -> start/stop_hotspot (/api/hotspot/start|stop)
- enable/disable_bluetooth -> unblock/block_bluetooth (rfkill semantics,
/api/bluetooth/unblock|block)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add daily_limit_reached to the DownloadManager summary (/api/status):
True when due (new, non-recurring) downloads exist and every one of
them is blocked by the global or per-domain daily download limit.
Show a "Max Daily Downloads Reached" banner on the Downloads page when
set (and the manager is neither disabled nor stopped).
Previously a queue stalled by the daily limit looked broken: the UI
reported 0 pending with no explanation, and the skip reason was only
visible at DEBUG log level.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Reset a saved protocol the newly probed device does not support
instead of offering it (both UIs); a successful probe is
authoritative, the saved value is only kept when probing fails.
- Ignore out-of-order protocol probe responses when switching devices
quickly (both UIs).
- Verify the WPA2 rollback after a failed WPA3 activation; report that
the hotspot may be down instead of claiming the revert succeeded.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Probe the selected WiFi device with iw to determine which hotspot
protocols it can host: WPA2 requires AP mode; WPA3 requires SAE in AP
mode (SAE_OFFLOAD_AP, or a mac80211 driver with the CMAC cipher for
software SAE). The Raspberry Pi's brcmfmac supports neither, so Pis
correctly offer WPA2 only (verified against real Pi 4/Pi 5 iw output).
The protocol is saved as hotspot.protocol in controller.yaml.
enable_hotspot() now always sets the Hotspot connection's key-mgmt
explicitly (sae + required PMF for WPA3) so a reused profile cannot
keep a stale protocol, and reverts to WPA2 if the driver rejects SAE.
New GET /api/hotspot/protocols endpoint; protocol dropdowns added to
both the ControllerPage React UI and the fallback controller UI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The RPi 5 firmware stops the fan entirely when cool, so a 0 RPM
reading is normal and must not be hidden. On multi-fan machines,
prefer a CPU-labeled hwmon group over faster case fans.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The RPi 5 fan connector exposes a tachometer via the pwmfan hwmon
device. Report it as fan_rpm in cpu_stats and show it in both the
Controller fallback UI and the React Status page. Hidden when no fan
is present.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review feedback: skip the service stop when the primary mount is already
unmounted (a no-op request must not cause an API outage), and restart
wrolpi-api (best effort) when umount fails since the drive is still
mounted and usable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Support swapping in a backup drive: /api/disks/unmount now accepts the
primary mount (previously rejected as reserved). It stops wrolpi-api so
open files are released, then umounts directly since the reconciler
never manages the primary mount. The /etc/fstab entry is left in place
so a reboot restores the mount.
Both UIs now offer Unmount for the primary mount and warn that the API
service will be stopped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The React ControllerPage partitions services on `service.group`, but
ServiceStatusResponse did not declare the new kind/group fields, so
Pydantic silently stripped them from /api/services and the React UI
rendered one flat, ungrouped table. The fallback UI was unaffected
because it is server-rendered without the response model.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
wrolpi-app crash-loops with 203/EXEC when /usr/local/bin/serve is missing.
Two independent causes, both fixed here:
1. The `single-file --version ||` guard in the install scripts only installed
serve (and readability) when single-file was ABSENT. On a machine that
already has single-file (a prior install or a baked image), serve was
skipped, and re-running install.sh could never repair it. Install each npm
global independently based on its own presence.
2. The service hardcoded /usr/local/bin/serve, which breaks when npm's global
prefix puts serve in /usr/bin (depends on how node was installed). Resolve
serve via PATH with `/usr/bin/env serve`.
Verified on a fresh Raspberry Pi OS Trixie install (10.0.0.8): wrolpi-app is
active with 0 restarts and serves 200 with the PATH-resolved ExecStart.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses two processing_domains leaks raised in review; each could permanently
exclude a domain and eventually stall the scheduler until restart.
1. Claim-transaction failure (Greptile P1): `_add_processing_domain` mutated shared,
non-transactional state before the claim transaction committed. A commit failure
skipped dispatch but left the domain claimed forever. Fix: defer both the
processing_domain add AND the signal dispatch to after the transaction commits;
track claimed domains in a local `claimed` list so the loop's own dedup /
concurrency cap is unchanged.
2. Row deleted before dispatch (Greptile P1): if a claimed download was deleted
between commit and the signal running, `Download.find_by_id` raised before
`download_domain` was set, so the handler's `finally` released nothing. Fix:
derive `download_domain` from the URL (`urlparse(url).netloc`, the same value the
dispatcher claimed) up front, so cleanup always runs.
Regression tests written bug-first (both fail before this change):
- test_signal_releases_processing_domain_when_download_missing
- test_dispatch_releases_claimed_domains_when_transaction_fails
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`_dispatch_new_downloads` claimed downloads inside a `get_immediate_db_session()`
(BEGIN IMMEDIATE) transaction and then `await`ed `api_app.dispatch(...)` while that
write transaction was still open. Under NullPool (production, one connection per
session) the dispatched `signal_download_download` opens its OWN connection and —
inheriting the thread-local `_immediate_txn` flag — also issues BEGIN IMMEDIATE,
which blocks on the write lock the outer transaction still holds until busy_timeout
(30s) expires: `(sqlite3.OperationalError) database is locked [SQL: BEGIN IMMEDIATE]`.
Every dispatch cycle died on the first download, so nothing downloaded at all.
Fix: claim the downloads inside the transaction, collect (id, url), and dispatch the
signals AFTER the `with` block commits and releases the write lock.
Diagnosed live on 10.0.0.9 (downloads had silently stopped); with this change the
14 stuck recurring downloads drained immediately and no lock errors recurred.
Why tests missed it: the `test_session` fixture returns a single shared
(engine, session) for every `_get_db_session` call, so all sessions in a test ride
one connection -- the second-connection write-lock contention that causes the
deadlock cannot occur. Added a regression test asserting the invariant instead: the
download signal must be dispatched with no immediate transaction active.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Investigating downloads stalling on 10.0.0.9 (recurring downloads going overdue
without dispatching). The dispatch path had blind spots that hid the cause, so add
targeted DEBUG logging, change-detected where the cadence would otherwise flood:
- blocked_download_reason(): single source of truth for can_download that returns
*why* downloading is blocked. dispatch_downloads() logs the reason (and the
unblock transition) only when it changes, instead of silently returning every 5s.
- is_within_download_window(): logs the OPEN/CLOSED transition with the computed
local time, window, and timezone — surfaces any clock/timezone mismatch (the box
shows system UTC-5 vs configured America/Denver) at the moment of evaluation.
- _dispatch_new_downloads(): per-cycle snapshot (new count, processing_domains,
daily global_count, download_wait) plus a bounded tally of why due downloads were
NOT dispatched. Prime suspect is a leaked processing_domains slot (a dispatch that
dies without cleanup blocks that domain, and once 4 leak, everything) — that path
previously had no logging.
Behavior-preserving: can_download semantics are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Preserve MEDIA_DIRECTORY/WROLPI_DB across the sudo re-exec (sudo
strips them from the environment, discarding the override).
- Reject unknown options and a second positional path with a usage
error instead of silently checking the wrong database.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
It has a python3 shebang but was missing the executable bit; all other
scripts in ./scripts are executable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PRAGMA quick_check reads every page of the database, so on a large DB or
a slow (e.g. USB spinning) disk it can run for many minutes, making
help.sh (run by the wrolpi-help-script service) appear to hang.
Drop the quick_check from help.sh's fast startup diagnostics and add a
standalone scripts/check_database_integrity.sh for on-demand checks.
The new script runs sqlite3 as the wrolpi user (WAL -shm access),
reports elapsed time, and supports --full for an exhaustive
integrity_check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Controller listed every wrolpi-* service in one flat table, which was
confusing for new users who couldn't tell always-on infrastructure from
one-off/dev/maintenance units.
Services are now split into two groups computed once in the backend and
exposed as `service.group`:
* Core Services - persistent services enabled at boot (api, app, caddy,
kiwix, ...), the normal running system.
* Optional & Maintenance - one-off/dev/maintenance tasks (bootstrap, repair,
upgrade, *-dev, discovered wrolpi-*), plus persistent
services the user has disabled at boot (e.g. Samba with
no shares configured).
Rule: core iff kind == "persistent" and enabled; otherwise optional. Group
membership only changes when the user toggles Boot, never on start/stop or
polling, so rows stay put. Within each group, failed/stopped services sort
first so problems stay visible.
Feature parity across all three render paths: the React ControllerPage, the
server-side Jinja template, and the client-side poll refresh. Docker
containers have no group field and fall back to a single ungrouped list.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fresh installs (and SSH sessions that forward LC_* variables from the client)
can reference an ungenerated locale, causing apt/perl/python to emit
"Cannot set LC_CTYPE to default locale" warnings throughout the install log.
Export LC_ALL=C.UTF-8 for the install run. C.UTF-8 is built into glibc, always
present, and locale-neutral, so it silences the warnings without generating a
locale, without adding a dependency, and without changing the user's configured
locale (e.g. an en_GB.UTF-8 system stays en_GB.UTF-8).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Controller venv (/opt/wrolpi/controller/venv) was only ever created by
upgrade.sh and reset_virtual_environments.sh, neither of which runs during a
fresh install. install_raspberrypios.sh and install_debian_12.sh created only
the main API venv, so on a clean machine wrolpi-controller crash-looped with
203/EXEC ("Unable to locate executable /opt/wrolpi/controller/venv/bin/python")
because repair.sh unconditionally starts the service.
Create the Controller venv inline right after the main venv, mirroring how the
main venv is already built in these scripts. Verified on a fresh Raspberry Pi OS
Trixie install (10.0.0.8): wrolpi-controller comes up active with 0 restarts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two fixes from PR review:
- `PRAGMA journal_mode=WAL` returns the mode SQLite actually selected and can leave the
prior rollback journal without raising. Confirm the returned mode instead of assuming
the request succeeded, so a silent WAL refusal correctly falls back to TRUNCATE +
synchronous=FULL rather than caching WAL with the weaker synchronous=NORMAL.
- `fuseblk` (block-backed FUSE such as ntfs-3g) has unreliable byte-range locking that
can corrupt the database under the workers' concurrent access. Move it out of the
WAL-incompatible allow-list into an UNSAFE_LOCKING_FS_TYPES refuse-list (renamed from
NETWORK_FS_TYPES). NTFS on a modern kernel mounts via the native `ntfs3` driver, which
stays allowed; exFAT is unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A media drive formatted for Windows compatibility (exFAT/FAT/NTFS) cannot support
SQLite WAL: it needs mmap'd shared-memory those filesystems don't provide, so
`PRAGMA journal_mode=WAL` raised `(sqlite3.OperationalError) disk I/O error` on
every connection, taking down the whole API (observed on 10.0.0.8; the tell was
`Media directory has the wrong permissions: 0o40777`, since these filesystems can't
store Unix permission bits).
The engine now tries WAL first and, on failure, falls back to a TRUNCATE rollback
journal with synchronous=FULL (crash-safe, since WROLPi is off-grid). The decision
is memoized per-engine so WAL isn't re-attempted or re-logged on every fresh
NullPool connection. The bootstrap environment check no longer refuses these
drives; it warns once that write-concurrency is reduced and recommends ext4.
Network filesystems remain refused: their locking is unsafe regardless of journal mode.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The download dispatcher (_dispatch_new_downloads) reads the `new` downloads
then claims each by writing last_download_attempt, all in one ORM
transaction. Since ORM transactions begin *deferred*, that write is a lock
upgrade (SHARED->RESERVED). SQLite skips busy_timeout for lock upgrades to
avoid deadlock, so under concurrency the write failed instantly with
"database is locked" instead of waiting out the 30s busy_timeout. This
surfaced repeatedly on the SQLite build after the Postgres->SQLite cutover.
Add get_immediate_db_session(), a committing session that takes the write
lock up front (BEGIN IMMEDIATE) so busy_timeout absorbs the contention --
mirroring what the raw-cursor get_db_curs path already does. Plain
get_db_session stays deferred so ordinary writes and reads keep WAL
read-concurrency; only read-then-write call sites opt in. Convert the
dispatcher to use it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Triage of Greptile findings on the default-Zim / configurable-directory PR:
- Stale config on restart (P1): settings update() only queues an async
background_save, and the restart switch has no ordering guarantee (the switch
worker pops LIFO), so start_kiwix_serve.sh could read the old zims_destination.
Save the config synchronously before activating the restart switch.
- YAML null (P1): read_config_value.sh now treats an unquoted null/~ as unset
(returns the default), matching PyYAML/the Python config layer. A quoted
"null" remains the literal string.
- Path traversal (P1/security): the settings API now rejects a zims_destination
containing "..", and start_kiwix_serve.sh defends in depth by falling back to
`zims` for any absolute/traversal/empty value, so Kiwix can never scan or
rewrite library.xml outside the media root.
- Reproducible build (P2): pin the zim-tools image to 3.7.0 instead of :latest.
Tests added for null handling, traversal rejection (API + script).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The image builds (pi-gen, debian-live, Docker) already ship wrolpi/blobs, but a
fresh install on existing Debian and an in-place upgrade did not pick up the
default Zim. repair.sh runs on both paths (both install scripts and upgrade.sh
call it) and already manages /opt/wrolpi-blobs immutability, so copy the
committed blobs there -- after the immutable flag is cleared and before it is
re-applied. Local copy only, preserving repair.sh's no-internet guarantee.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two related gaps in how WROLPi serves Kiwix content:
1. A fresh WROLPi (or one whose zims are all corrupt/absent) showed a blank
Kiwix. Every image now ships a small built-in default Zim that Kiwix falls
back to, telling the user to download real Zim files.
2. The startup scripts hardcoded the `/zims` directory, ignoring wrolpi.yaml's
`zims_destination` (which the Python side already honors).
Changes:
- New wrolpi/scripts/read_config_value.sh: a zero-dependency (awk) reader so
non-Python scripts can reliably read top-level wrolpi.yaml scalars. Also a
config_value() wrapper in wrolpi/scripts/lib.sh.
- New wrolpi/scripts/default_zim/: HTML source + a reproducible builder
(ghcr.io/openzim/zim-tools) that outputs the committed wrolpi/blobs/default.zim.
- New wrolpi/blobs/: any committed blob here is copied to /opt/wrolpi-blobs by
the pi-gen and debian-live builds; the Docker zim image bakes it in.
- start_kiwix_serve.sh honors zims_destination and falls back to the default
Zim when no functioning zim is found.
- docker/zim/entrypoint.sh gains the same default-Zim fallback. It needs no
config read: it already scans the whole mounted media dir, and
zims_destination is always a subdirectory of that mount.
- Changing zims_destination via the settings API now restarts Kiwix so it
serves the new directory without a reboot.
Tests: new test_read_config_value.py, extended test_start_kiwix_serve.py
(configured dir + default-Zim fallback), and a root_api test for the restart.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Kiwix section only checked library.xml and the Caddy HTTPS proxy on 8085,
which made a common failure hard to diagnose: kiwix-serve down while the
wrolpi-kiwix unit still reads "running" (its ExecStart is a bash wrapper), and
corrupt/incomplete zims silently missing from the library.
- Check the wrolpi-kiwix systemd unit exists and the service is up (parity
with the api/app/controller sections).
- Check kiwix-serve is actually listening on 9085 and responds over plain
HTTP there, isolating a kiwix failure from a Caddy proxy failure on 8085.
- Compare *.zim files on disk to <book> entries loaded in library.xml; a
shortfall flags corrupt or incompletely-downloaded zims.
Verified against a healthy Debian 13 RPi (10.0.0.9).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
