The library has been uninstalled for some time, but the codebase still read as
though it were there: identifiers named after it, rules written against its
class names, and about 180 comments explaining a rule in terms of a thing that
is no longer installed.
Renamed, keeping the vocabulary the app actually uses: semanticUIColorMap ->
navColorHexMap, semanticColorNames -> paletteColorNames, semanticNames ->
iconNameAliases, semanticSizes -> sizeAliases, and SemanticUIColors ->
NavColors in wrolpi/schema.py, which is what validates nav_color on the way in.
ColorToSemanticHexColor had no callers at all and is gone.
The baseline ratchet has done its job -- the list reached zero, so the JSON is
deleted and the guard keeps only the checks with teeth: no package, no import,
no `className="ui "`, no toast library. Its Theme.tsx check went with the
wrappers it was watching for. It also asserts its own walk found files, so a
clean result cannot mean it scanned nothing.
Deleted from App.css: every rule requiring `.ui`, `.inverted`, `.segment` or
`.modal`, none of which any component has set since the stylesheet stopped
loading, plus the form skeletons, ui-alerts and progress-text rules that
nothing renders. Two of them were doing harm rather than nothing: the drag
tints had a dark-theme variant under `.inverted` that stopped applying, leaving
15% of a fixed blue over a dark row, and now mix a token instead.
Two selectors naming markup that no longer exists were live bugs. The file
browser's filter had `.ui.input.file-browser-filter`, so it rendered at the
browser default width on a bar laid out to give it 34em -- now qualified by its
container, because `.wrolpi-action-input` sets `width: 100%` at the same
specificity and is loaded later. And SearchBox takes named props rather than a
rest spread, so the `id` DestinationForm hands it was dropped in silence and
the label it renders pointed at nothing; the download forms' Destination field
has had no associated label since the migration.
"Semantic roles" survives in four places, where it means what it says.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Panels shared edges: two 1px borders meeting, which reads as one tall
surface with a rule across it rather than a seam between two.
Done as `margin-bottom` on the panel first, which is the anti-pattern, and
it failed in both of the ways that rule is known to fail before the reading
was done. It compounded with the gap of every Stack that already held
panels -- four of them, spacing those lists at 35.2px against the app's
17.6px -- and the `:last-child` reset meant to keep trailing space off the
bottom of a container silently did nothing, because Media wrapping makes a
panel the only child of its own wrapper. On Status that separated four
pairs out of five and left the first touching.
So the space belongs to whatever stacks the blocks. `.wrolpi-stack` is on
PageContainer, which every page uses, and on the three wrappers that stack
blocks somewhere the page's own stack cannot reach: Settings' panel wrapper,
the block below the video player, and the Zim and Map route wrappers, which
hold their tab bar outside PageContainer.
A gap rather than `> * + * { margin-top }`, which was tried and is quietly
broken here: Media injects `.fresnel-container { margin: 0; padding: 0 }` at
runtime, same specificity, later source order, so every wrapped block threw
the margin away. A child cannot zero its parent's gap. It also means
Media's hidden half contributes nothing, where the owl selector spaced
around something invisible.
With the page owning the spacing, the blocks that owned some of it had to
give it up, or their margin was added to the gap: the tab bar (33px against
17.6 everywhere else), the shared search control bar and paginator on
Videos/Archives/Docs/Files, the card grid, the file browser's filter row and
the inventory search. Two margins deliberately stay -- the file browser's
4em, which reserves room for the sticky buttons (62645831), and heading
margins, which bind a heading to the content below it.
Tests plant each of the three failures: the wrapped block, a child that
zeroes its own margin, and a hidden Media half. Verified in the browser on
sixteen routes and in all four themes -- every stacked pair 17.6px, the
map's 85vh box and the video player unchanged, no horizontal overflow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The interface reads about 10% smaller than the pre-migration Semantic build, and the
drift is not one number. Measured on the QA Pi at 1280px, Semantic's headings were
h1 28 / h2 24 / h3 18 / h4 15 / h5 14 against our 26 / 21 / 17 / 14 / 13, and its video
card a fixed 290px with an 18px title against our 238px with a 13px one -- while our
BODY text and chrome went the other way, a 16px base against Semantic's 14 and 42px
buttons against 36. No per-component nudge reproduces what a user comparing the two
sees, which is the whole page at once.
Ten percent is not a fudge factor: our heading scale times 1.1 is 28.6 / 23.1 / 18.7 /
15.4 / 14.3, which is Semantic's scale to within a pixel at every step. Verified in
the browser -- those are the numbers it now renders.
`--ui-scale: 1.1` and `html { font-size: calc(100% * var(--ui-scale)) }`, with
everything that should grow written in rem:
- 102 lengths in ui.css, App.css and tokens.css converted, restricted by property:
font-size, padding, margin, gap, width/height, border-radius, offsets.
- Borders, outlines and shadows deliberately NOT converted. A hairline is a hairline
at any text size.
- 46 inline `fontSize` px values across 8 files. `em` values were left alone: they
already resolve against the parent.
- Ten inline-styled blocks in the library (Card's text block, Loading, Placeholder,
ColoredInput, Confirm's action row, CardGroup's gap) moved into ui.css, because an
inline px size cannot be reached by a scale or seen by a test.
- CardGroup converts its `minWidth` prop to rem, which is what returns a phone to one
card per row: as px the track ignored the scale, so a 430px screen fitted two 200px
cards where the old build fitted one 290px card.
Mantine needs nothing: its 644 size declarations are `calc(Xrem * var(--mantine-scale))`
and scale with the root. `theme.scale` must stay at 1 -- it exists to CANCEL a
customised root font-size, so setting it to 1.1 as well would leave Mantine's half of
the interface at the old size beside our larger text. A test pins that.
`zoom: 1.1` on the root was tried first and rejected. It works and it is one line, but
it scales at paint: every 1px hairline becomes 1.1px and softens on a 1x display, and
viewport units do not shrink to compensate, so `100vh` paints 110% of the screen -- a
95vw modal overflowed sideways and the CBZ viewer's fullscreen overflowed down. Rem
scales real lengths, keeps borders crisp, leaves viewport units alone, and preserves a
user's own browser font-size preference instead of overriding it.
A percentage rather than a px root size, for that last reason.
Five guards in themes/ui-scale.test.js, each planted against: the scale itself, that
every scalable declaration is rem, that borders are NOT, that no inline px font-size
survives anywhere including the gallery, and that Mantine's scale stays 1.
Also removes `.ui.modal.themed-confirm`, a dead Semantic-era rule and the source of the
one `95vw` that would have overflowed: `.ui.modal` no longer exists and nothing sets
that class.
Verified: 1141 tests / 65 suites; clean build; browser-checked at 1280px and at a 430px
phone; navbar single-row from 780 to 1800px; hairlines still 1px and no horizontal
overflow in all four themes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
These five carried no `size` at all, so they rendered at Mantine's 440px default
-- a width nobody had chosen, inherited from the library rather than decided.
Roland audited them:
FileBrowser rename small (440, unchanged)
FileBrowser move large (620)
FileBrowser ignore small (440, unchanged)
FileBrowser make directory large (620)
Files filter small (440, unchanged)
The three "unchanged" ones are written down rather than left implicit, so the
decision lives in the source and a future change to Mantine's default cannot
move them.
Roland asked for `medium` on the ignore dialog and there is no such modal size:
the vocabulary is mini/tiny/small/large/fullscreen, and Mantine has no token
between 440 and 620. An unknown name would have passed through untranslated,
rendered at 440 anyway, and left the source claiming otherwise -- which is what
the size guard in ui.test.js was added to catch. Asked rather than guessed;
`small` was the answer, and the guard would have failed the build either way.
Verified in the browser rather than from the props: move 620, make directory 620,
ignore 440, rename 440. Move and ignore need a selection before they can be
opened, so a directory was selected first.
1134 jest / 64 suites, clean npm run build. The full suite needs
`--maxWorkers=2` on this machine at the moment: at load average 38 four unrelated
suites (ControllerPage, CollectionReorganizeModal, CollectionTagModal, Flasher)
time out and the run takes 183s instead of 5s. They pass in isolation and with
reduced parallelism; nothing in this change touches them.
Two things, both reported from the running app.
Nested modals could not exist. Mantine binds a window-level keydown listener
per open modal, gated only on whether that modal is open -- nothing checks which
one the user is looking at:
useWindowEvent('keydown', (event) => {
if (event.key === 'Escape' && closeOnEscape && ... && opened) onClose();
}, {capture: true});
Two open modals meant two listeners, so one Escape closed both. Reproduced on
the dashboard: the download modal plus the search modal over it, one press, both
gone, back to the page. Both also sat at z-index 200, so which one drew on top
was DOM order rather than which the user opened.
Mantine's own answer is Modal.Stack with a stackId, which is the only thing that
gates closeOnEscape. Not used, for two reasons: every call site would have to
be rewritten to hoist its modal into a stack, and it sets `__hidden` on
everything below the top -- the parent disappears while the child is open. The
parent should be dimmed and still there, as it was before the migration.
So the wrapper keeps the register itself: a module-level list of open modals,
most recent on top, from which each modal takes closeOnEscape, trapFocus and a
z-index two above its parent. Module-level rather than React context because
the two modals in the bug are not nested in the JSX at all -- the search modal
is rendered by KeyboardShortcutsProvider near the app root and the download
modal by the dashboard. They are siblings that happen to be open together, and
no context relates them. The increment is small on purpose: Mantine puts
popovers at 300, and a stack that climbed past it would paint over the select
dropdowns inside its own forms.
Widths: Roland audited the modal list and chose sizes against Mantine's scale.
Sixteen get wider -- twelve to `large` (620), four to `fullscreen` -- and eleven
keep the width they had, with the size now written down so the decision is in
the source rather than in a default. Flasher was passing Mantine's raw `sm`,
which worked but bypassed the naming the audit used; it says `tiny` now.
The size table itself is guarded, because it is the single thing that halved
every modal during the migration and a quiet edit to it moves seventy at once.
Semantic's widths, measured on the pre-migration build still running on the QA
Pi: mini 360, tiny 540, small 720, large 1080, no-size 900. Mantine's, read out
of its stylesheet: 320/380/440/620/780. A `small` modal went 720 -> 440 and one
with no size went 900 -> 440. Three cases now hold the scale, the mapping, and
the rule that no call site may name a size the table does not know -- an unknown
name is passed to Mantine, ignored, and silently rendered at the default.
Tested in a real browser: none of it is visible to jsdom -- the listener is on
window with capture, the stacking is z-index, and "still visible behind" is
geometry. Note that `should('be.visible')` is the wrong assertion for the
parent: it is an occlusion check, so a child drawn correctly over its parent
trips it. The test uses a wide viewport and asserts the parent is rendered and
shows to both sides of the child. Planting the pre-fix behaviour reddens three.
1124 jest / 62 suites, 8 in modal-stack.cy.js, clean npm run build. The 12
failures in Tags/Channels/Download/Inventory.cy.js are the known pre-existing
Semantic-era selector failures, unchanged.
It sat hard against the left edge of its container despite three call
sites wrapping it in `<center>` or `text-align: center`.
Mantine builds a Pagination as a BLOCK root -- full width, so those
wrappers had nothing to centre -- holding a flex Group that carries the
controls and lays them out at `flex-start`. So `justify-content` on the
root does nothing at all; it has to reach the Group.
Setting `--group-justify` on the root does not work either, which cost a
detour: Group defaults that prop and writes the variable INLINE on its own
element, where an inherited value cannot reach it. The property itself is
fair game, since Mantine only ever reads the variable from a hashed class,
so `.wrolpi-pagination .mantine-Group-root { justify-content: center }` at
(0,2,0) is what lands.
The three wrappers are now plain divs keeping their margin. `<center>` is
a deprecated element that was doing nothing here.
Two tests: the gap either side is equal -- measured, not read off
`justify-content` -- with an assertion that the container really is wider
than the strip, since otherwise the equality holds trivially at 0 and 0.
And an inverse: at 240px nothing hangs off the LEFT edge. That one is the
risk centring introduces -- overflow in a flex-start row spills right,
where it can be scrolled to, but centred it spills both ways and the first
page becomes unreachable. It passed before this change and must keep
passing after.
Verified red first (0px left against 230px right), and the wrong-element
fix -- centring the root rather than the Group -- reddens it again.
1080 jest, 160 in ui-layout.cy.js, clean build.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sixteen call sites, chosen by reading each one rather than by grepping for a
hue. Four sub-agents judged them independently against one rule: convert only
what genuinely means success or failure; identification, category and
decoration stay hue-named.
Converted:
SMART drive health (all four branches -- PASS/WARN/FAIL/unknown)
config validity: valid / invalid / empty
"Failed to load preview"
the two "Download error" triggers
expired inventory items, desktop and mobile
the refresh phases: complete / active / not reached
a refresh Progress bar in its error state
cookie status: Unlocked / Locked
a chip-mismatch label on the flasher
CPU load thresholds (LoadStatistic)
the outcome of an API call in useAPIButton
the gallery's "Danger zone" heading and its load readings
Left alone, deliberately:
the file browser toolbar -- every action has its own colour to be told apart,
and delete already uses role='danger' (Roland's ruling)
diff add/remove in the config and inventory import previews. Adding is not a
success and removing is not a failure; in Overwrite mode the removals are
exactly what the user asked for, and painting them danger would report their
intended action as broken
seasons in DatesSelector, the uploader star and favourite heart, TagIcon,
downloader category buttons, "Recommended to Keep"
Three sites were wrong in a way worth naming. Both "Download error" triggers
guard on the same condition and render the same modal, but one was `orange` and
the other `red` -- and `orange` is the single worst choice, byte-identical to
`--text` in night, so that error button was indistinguishable from an ordinary
control. Config validity drew its valid state in `--violet`, which no user
parses as "valid". And `LoadStatistic` -- the thing users actually watch on the
Status page -- rendered its warning threshold in that same invisible orange.
Also fixed a stale comment in Videos.js that claimed no token existed for these.
Two notes on the tests:
The first LoadStatistic test I wrote rendered the component and asserted the
inline colour. jsdom REJECTS `color: var(--danger)` as an invalid declaration
and drops it, so every colour reads back as the empty string -- two of the four
assertions passed for that reason alone and would have kept passing with the
hues restored. It is a source guard now, with the painted result asserted in
the browser instead.
That browser test then found something real: an uncoloured reading inherits
`--text`, which in night is 4.98:1 against `--warning`'s 4.91:1, so a machine
under load is marginally QUIETER than one that is fine. Not fixed here. The
only way up on a dark panel is lightness, and night's danger would desaturate
from #ff3e3e to about #ff5d5d -- trading the theme's "red only" character for
loudness. That is a palette decision, so the test documents it and asserts what
does hold rather than quietly making it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two agents independently dropped it, each because our Card had no `color` prop
and each citing the other's precedent -- which is how a deliberate feature
quietly becomes a convention. Roland wanted it back.
Card takes a token colour name and draws it as a 3px edge along the top rather
than tinting the surface: surfaces stay flat per the design rules, and an edge
survives night mode without becoming a bright patch. Because it is a token, the
accent remaps per theme -- four hues in light and dark, a single one in night and
amber, which is the point of those themes.
Restored on every card that shows a file: the video, image and generic file cards
in Files.js, and the archive card. All of them take it from
`mimetypeColor(mimetype, path)`, the same helper that colours the file icons, so
a card and its icon agree.
Phase 2, part 4. Baseline 50 -> 15.
Migrated: all eight calculators, the rest of inventory and collections, the
Map/Zim/Docs cluster, Videos, Files, Search, Archive, DashboardPage,
DatesSelector, HelpModal, Events, Tags, Flasher's neighbours, admin Settings
and Status, Download, and App.
Also the three shared modules everything imports: api.js and customHooks.js
swap the Semantic toast library for the library's shim (same call shape), and
useForm.js renders the library's Message. useForm accepts both the old
`{content, header, positive|negative}` shape and the new `{kind, title}`,
because the pages building those objects migrate separately.
Two library gaps closed, both found by call sites needing them:
ActionInput is a forwardRef now. Its ref reaches the input rather than the
wrapper, because callers use it to focus or select the field -- a ref to the
surrounding div would do neither, which is why one call site had resorted to
hand-copying the component's markup.
Header takes a `color` token name. Three separate call sites had reached for
an inline style to colour a heading that carries meaning ("3 items to add"
against "2 to remove"), which is how a hex ends up in markup.
Two files needed repair after being interrupted mid-edit: MapViewer had a
Segment left in its error branch, and VideoPlayer had lost the useContext
import its own code still used. Neither jest nor tsc noticed; only
`npm run build` did, which is the whole reason that check exists.
Fifteen files remain: Archive, BulkTagModal, Channels, Docs, FileBrowser,
FilePreview, Flasher, Nav, Playlists, Theme.tsx, Upload, VideoPlayer, Zim,
ControllerPage, admin/Downloads. Theme.tsx is deliberately last -- the
unmigrated files still import its Semantic-backed wrappers, so it can only go
once they have.
The DeepSearchHint only appeared on zero results, so a fast search with
matches gave no indication that captions/contents were not searched.
Module searches have no deep estimate, so show a generic reminder
whenever a fast search is active: "More results may be available in
deep search." with results, "Not finding what you need?" without.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The file_group.textsearch tsvector indexes all four text columns;
d_text (captions, document text, zip contents) is ~90% of the data,
making ts_rank searches take seconds on large libraries. Add a
textsearch_abc generated column (title/author/description only, ~8x
smaller) and default all searches to it, with deep=true opting into the
full column.
Database:
- Migration adds the generated column and a GIN index (rewrites the
table; a few minutes on large libraries).
Backend:
- FilesSearchRequest, VideoSearchRequest, ArchiveSearchRequest, and
DocSearchRequest accept `deep`; each search function picks the column.
- /api/search_file_estimates returns both counts in one index-bound
query (COUNT FILTER on the abc subset): `file_groups` (fast) and
`file_groups_deep`.
UI (one coherent story across the count surfaces):
- The typing dropdown always suggests the deep count (what is findable).
- The tab badge shows the active mode's count so it matches the result
list.
- The gap is reconciled by a hint offering deep search: count-aware on
the global search ("No results in fast search. N results available in
deep search." / "N more results available..."), generic on the module
pages (no deep estimate there). The hint resets pagination.
- "Search Depth" toggle in the search filter modals, persisted as
?deep=true.
Tests that relied on d_text matches now pass deep=True, with fast-mode
assertions alongside. Also skip flaky test_download_channel on
CircleCI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Restore the censored-video search filter
Censored videos (no longer publicly available — detected when a channel
refresh drops them from the channel list, or when comment downloads fail
with "video unavailable/private/removed") are already flagged on
FileGroup.censored, but there was no way to find them since the search
rework dropped the old filter.
Add a "Censored" toggle to the reworked search filter modal, shown only on
the Videos page (showCensored). When enabled the video search returns only
videos with FileGroup.censored = TRUE, tracked via a ?censored=true query
param (useSearchCensored).
* Move Availability filter below the Tags/Results row
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Consolidate search options into a single Filter modal
Reworks the file-search UI on the Dashboard, Videos, Archives, and Docs
pages. The crowded row of controls (sort, tags, limit, file-type filter,
date selector) is replaced with a single "Filter" button that opens one
comprehensive modal.
- Add SearchFilterModal + SearchFilterButton (violet + count badge when
active). Sections (Sort, File Type, Tags, Results Per Page, Published
Date) are shown per-page via props; layout uses a Semantic Grid.
- Selections are drafted locally and applied in a single updateQuery only
on close/Done, so the search isn't re-run on every field change.
- Results Per Page and Sort use buttons (fewer clicks, no clipped dropdown).
- Extract shared SearchControlBar (view toggle + input + Filter) for the
three results pages; move the global-search view toggle onto the tab row.
- Rework the Dashboard search line into one adaptive flex row; clear button
only shows when there's an active search.
- Remove now-dead FilesView limit/tag returns and SearchLimitDropdown.
* Address review: reset limit on Clear All; use replace over replaceAll
- handleClearAll now resets Results Per Page to the default page size
(DEFAULT_SEARCH_LIMIT = 24) so "Clear All" actually clears every setting.
- Replace start-anchored replaceAll(/^-/g) with replace(/^-/) for the sort key.
- Add a test covering the Clear All limit reset.
* Remove now-unused Media import from Files.js
The FilesSearchView refactor dropped the last <Media> usage in this file,
leaving an unused import (a build warning). Remove it.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Create a new Docs module that consolidates PDF and eBook handling into a
unified system with subjects and authors support. This replaces the
separate PDF/eBook modelers with a single Doc model backed by an
alembic migration (rename ebook to doc).
Key changes:
- New modules/docs package with models, API, extractors, and lib
- Subject and author metadata extraction with case-insensitive
normalization for cleaner PDF metadata
- Frontend Docs pages with AuthorsPage, SubjectsPage using
CollectionTable, and doc card deletion
- Global search integration for subjects and authors
- Generic collection item count support
- Frontend file card metadata adjusted to match collection sort
- Bump yt-dlp version
* Batch reorganization is supported for both Domain and Channel Collections.
* Batch reorganization happens when any Collection does not match the respective file format. For example, a Channel Collection file format that does not match the `yt_dlp_options.file_name_format` will show `needs_reorganization`.
* Individual Collections can always be reorganized. The reorganization is a little more aggressive. Useful for when some files may have been missed in the batch reorganization.
Introduces FileWorker for background file operations with dual-queue
architecture (public/private). Handles refresh, move, and rename tasks
with progress tracking, job completion monitoring, and batch processing.
Compares DB FileGroups to filesystem state using FileGroupDiff to detect
new/modified/deleted files. Designed for memory efficiency on Raspberry Pi.
Storing video settings in session storage to help user start downloads consistently.
Applying channel tag name to new channels when download videos or channels.
Initializing map tiles when importing and resetting map.
Parsing SingleFile datetime using dates.strpdate.
Adding timezone when no timezone info is available.
Installing python-dateutil to assist in handling diverse datetime strings.
Reworking refresh progress to use idempotency to display more accurate progress. Only logging that files have been invalidated if they used to be indexed.
Clearing archives to force re-index.
Ignoring 'Unknown' author from PDF metadata. Using file_title when pdf metadata title is empty.
* A new zim module has been added, this module allows several things to interact with Zim files.
* Allow searching of Zim files using libzim.
* Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
* Creation of a new zim docker container which allows viewing of the Zim file contents.
* This commit also adds move/rename/mkdir features in the File Browser.
* File tags are preserved when moving/renaming.
Moving upload destination input out of drop Segment.
Disabling uploads when downloads are disabled.
Inverting labels for checkboxes in inverted modal.
Deleting file_groups which no longer contain the exclusive primary path.
Adding debug logging for all API errors.
Checking that upload output is not a directory.
Expanding nginx request size.
Adding docs to upload endpoint.
Using dot-circle icon for more image mimetypes.
Using Form.Field to make upload destination required.
Adding border to inverted Segments.
Many files can be uploaded in chunks.
Files can be uploaded, progress is displayed.
Showing white border around share QR code.
Displaying WROLPi colors for download icons.
Creating button to clear DirectorySearch.
Excluding channel/domain directories so other directories can be returned.
Ordering search results.
Ignoring when no channel is found only if we have a destination.
Limiting directory search results.
Displaying categories when searching destination directories.
Handling when live_chat.json file exists.
Allowing refresh of directory directly.
Logging which download was successful.
Removing timeout.
Creating index for directory names.
Sleeping to catch cancel.
Arbitrary files can be downloaded using the new FileDownloader.
