Four defects on /videos, /docs and /archives.
Titles were invisible outside light. `.card-link` carried a literal
`color: black` from before the themes existed -- about 1.1:1 on the dark,
night and amber panels. It applies to every card title and to the
channel/author/domain line beside it. The date was fine, since it takes
`--muted` from the card body, which is what made this look like a card
bug rather than a link bug. Titles now take `--text`: they are links,
but they are the card's text, not a wall of blue.
Posters were cropped, not mis-proportioned. The image carried a fixed
290x163 box; a card in the default grid is about 233px wide and Mantine's
Card sets `overflow: hidden`, so the browser painted the middle 233px of
a 290px image. The ratio was right in the layout tree and wrong on
screen. Replaced with caps on both axes -- the card's width and
`--card-poster-max-height: 200px` -- with `width`/`height` left `auto` so
the pair behaves as "contain". Bounding the width alone was tried first
and is not enough: a square thumbnail then stands as tall as the card is
wide, and a portrait book cover taller still.
The meta line was foot-anchored, so on /docs a PDF with no author had its
date shoved up to 55px below the title, at the bottom of a card that was
tall only because of its neighbour in the row. It now sits directly
under the title.
Archive cards read buttons-then-domain, because the buttons were children
and the meta was what got pushed to the foot. Card gains an `actions`
slot, rendered last and anchored to the foot, so a row lines its buttons
up however many lines each title took.
Gallery gains a poster row covering all five ratios and a row showing
meta/actions against unequal titles.
1076 jest tests, 145 in ui-layout.cy.js, clean build. Every new guard was
checked against a planted regression: dropping the height cap reddens the
square and portrait cases and leaves 16:9 green; `width: 100%` reddens
containment; restoring `color: black` reddens dark/night/amber and leaves
light green; re-anchoring the meta and swapping the slot order redden both
the jest order test and the cypress geometry tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two agents independently dropped it, each because our Card had no `color` prop
and each citing the other's precedent -- which is how a deliberate feature
quietly becomes a convention. Roland wanted it back.
Card takes a token colour name and draws it as a 3px edge along the top rather
than tinting the surface: surfaces stay flat per the design rules, and an edge
survives night mode without becoming a bright patch. Because it is a token, the
accent remaps per theme -- four hues in light and dark, a single one in night and
amber, which is the point of those themes.
Restored on every card that shows a file: the video, image and generic file cards
in Files.js, and the archive card. All of them take it from
`mimetypeColor(mimetype, path)`, the same helper that colours the file icons, so
a card and its icon agree.
Replaces every semantic-ui-react and ./Theme import with the token-driven
./ui library: Segment->Panel, Tab/Tab.Pane->Tabs (Tabs.List/Tabs.Tab/
Tabs.Panel), Loader active inline='centered'->Loading, Message/Message.Header
->Message kind/title, Statistic/StatisticValue/StatisticLabel->Statistic
value/label, Card/CardContent/CardHeader/CardMeta/CardDescription->Card
media/title/meta/children, Input label={...}->ActionInput action={...},
Form.Dropdown->Select, Form/Form.Field/TextArea->plain markup + Textarea.
Converts all three Semantic grid spellings (Grid.Row/Grid.Column, bare
GridRow/GridColumn) to flexbox for the two-up rows in ArchivePage and
DomainsPage, and to plain "row"/"column" divs (CollectionEditForm's existing
CSS-class convention, shared with still-unmigrated Channels.js/Playlists.js)
for the fields passed into CollectionEditForm in DomainEditPage.
ArchiveFileNameForm no longer goes through hooks/useForm's InputForm helper,
which still renders Semantic's Form.Input internally (hooks/useForm.js is
off-limits for this migration) -- it now builds its field directly against
form.getInputProps, mirroring the pattern Download.js already established.
Drops ThemeContext's obsolete i/s/t/inverted spreads throughout, and the
mimetype-color accent on Card (the new Card has no color prop; Videos.js's
already-migrated VideoCard dropped the same accent for the same reason).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Greptile P1, and a real one: App.js migrated and took `<SemanticToastContainer/>`
with it, but seven pages -- Archive, Channels, Docs, FilePreview, Playlists,
Zim, ControllerPage -- still raised their toasts through the Semantic library.
With no container mounted those notifications went nowhere. No error, no
warning; a user deletes a document and is simply never told whether it worked.
Fixed by pointing those seven at the library's `toast`, which takes the same
options, rather than by mounting a second toast system. Nothing calls the
Semantic toast library now, so a test asserts that at zero: a file that goes
back to it would raise notifications that silently never appear, and that is
not something a reviewer would spot.
Also reverted Archive.js to its last good state. An interrupted agent had
migrated part of it -- Panel, Tabs, and Mantine's Grid.Col -- while the rest of
the file still used Semantic's Segment, Tab and Grid.Row, so a single `Grid`
identifier had to be two different components at once. A half-migrated
1400-line file is worse than an unmigrated one; it goes back on the list and
gets done in one piece.
That break was masked: react-scripts caches ESLint results per file, so the
earlier build passed while Archive.js sat on a stale pass. Editing its import
invalidated the entry and the errors appeared. Verified with the cache cleared
this time, which is the only way that check means anything after a wide change.
Phase 2, part 4. Baseline 50 -> 15.
Migrated: all eight calculators, the rest of inventory and collections, the
Map/Zim/Docs cluster, Videos, Files, Search, Archive, DashboardPage,
DatesSelector, HelpModal, Events, Tags, Flasher's neighbours, admin Settings
and Status, Download, and App.
Also the three shared modules everything imports: api.js and customHooks.js
swap the Semantic toast library for the library's shim (same call shape), and
useForm.js renders the library's Message. useForm accepts both the old
`{content, header, positive|negative}` shape and the new `{kind, title}`,
because the pages building those objects migrate separately.
Two library gaps closed, both found by call sites needing them:
ActionInput is a forwardRef now. Its ref reaches the input rather than the
wrapper, because callers use it to focus or select the field -- a ref to the
surrounding div would do neither, which is why one call site had resorted to
hand-copying the component's markup.
Header takes a `color` token name. Three separate call sites had reached for
an inline style to colour a heading that carries meaning ("3 items to add"
against "2 to remove"), which is how a hex ends up in markup.
Two files needed repair after being interrupted mid-edit: MapViewer had a
Segment left in its error branch, and VideoPlayer had lost the useContext
import its own code still used. Neither jest nor tsc noticed; only
`npm run build` did, which is the whole reason that check exists.
Fifteen files remain: Archive, BulkTagModal, Channels, Docs, FileBrowser,
FilePreview, Flasher, Nav, Playlists, Theme.tsx, Upload, VideoPlayer, Zim,
ControllerPage, admin/Downloads. Theme.tsx is deliberately last -- the
unmigrated files still import its Semantic-backed wrappers, so it can only go
once they have.
Catch useForm fetcher errors and surface them so DomainEditPage can show a not-found message instead of hanging on the loader (and so the webpack overlay no longer flakes Cypress).
The file_group.textsearch tsvector indexes all four text columns;
d_text (captions, document text, zip contents) is ~90% of the data,
making ts_rank searches take seconds on large libraries. Add a
textsearch_abc generated column (title/author/description only, ~8x
smaller) and default all searches to it, with deep=true opting into the
full column.
Database:
- Migration adds the generated column and a GIN index (rewrites the
table; a few minutes on large libraries).
Backend:
- FilesSearchRequest, VideoSearchRequest, ArchiveSearchRequest, and
DocSearchRequest accept `deep`; each search function picks the column.
- /api/search_file_estimates returns both counts in one index-bound
query (COUNT FILTER on the abc subset): `file_groups` (fast) and
`file_groups_deep`.
UI (one coherent story across the count surfaces):
- The typing dropdown always suggests the deep count (what is findable).
- The tab badge shows the active mode's count so it matches the result
list.
- The gap is reconciled by a hint offering deep search: count-aware on
the global search ("No results in fast search. N results available in
deep search." / "N more results available..."), generic on the module
pages (no deep estimate there). The hint resets pagination.
- "Search Depth" toggle in the search filter modals, persisted as
?deep=true.
Tests that relied on d_text matches now pass deep=True, with fast-mode
assertions alongside. Also skip flaky test_download_channel on
CircleCI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Consolidate search options into a single Filter modal
Reworks the file-search UI on the Dashboard, Videos, Archives, and Docs
pages. The crowded row of controls (sort, tags, limit, file-type filter,
date selector) is replaced with a single "Filter" button that opens one
comprehensive modal.
- Add SearchFilterModal + SearchFilterButton (violet + count badge when
active). Sections (Sort, File Type, Tags, Results Per Page, Published
Date) are shown per-page via props; layout uses a Semantic Grid.
- Selections are drafted locally and applied in a single updateQuery only
on close/Done, so the search isn't re-run on every field change.
- Results Per Page and Sort use buttons (fewer clicks, no clipped dropdown).
- Extract shared SearchControlBar (view toggle + input + Filter) for the
three results pages; move the global-search view toggle onto the tab row.
- Rework the Dashboard search line into one adaptive flex row; clear button
only shows when there's an active search.
- Remove now-dead FilesView limit/tag returns and SearchLimitDropdown.
* Address review: reset limit on Clear All; use replace over replaceAll
- handleClearAll now resets Results Per Page to the default page size
(DEFAULT_SEARCH_LIMIT = 24) so "Clear All" actually clears every setting.
- Replace start-anchored replaceAll(/^-/g) with replace(/^-/) for the sort key.
- Add a test covering the Clear All limit reset.
* Remove now-unused Media import from Files.js
The FilesSearchView refactor dropped the last <Media> usage in this file,
leaving an unused import (a build warning). Remove it.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add Playlists: ordered, mixed-type collections
A playlist (Collection kind='playlist') is a manually-curated, ordered list that
mixes item types. Squash of the playlists feature branch (24 commits).
Data model & API:
- CollectionItem is polymorphic via item_kind: 'file' (FileGroup), 'zim'
(zim_id + zim_entry), or 'url' (link to anything the WROLPi browser opens,
e.g. /map?lat=&lon=&z=). CHECK constraint enforces exactly one kind; per-kind
unique constraints; migration a7b1c2d3e4f5.
- Endpoints: create playlist; add/remove/reorder items (idempotent adds: 201 new,
200 existing); GET returns the ordered items. All mutations wrol_mode_check'd.
Config (source of truth):
- playlists.yaml (PlaylistsConfig) stores each playlist's metadata AND ordered
items (file by relative path, zim by path+entry, url inline); round-trips a
DB wipe. Supports backup import (DB_CONFIG_FILE_NAMES).
- The redundant generic CollectionsConfig (collections.yaml) is deleted:
channels/domains own their configs, authors/subjects are re-derived.
On-disk sync (sync_playlists_directory switch):
- Each playlist materializes into the configurable Playlists Directory
(playlists_destination, Settings UI input, excluded from indexing) with
position-prefixed entries: NNNN_<file> hard links (copy fallback across
filesystems) and NNNN_<title>.html stubs for zim/url items.
- Tagged playlists are namespaced like channels: <playlists>/<tag>/<name>;
a tag change moves the directory and prunes the empty old tag dir.
- Deletion safety: managed files are only removed when they are our marked
stubs or still have another hard link (st_nlink > 1) -- never a last copy.
Stray loose files at the playlists root are removed; stub URLs are
scheme-allowlisted (no javascript:/data:).
Frontend:
- /playlists list page (CollectionTable, name filter, New Playlist modal, Tag
column) between Files and Zim in the nav.
- View-only detail page (Name link) and edit page (Edit button): themed table
with model icons/colors + poster thumbnails, items link to their model page
(video/archive/doc) or preview, up/down reorder, remove, add-URL form,
single-tag selector; all editing disabled in WROL Mode.
- Reusable green "Add to Playlist" button on Video/Archive/Doc detail pages,
the FilePreview toolbar, Zim search results, and Map pins (added as a /map
URL item with a map-marker icon).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Playlist edit page parity: rename, custom directory, tag modal move suggestion
Match the ChannelEditPage/DomainEditPage UX on the playlist edit page:
Back/View buttons, an edit form (name, description, directory), and the
shared CollectionTagModal with the violet "Tag" button. The tag modal now
suggests moving the playlist to its managed tag location
(playlists/<tag>/<name>); accepting the suggestion stores directory=None so
the sync keeps auto-managing the location. Tag buttons on the Channel and
Domain edit pages are now violet to match.
Backend: update_collection learns rename (with per-kind validation and
duplicate checks) and playlist custom directories (cleanup of the abandoned
directory, config round-trip, commit-before-switch to avoid the background
sync racing pre-commit state). tag_collection's duplicated untag/tag
branches are merged, and the per-kind config-save dispatch is consolidated
into _activate_config_save().
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Deduplicate Collection error handling and Tag creation
Delete the duplicate UnknownCollection in wrolpi/collections/errors.py and
re-export the canonical one from wrolpi/errors.py, which carries
status_code=NOT_FOUND (the local copy inherited 500). Replace the repeated
query/raise pattern in reorganize.py with Collection.find_by_id.
Add Tag.get_or_create_tag and use it for collection tagging; flush the new
Tag before returning so callers can read .id immediately -- previously
tagging a collection with a brand-new tag name set tag_id=None. Regression
test included.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Create a new Docs module that consolidates PDF and eBook handling into a
unified system with subjects and authors support. This replaces the
separate PDF/eBook modelers with a single Doc model backed by an
alembic migration (rename ebook to doc).
Key changes:
- New modules/docs package with models, API, extractors, and lib
- Subject and author metadata extraction with case-insensitive
normalization for cleaner PDF metadata
- Frontend Docs pages with AuthorsPage, SubjectsPage using
CollectionTable, and doc card deletion
- Global search integration for subjects and authors
- Generic collection item count support
- Frontend file card metadata adjusted to match collection sort
- Bump yt-dlp version
* Batch reorganization is supported for both Domain and Channel Collections.
* Batch reorganization happens when any Collection does not match the respective file format. For example, a Channel Collection file format that does not match the `yt_dlp_options.file_name_format` will show `needs_reorganization`.
* Individual Collections can always be reorganized. The reorganization is a little more aggressive. Useful for when some files may have been missed in the batch reorganization.
Replacing Channels and Domains with Collection.
Moving downloading relationships from Channel to Collection.
Created domains.yaml config to manage tagged domain collections.
Adding turn-key upgrade solution in the UI.
Storing video settings in session storage to help user start downloads consistently.
Applying channel tag name to new channels when download videos or channels.
Initializing map tiles when importing and resetting map.