Commit graph

118 commits

Author SHA1 Message Date
Roland
687b41bf20
ui: cards get legible titles, contained posters, and meta beside the title
Four defects on /videos, /docs and /archives.

Titles were invisible outside light.  `.card-link` carried a literal
`color: black` from before the themes existed -- about 1.1:1 on the dark,
night and amber panels.  It applies to every card title and to the
channel/author/domain line beside it.  The date was fine, since it takes
`--muted` from the card body, which is what made this look like a card
bug rather than a link bug.  Titles now take `--text`: they are links,
but they are the card's text, not a wall of blue.

Posters were cropped, not mis-proportioned.  The image carried a fixed
290x163 box; a card in the default grid is about 233px wide and Mantine's
Card sets `overflow: hidden`, so the browser painted the middle 233px of
a 290px image.  The ratio was right in the layout tree and wrong on
screen.  Replaced with caps on both axes -- the card's width and
`--card-poster-max-height: 200px` -- with `width`/`height` left `auto` so
the pair behaves as "contain".  Bounding the width alone was tried first
and is not enough: a square thumbnail then stands as tall as the card is
wide, and a portrait book cover taller still.

The meta line was foot-anchored, so on /docs a PDF with no author had its
date shoved up to 55px below the title, at the bottom of a card that was
tall only because of its neighbour in the row.  It now sits directly
under the title.

Archive cards read buttons-then-domain, because the buttons were children
and the meta was what got pushed to the foot.  Card gains an `actions`
slot, rendered last and anchored to the foot, so a row lines its buttons
up however many lines each title took.

Gallery gains a poster row covering all five ratios and a row showing
meta/actions against unequal titles.

1076 jest tests, 145 in ui-layout.cy.js, clean build.  Every new guard was
checked against a planted regression: dropping the height cap reddens the
square and portrait cases and leaves 16:9 green; `width: 100%` reddens
containment; restoring `color: black` reddens dark/night/amber and leaves
light green; re-anchoring the meta and swapping the slot order redden both
the jest order test and the cypress geometry tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 18:05:47 -06:00
Roland
6b1e05be86
ui: bring back the card mimetype accent
Two agents independently dropped it, each because our Card had no `color` prop
and each citing the other's precedent -- which is how a deliberate feature
quietly becomes a convention.  Roland wanted it back.

Card takes a token colour name and draws it as a 3px edge along the top rather
than tinting the surface: surfaces stay flat per the design rules, and an edge
survives night mode without becoming a bright patch.  Because it is a token, the
accent remaps per theme -- four hues in light and dark, a single one in night and
amber, which is the point of those themes.

Restored on every card that shows a file: the video, image and generic file cards
in Files.js, and the archive card.  All of them take it from
`mimetypeColor(mimetype, path)`, the same helper that colours the file icons, so
a card and its icon agree.
2026-08-08 18:04:03 -06:00
Roland
06f1ce46bc
ui: migrate Archive.js off Semantic UI React
Replaces every semantic-ui-react and ./Theme import with the token-driven
./ui library: Segment->Panel, Tab/Tab.Pane->Tabs (Tabs.List/Tabs.Tab/
Tabs.Panel), Loader active inline='centered'->Loading, Message/Message.Header
->Message kind/title, Statistic/StatisticValue/StatisticLabel->Statistic
value/label, Card/CardContent/CardHeader/CardMeta/CardDescription->Card
media/title/meta/children, Input label={...}->ActionInput action={...},
Form.Dropdown->Select, Form/Form.Field/TextArea->plain markup + Textarea.

Converts all three Semantic grid spellings (Grid.Row/Grid.Column, bare
GridRow/GridColumn) to flexbox for the two-up rows in ArchivePage and
DomainsPage, and to plain "row"/"column" divs (CollectionEditForm's existing
CSS-class convention, shared with still-unmigrated Channels.js/Playlists.js)
for the fields passed into CollectionEditForm in DomainEditPage.

ArchiveFileNameForm no longer goes through hooks/useForm's InputForm helper,
which still renders Semantic's Form.Input internally (hooks/useForm.js is
off-limits for this migration) -- it now builds its field directly against
form.getInputProps, mirroring the pattern Download.js already established.

Drops ThemeContext's obsolete i/s/t/inverted spreads throughout, and the
mimetype-color accent on Card (the new Card has no color prop; Videos.js's
already-migrated VideoCard dropped the same accent for the same reason).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 18:04:02 -06:00
Roland
f8bef9fc23
ui: keep the unmigrated pages' notifications working
Greptile P1, and a real one: App.js migrated and took `<SemanticToastContainer/>`
with it, but seven pages -- Archive, Channels, Docs, FilePreview, Playlists,
Zim, ControllerPage -- still raised their toasts through the Semantic library.
With no container mounted those notifications went nowhere.  No error, no
warning; a user deletes a document and is simply never told whether it worked.

Fixed by pointing those seven at the library's `toast`, which takes the same
options, rather than by mounting a second toast system.  Nothing calls the
Semantic toast library now, so a test asserts that at zero: a file that goes
back to it would raise notifications that silently never appear, and that is
not something a reviewer would spot.

Also reverted Archive.js to its last good state.  An interrupted agent had
migrated part of it -- Panel, Tabs, and Mantine's Grid.Col -- while the rest of
the file still used Semantic's Segment, Tab and Grid.Row, so a single `Grid`
identifier had to be two different components at once.  A half-migrated
1400-line file is worse than an unmigrated one; it goes back on the list and
gets done in one piece.

That break was masked: react-scripts caches ESLint results per file, so the
earlier build passed while Archive.js sat on a stale pass.  Editing its import
invalidated the entry and the errors appeared.  Verified with the cache cleared
this time, which is the only way that check means anything after a wide change.
2026-08-08 18:04:01 -06:00
Roland
e8dd804010
ui: thirty-five files off Semantic, leaving fifteen
Phase 2, part 4.  Baseline 50 -> 15.

Migrated: all eight calculators, the rest of inventory and collections, the
Map/Zim/Docs cluster, Videos, Files, Search, Archive, DashboardPage,
DatesSelector, HelpModal, Events, Tags, Flasher's neighbours, admin Settings
and Status, Download, and App.

Also the three shared modules everything imports: api.js and customHooks.js
swap the Semantic toast library for the library's shim (same call shape), and
useForm.js renders the library's Message.  useForm accepts both the old
`{content, header, positive|negative}` shape and the new `{kind, title}`,
because the pages building those objects migrate separately.

Two library gaps closed, both found by call sites needing them:

  ActionInput is a forwardRef now.  Its ref reaches the input rather than the
  wrapper, because callers use it to focus or select the field -- a ref to the
  surrounding div would do neither, which is why one call site had resorted to
  hand-copying the component's markup.

  Header takes a `color` token name.  Three separate call sites had reached for
  an inline style to colour a heading that carries meaning ("3 items to add"
  against "2 to remove"), which is how a hex ends up in markup.

Two files needed repair after being interrupted mid-edit: MapViewer had a
Segment left in its error branch, and VideoPlayer had lost the useContext
import its own code still used.  Neither jest nor tsc noticed; only
`npm run build` did, which is the whole reason that check exists.

Fifteen files remain: Archive, BulkTagModal, Channels, Docs, FileBrowser,
FilePreview, Flasher, Nav, Playlists, Theme.tsx, Upload, VideoPlayer, Zim,
ControllerPage, admin/Downloads.  Theme.tsx is deliberately last -- the
unmigrated files still import its Semantic-backed wrappers, so it can only go
once they have.
2026-08-08 18:04:01 -06:00
Roland
0850b6a7fe
app: handle form fetch failures without unhandled rejection
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
Catch useForm fetcher errors and surface them so DomainEditPage can show a not-found message instead of hanging on the loader (and so the webpack overlay no longer flakes Cypress).
2026-07-16 19:40:28 -06:00
Roland
73fafcd645
Search: fast ABC mode by default, opt-in deep search
The file_group.textsearch tsvector indexes all four text columns;
d_text (captions, document text, zip contents) is ~90% of the data,
making ts_rank searches take seconds on large libraries.  Add a
textsearch_abc generated column (title/author/description only, ~8x
smaller) and default all searches to it, with deep=true opting into the
full column.

Database:
- Migration adds the generated column and a GIN index (rewrites the
  table; a few minutes on large libraries).

Backend:
- FilesSearchRequest, VideoSearchRequest, ArchiveSearchRequest, and
  DocSearchRequest accept `deep`; each search function picks the column.
- /api/search_file_estimates returns both counts in one index-bound
  query (COUNT FILTER on the abc subset): `file_groups` (fast) and
  `file_groups_deep`.

UI (one coherent story across the count surfaces):
- The typing dropdown always suggests the deep count (what is findable).
- The tab badge shows the active mode's count so it matches the result
  list.
- The gap is reconciled by a hint offering deep search: count-aware on
  the global search ("No results in fast search. N results available in
  deep search." / "N more results available..."), generic on the module
  pages (no deep estimate there).  The hint resets pagination.
- "Search Depth" toggle in the search filter modals, persisted as
  ?deep=true.

Tests that relied on d_text matches now pass deep=True, with fast-mode
assertions alongside.  Also skip flaky test_download_channel on
CircleCI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 10:32:16 -06:00
lrnselfreliance
264cc8cd58
Consolidate search options into a single Filter modal (#497)
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
* Consolidate search options into a single Filter modal

Reworks the file-search UI on the Dashboard, Videos, Archives, and Docs
pages. The crowded row of controls (sort, tags, limit, file-type filter,
date selector) is replaced with a single "Filter" button that opens one
comprehensive modal.

- Add SearchFilterModal + SearchFilterButton (violet + count badge when
  active). Sections (Sort, File Type, Tags, Results Per Page, Published
  Date) are shown per-page via props; layout uses a Semantic Grid.
- Selections are drafted locally and applied in a single updateQuery only
  on close/Done, so the search isn't re-run on every field change.
- Results Per Page and Sort use buttons (fewer clicks, no clipped dropdown).
- Extract shared SearchControlBar (view toggle + input + Filter) for the
  three results pages; move the global-search view toggle onto the tab row.
- Rework the Dashboard search line into one adaptive flex row; clear button
  only shows when there's an active search.
- Remove now-dead FilesView limit/tag returns and SearchLimitDropdown.

* Address review: reset limit on Clear All; use replace over replaceAll

- handleClearAll now resets Results Per Page to the default page size
  (DEFAULT_SEARCH_LIMIT = 24) so "Clear All" actually clears every setting.
- Replace start-anchored replaceAll(/^-/g) with replace(/^-/) for the sort key.
- Add a test covering the Clear All limit reset.

* Remove now-unused Media import from Files.js

The FilesSearchView refactor dropped the last <Media> usage in this file,
leaving an unused import (a build warning). Remove it.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 19:49:25 -06:00
lrnselfreliance
9a5d1f1630
Add ordered collections (playlists): polymorphic items + item API (#475)
* Add Playlists: ordered, mixed-type collections

A playlist (Collection kind='playlist') is a manually-curated, ordered list that
mixes item types. Squash of the playlists feature branch (24 commits).

Data model & API:
- CollectionItem is polymorphic via item_kind: 'file' (FileGroup), 'zim'
  (zim_id + zim_entry), or 'url' (link to anything the WROLPi browser opens,
  e.g. /map?lat=&lon=&z=). CHECK constraint enforces exactly one kind; per-kind
  unique constraints; migration a7b1c2d3e4f5.
- Endpoints: create playlist; add/remove/reorder items (idempotent adds: 201 new,
  200 existing); GET returns the ordered items. All mutations wrol_mode_check'd.

Config (source of truth):
- playlists.yaml (PlaylistsConfig) stores each playlist's metadata AND ordered
  items (file by relative path, zim by path+entry, url inline); round-trips a
  DB wipe. Supports backup import (DB_CONFIG_FILE_NAMES).
- The redundant generic CollectionsConfig (collections.yaml) is deleted:
  channels/domains own their configs, authors/subjects are re-derived.

On-disk sync (sync_playlists_directory switch):
- Each playlist materializes into the configurable Playlists Directory
  (playlists_destination, Settings UI input, excluded from indexing) with
  position-prefixed entries: NNNN_<file> hard links (copy fallback across
  filesystems) and NNNN_<title>.html stubs for zim/url items.
- Tagged playlists are namespaced like channels: <playlists>/<tag>/<name>;
  a tag change moves the directory and prunes the empty old tag dir.
- Deletion safety: managed files are only removed when they are our marked
  stubs or still have another hard link (st_nlink > 1) -- never a last copy.
  Stray loose files at the playlists root are removed; stub URLs are
  scheme-allowlisted (no javascript:/data:).

Frontend:
- /playlists list page (CollectionTable, name filter, New Playlist modal, Tag
  column) between Files and Zim in the nav.
- View-only detail page (Name link) and edit page (Edit button): themed table
  with model icons/colors + poster thumbnails, items link to their model page
  (video/archive/doc) or preview, up/down reorder, remove, add-URL form,
  single-tag selector; all editing disabled in WROL Mode.
- Reusable green "Add to Playlist" button on Video/Archive/Doc detail pages,
  the FilePreview toolbar, Zim search results, and Map pins (added as a /map
  URL item with a map-marker icon).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Playlist edit page parity: rename, custom directory, tag modal move suggestion

Match the ChannelEditPage/DomainEditPage UX on the playlist edit page:
Back/View buttons, an edit form (name, description, directory), and the
shared CollectionTagModal with the violet "Tag" button.  The tag modal now
suggests moving the playlist to its managed tag location
(playlists/<tag>/<name>); accepting the suggestion stores directory=None so
the sync keeps auto-managing the location.  Tag buttons on the Channel and
Domain edit pages are now violet to match.

Backend: update_collection learns rename (with per-kind validation and
duplicate checks) and playlist custom directories (cleanup of the abandoned
directory, config round-trip, commit-before-switch to avoid the background
sync racing pre-commit state).  tag_collection's duplicated untag/tag
branches are merged, and the per-kind config-save dispatch is consolidated
into _activate_config_save().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Deduplicate Collection error handling and Tag creation

Delete the duplicate UnknownCollection in wrolpi/collections/errors.py and
re-export the canonical one from wrolpi/errors.py, which carries
status_code=NOT_FOUND (the local copy inherited 500).  Replace the repeated
query/raise pattern in reorganize.py with Collection.find_by_id.

Add Tag.get_or_create_tag and use it for collection tagging; flush the new
Tag before returning so callers can read .id immediately -- previously
tagging a collection with a brand-new tag name set tag_id=None.  Regression
test included.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 19:46:34 -06:00
Roland
ba63170e55
Consolidating deletion endpoints.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-05-03 19:14:30 -06:00
Roland
09a967535b Models like Archive/Video/Doc can be deleted, even if they are tagged. User is shown a warning and must confirm deletion of tagged files. 2026-05-03 19:14:15 -06:00
Roland
1294b14c10
Adding button to bulk tag file groups for collection items. Always displaying selection column on collection page tables.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-04-09 16:18:26 -06:00
Roland
a8b756b01a
Deleting archives by file group id, rather than archive id. 2026-04-09 15:29:28 -06:00
Roland
bf5b787fb3
Unifying Files tab appearance for model pages.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-03-28 14:19:09 -06:00
Roland
66a891c759
Refactoring videos and archives UI paths for consistent scheme. 2026-03-20 15:38:32 -06:00
Roland
aefc7b1def
Add Docs module for searching and organizing PDFs and EPUBs
Create a new Docs module that consolidates PDF and eBook handling into a
unified system with subjects and authors support. This replaces the
separate PDF/eBook modelers with a single Doc model backed by an
alembic migration (rename ebook to doc).

Key changes:
- New modules/docs package with models, API, extractors, and lib
- Subject and author metadata extraction with case-insensitive
  normalization for cleaner PDF metadata
- Frontend Docs pages with AuthorsPage, SubjectsPage using
  CollectionTable, and doc card deletion
- Global search integration for subjects and authors
- Generic collection item count support
- Frontend file card metadata adjusted to match collection sort
- Bump yt-dlp version
2026-03-20 14:17:06 -06:00
Roland
ad0d93714a
Adding videos page hotkeys to help play and navigate videos. Adding f hotkey for all local search fields (channel/domain/etc). Focusing when opening cookie modals. 2026-02-23 17:29:09 -07:00
Roland
8fc98741d0
Adding noreferrer to all external links. 2026-02-15 17:26:36 -07:00
Roland
2c8f765f2b
Adding statistics page for Archives. 2026-02-02 15:15:28 -07:00
Roland
9005c0deff
Adding toggle to disable the tags directory feature. 2026-01-31 18:08:42 -07:00
Roland
74cb5171da
FileGroups in Collections can be reorganized in the Collection's directory.
* Batch reorganization is supported for both Domain and Channel Collections.
* Batch reorganization happens when any Collection does not match the respective file format. For example, a Channel Collection file format that does not match the `yt_dlp_options.file_name_format` will show `needs_reorganization`.
* Individual Collections can always be reorganized. The reorganization is a little more aggressive. Useful for when some files may have been missed in the batch reorganization.
2026-01-31 18:08:41 -07:00
Roland
5c8f164c1a
Adding Upload button to the FileBrowser. Adding ErrorBoundaries to prevent entire app from crashing. 2025-12-25 09:14:00 -07:00
Roland
5d50562768
Reworking Cards in the App so that they use the new FileGroup.files and .data. 2025-12-24 12:30:04 -07:00
Roland
4ba5226c68
Adding ext to archive file format to match video file format.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2025-12-23 19:11:07 -07:00
Roland
1ebcae0965
Allowing user to configure collection download destination directories, as well as directories in the file structure. This will allow sorting archives by destination year, for example. 2025-12-23 16:46:15 -07:00
Roland
8a01ddc040
Adding links between domain page and archives page. 2025-12-17 17:27:17 -07:00
Roland
a92104728d Converting Theme.js to typescript so we can use shorthands like Card.Icon.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2025-12-07 12:41:10 -07:00
Roland
8ea3ea8a67
Updating code to use new version of react-router. 2025-12-07 11:54:20 -07:00
Roland
8a7f0737f0 Reformatting code. 2025-12-06 15:11:37 -07:00
Roland
47044cdd1f Ensuring domain config saves when tagging/untagging a domain. 2025-12-06 15:11:37 -07:00
Roland
b339c21515 Moving form fields. 2025-12-06 15:11:37 -07:00
Roland
63a77b87e6 Removing collection metadata. 2025-12-06 15:11:37 -07:00
Roland
a1e0fb4e92 Creating a new Collection model for use in organizing FileGroups.
Replacing Channels and Domains with Collection.

Moving downloading relationships from Channel to Collection.

Created domains.yaml config to manage tagged domain collections.

Adding turn-key upgrade solution in the UI.
2025-12-06 15:11:37 -07:00
Roland
1ed0d63bac Creating Generate Screenshot button for Archives which are missing a screenshot. 2025-12-06 15:11:37 -07:00
Roland
531ec3d65e
Creating new /api/archive/upload which accepts a file from the SingleFile browser extension and converts it to an Archive. 2025-01-30 18:41:32 -07:00
Roland
ee36b90e8e
Renaming HelpPopup to InfoPopup to avoid confusion with future Help features. 2025-01-24 14:28:04 -07:00
Roland
5621d47309
Video downloads have many more options. Downloads are migrated.
Storing video settings in session storage to help user start downloads consistently.

Applying channel tag name to new channels when download videos or channels.

Initializing map tiles when importing and resetting map.
2024-11-28 09:48:57 -07:00
Roland
feb4e3db52
Displaying human readable 'ago' when hovering over FileCard datetime. 2024-10-16 18:56:30 -06:00
Roland
0f232fd533
Moving FileGroups instead of refreshing files after they have been moved. 2024-10-16 18:56:25 -06:00
Roland
53d2402d32
Only files that have been tagged will be returned when new 'Any' tag button is used to filter file search results. 2024-10-16 18:55:56 -06:00
Roland
2ee39f0ace
Restoring Read button on ArchivePage. 2024-05-25 09:40:13 -06:00
Roland
fd619853b6
Videos/Archives search input is clearable. 2024-05-08 17:44:44 -06:00
Roland
65f94b4807
Reformatting app files. 2024-04-14 14:05:22 -06:00
Roland
5be4726493
Displaying error when re-download of Archive request fails. 2024-04-14 14:02:02 -06:00
Roland
fe8965b5d4
Fixing bug where videos and archive search input would not allow input. 2024-02-19 15:08:04 -07:00
Roland
bbfe76890b
Converting Search Suggetions to a Context. Search estimates is removed and suggestions is used instead. 2023-11-22 12:19:49 -07:00
Roland
e59ec3e9ac
Search Suggestions are debounced and only the latest response is used to set the results. 2023-11-21 21:15:24 -07:00
Roland
eefc4d4d79
Removing <SearchDomain/> from ArchivesPage. 2023-11-21 13:13:29 -07:00
Roland
7d9becdde6
Downloading requires a list of URLs. 2023-11-20 09:43:21 -07:00
Roland
2870612436
Adding data column to file list view, this displays data of the current order_by. 2023-11-02 16:56:05 -06:00