Commit graph

93 commits

Author SHA1 Message Date
Roland
3204fb51ea
ui: centre the pagination strip
It sat hard against the left edge of its container despite three call
sites wrapping it in `<center>` or `text-align: center`.

Mantine builds a Pagination as a BLOCK root -- full width, so those
wrappers had nothing to centre -- holding a flex Group that carries the
controls and lays them out at `flex-start`.  So `justify-content` on the
root does nothing at all; it has to reach the Group.

Setting `--group-justify` on the root does not work either, which cost a
detour: Group defaults that prop and writes the variable INLINE on its own
element, where an inherited value cannot reach it.  The property itself is
fair game, since Mantine only ever reads the variable from a hashed class,
so `.wrolpi-pagination .mantine-Group-root { justify-content: center }` at
(0,2,0) is what lands.

The three wrappers are now plain divs keeping their margin.  `<center>` is
a deprecated element that was doing nothing here.

Two tests: the gap either side is equal -- measured, not read off
`justify-content` -- with an assertion that the container really is wider
than the strip, since otherwise the equality holds trivially at 0 and 0.
And an inverse: at 240px nothing hangs off the LEFT edge.  That one is the
risk centring introduces -- overflow in a flex-start row spills right,
where it can be scrolled to, but centred it spills both ways and the first
page becomes unreachable.  It passed before this change and must keep
passing after.

Verified red first (0px left against 230px right), and the wrong-element
fix -- centring the root rather than the Group -- reddens it again.

1080 jest, 160 in ui-layout.cy.js, clean build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 18:05:47 -06:00
Roland
7a6b09aee6
ui: route genuine success and failure through the roles
Sixteen call sites, chosen by reading each one rather than by grepping for a
hue.  Four sub-agents judged them independently against one rule: convert only
what genuinely means success or failure; identification, category and
decoration stay hue-named.

Converted:
  SMART drive health (all four branches -- PASS/WARN/FAIL/unknown)
  config validity: valid / invalid / empty
  "Failed to load preview"
  the two "Download error" triggers
  expired inventory items, desktop and mobile
  the refresh phases: complete / active / not reached
  a refresh Progress bar in its error state
  cookie status: Unlocked / Locked
  a chip-mismatch label on the flasher
  CPU load thresholds (LoadStatistic)
  the outcome of an API call in useAPIButton
  the gallery's "Danger zone" heading and its load readings

Left alone, deliberately:
  the file browser toolbar -- every action has its own colour to be told apart,
    and delete already uses role='danger' (Roland's ruling)
  diff add/remove in the config and inventory import previews.  Adding is not a
    success and removing is not a failure; in Overwrite mode the removals are
    exactly what the user asked for, and painting them danger would report their
    intended action as broken
  seasons in DatesSelector, the uploader star and favourite heart, TagIcon,
  downloader category buttons, "Recommended to Keep"

Three sites were wrong in a way worth naming.  Both "Download error" triggers
guard on the same condition and render the same modal, but one was `orange` and
the other `red` -- and `orange` is the single worst choice, byte-identical to
`--text` in night, so that error button was indistinguishable from an ordinary
control.  Config validity drew its valid state in `--violet`, which no user
parses as "valid".  And `LoadStatistic` -- the thing users actually watch on the
Status page -- rendered its warning threshold in that same invisible orange.

Also fixed a stale comment in Videos.js that claimed no token existed for these.

Two notes on the tests:

The first LoadStatistic test I wrote rendered the component and asserted the
inline colour.  jsdom REJECTS `color: var(--danger)` as an invalid declaration
and drops it, so every colour reads back as the empty string -- two of the four
assertions passed for that reason alone and would have kept passing with the
hues restored.  It is a source guard now, with the painted result asserted in
the browser instead.

That browser test then found something real: an uncoloured reading inherits
`--text`, which in night is 4.98:1 against `--warning`'s 4.91:1, so a machine
under load is marginally QUIETER than one that is fine.  Not fixed here.  The
only way up on a dark panel is lightness, and night's danger would desaturate
from #ff3e3e to about #ff5d5d -- trading the theme's "red only" character for
loudness.  That is a palette decision, so the test documents it and asserts what
does hold rather than quietly making it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 18:05:46 -06:00
Roland
6b1e05be86
ui: bring back the card mimetype accent
Two agents independently dropped it, each because our Card had no `color` prop
and each citing the other's precedent -- which is how a deliberate feature
quietly becomes a convention.  Roland wanted it back.

Card takes a token colour name and draws it as a 3px edge along the top rather
than tinting the surface: surfaces stay flat per the design rules, and an edge
survives night mode without becoming a bright patch.  Because it is a token, the
accent remaps per theme -- four hues in light and dark, a single one in night and
amber, which is the point of those themes.

Restored on every card that shows a file: the video, image and generic file cards
in Files.js, and the archive card.  All of them take it from
`mimetypeColor(mimetype, path)`, the same helper that colours the file icons, so
a card and its icon agree.
2026-08-08 18:04:03 -06:00
Roland
e8dd804010
ui: thirty-five files off Semantic, leaving fifteen
Phase 2, part 4.  Baseline 50 -> 15.

Migrated: all eight calculators, the rest of inventory and collections, the
Map/Zim/Docs cluster, Videos, Files, Search, Archive, DashboardPage,
DatesSelector, HelpModal, Events, Tags, Flasher's neighbours, admin Settings
and Status, Download, and App.

Also the three shared modules everything imports: api.js and customHooks.js
swap the Semantic toast library for the library's shim (same call shape), and
useForm.js renders the library's Message.  useForm accepts both the old
`{content, header, positive|negative}` shape and the new `{kind, title}`,
because the pages building those objects migrate separately.

Two library gaps closed, both found by call sites needing them:

  ActionInput is a forwardRef now.  Its ref reaches the input rather than the
  wrapper, because callers use it to focus or select the field -- a ref to the
  surrounding div would do neither, which is why one call site had resorted to
  hand-copying the component's markup.

  Header takes a `color` token name.  Three separate call sites had reached for
  an inline style to colour a heading that carries meaning ("3 items to add"
  against "2 to remove"), which is how a hex ends up in markup.

Two files needed repair after being interrupted mid-edit: MapViewer had a
Segment left in its error branch, and VideoPlayer had lost the useContext
import its own code still used.  Neither jest nor tsc noticed; only
`npm run build` did, which is the whole reason that check exists.

Fifteen files remain: Archive, BulkTagModal, Channels, Docs, FileBrowser,
FilePreview, Flasher, Nav, Playlists, Theme.tsx, Upload, VideoPlayer, Zim,
ControllerPage, admin/Downloads.  Theme.tsx is deliberately last -- the
unmigrated files still import its Semantic-backed wrappers, so it can only go
once they have.
2026-08-08 18:04:01 -06:00
Roland
2162396212
Module pages: always show the deep-search reminder during fast searches
The DeepSearchHint only appeared on zero results, so a fast search with
matches gave no indication that captions/contents were not searched.
Module searches have no deep estimate, so show a generic reminder
whenever a fast search is active: "More results may be available in
deep search." with results, "Not finding what you need?" without.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 10:50:57 -06:00
Roland
73fafcd645
Search: fast ABC mode by default, opt-in deep search
The file_group.textsearch tsvector indexes all four text columns;
d_text (captions, document text, zip contents) is ~90% of the data,
making ts_rank searches take seconds on large libraries.  Add a
textsearch_abc generated column (title/author/description only, ~8x
smaller) and default all searches to it, with deep=true opting into the
full column.

Database:
- Migration adds the generated column and a GIN index (rewrites the
  table; a few minutes on large libraries).

Backend:
- FilesSearchRequest, VideoSearchRequest, ArchiveSearchRequest, and
  DocSearchRequest accept `deep`; each search function picks the column.
- /api/search_file_estimates returns both counts in one index-bound
  query (COUNT FILTER on the abc subset): `file_groups` (fast) and
  `file_groups_deep`.

UI (one coherent story across the count surfaces):
- The typing dropdown always suggests the deep count (what is findable).
- The tab badge shows the active mode's count so it matches the result
  list.
- The gap is reconciled by a hint offering deep search: count-aware on
  the global search ("No results in fast search. N results available in
  deep search." / "N more results available..."), generic on the module
  pages (no deep estimate there).  The hint resets pagination.
- "Search Depth" toggle in the search filter modals, persisted as
  ?deep=true.

Tests that relied on d_text matches now pass deep=True, with fast-mode
assertions alongside.  Also skip flaky test_download_channel on
CircleCI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 10:32:16 -06:00
Roland
2acdc2e16d
Expanding FileBrowser button widths when page is wide enough.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-07-01 19:52:15 -06:00
lrnselfreliance
cafc6514cb
Restore the censored-video search filter (#499)
* Restore the censored-video search filter

Censored videos (no longer publicly available — detected when a channel
refresh drops them from the channel list, or when comment downloads fail
with "video unavailable/private/removed") are already flagged on
FileGroup.censored, but there was no way to find them since the search
rework dropped the old filter.

Add a "Censored" toggle to the reworked search filter modal, shown only on
the Videos page (showCensored).  When enabled the video search returns only
videos with FileGroup.censored = TRUE, tracked via a ?censored=true query
param (useSearchCensored).
* Move Availability filter below the Tags/Results row

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 09:48:36 -06:00
lrnselfreliance
264cc8cd58
Consolidate search options into a single Filter modal (#497)
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
* Consolidate search options into a single Filter modal

Reworks the file-search UI on the Dashboard, Videos, Archives, and Docs
pages. The crowded row of controls (sort, tags, limit, file-type filter,
date selector) is replaced with a single "Filter" button that opens one
comprehensive modal.

- Add SearchFilterModal + SearchFilterButton (violet + count badge when
  active). Sections (Sort, File Type, Tags, Results Per Page, Published
  Date) are shown per-page via props; layout uses a Semantic Grid.
- Selections are drafted locally and applied in a single updateQuery only
  on close/Done, so the search isn't re-run on every field change.
- Results Per Page and Sort use buttons (fewer clicks, no clipped dropdown).
- Extract shared SearchControlBar (view toggle + input + Filter) for the
  three results pages; move the global-search view toggle onto the tab row.
- Rework the Dashboard search line into one adaptive flex row; clear button
  only shows when there's an active search.
- Remove now-dead FilesView limit/tag returns and SearchLimitDropdown.

* Address review: reset limit on Clear All; use replace over replaceAll

- handleClearAll now resets Results Per Page to the default page size
  (DEFAULT_SEARCH_LIMIT = 24) so "Clear All" actually clears every setting.
- Replace start-anchored replaceAll(/^-/g) with replace(/^-/) for the sort key.
- Add a test covering the Clear All limit reset.

* Remove now-unused Media import from Files.js

The FilesSearchView refactor dropped the last <Media> usage in this file,
leaving an unused import (a build warning). Remove it.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 19:49:25 -06:00
Roland
66cecce5e6
Adding Doc filter button so users can search just PDFs/EPUB/etc. 2026-04-14 15:35:35 -06:00
Roland
8a75cf9057 Linking to specific page or section of an EPUB/PDF when user searches. 2026-04-13 16:47:02 -06:00
Roland
1294b14c10
Adding button to bulk tag file groups for collection items. Always displaying selection column on collection page tables.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-04-09 16:18:26 -06:00
Roland
1c79c1c495
Refactoring docs to match new url scheme like videos and archives. 2026-03-20 15:53:21 -06:00
Roland
aefc7b1def
Add Docs module for searching and organizing PDFs and EPUBs
Create a new Docs module that consolidates PDF and eBook handling into a
unified system with subjects and authors support. This replaces the
separate PDF/eBook modelers with a single Doc model backed by an
alembic migration (rename ebook to doc).

Key changes:
- New modules/docs package with models, API, extractors, and lib
- Subject and author metadata extraction with case-insensitive
  normalization for cleaner PDF metadata
- Frontend Docs pages with AuthorsPage, SubjectsPage using
  CollectionTable, and doc card deletion
- Global search integration for subjects and authors
- Generic collection item count support
- Frontend file card metadata adjusted to match collection sort
- Bump yt-dlp version
2026-03-20 14:17:06 -06:00
Roland
f59a4198d9
Renaming all co-occurring to overlapping, easier to spell. Filtering by only overlapping tags when searching by a tag, this is because any non-overlapping will not produce results anyways. 2026-03-07 15:55:04 -07:00
Roland
c216e8ed26
Reworking file refresh button to only disable when global refresh is running. 2026-02-14 17:17:08 -07:00
Roland
6389541b51
Always displaying icon on steps, even when disabled. 2026-02-08 10:57:12 -07:00
Roland
8a03ceec52
Adding refresh steps to UI for better user feedback. Modifying file worker progress tracking for better accuracy.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-02-07 19:47:44 -07:00
Roland
74cb5171da
FileGroups in Collections can be reorganized in the Collection's directory.
* Batch reorganization is supported for both Domain and Channel Collections.
* Batch reorganization happens when any Collection does not match the respective file format. For example, a Channel Collection file format that does not match the `yt_dlp_options.file_name_format` will show `needs_reorganization`.
* Individual Collections can always be reorganized. The reorganization is a little more aggressive. Useful for when some files may have been missed in the batch reorganization.
2026-01-31 18:08:41 -07:00
Roland
173284699d
FileWorker: Async queue-based file synchronization system
Introduces FileWorker for background file operations with dual-queue
architecture (public/private). Handles refresh, move, and rename tasks
with progress tracking, job completion monitoring, and batch processing.
Compares DB FileGroups to filesystem state using FileGroupDiff to detect
new/modified/deleted files. Designed for memory efficiency on Raspberry Pi.
2026-01-23 15:29:15 -07:00
Roland
b6e82cde26
Removing label from Refresh button to make room for Tags button. 2025-12-24 11:48:53 -07:00
Roland
3f4d5798e2
Using image next to a PDF with the same stem as the PDF's poster.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2025-12-22 17:01:00 -07:00
Roland
a92104728d Converting Theme.js to typescript so we can use shorthands like Card.Icon.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2025-12-07 12:41:10 -07:00
Roland
8ea3ea8a67
Updating code to use new version of react-router. 2025-12-07 11:54:20 -07:00
Roland
5621d47309
Video downloads have many more options. Downloads are migrated.
Storing video settings in session storage to help user start downloads consistently.

Applying channel tag name to new channels when download videos or channels.

Initializing map tiles when importing and resetting map.
2024-11-28 09:48:57 -07:00
Roland
acf3dee179
Configs are displayed in new ConfigsPage under Admin.
- Configs can be imported and saved.
- Configs are validated before imported.
- API will refuse to save a config if it was not imported successfully.
2024-10-16 18:56:45 -06:00
Roland
feb4e3db52
Displaying human readable 'ago' when hovering over FileCard datetime. 2024-10-16 18:56:30 -06:00
Roland
0f232fd533
Moving FileGroups instead of refreshing files after they have been moved. 2024-10-16 18:56:25 -06:00
Roland
53d2402d32
Only files that have been tagged will be returned when new 'Any' tag button is used to filter file search results. 2024-10-16 18:55:56 -06:00
Roland
ae5d47fc21
Shortening refresh button for mobile. 2024-03-11 17:47:27 -06:00
Roland
00fe799b8f
User can select from/to years when searching files. Indexing doc and docx files. 2024-03-07 12:12:48 -07:00
Roland
2870612436
Adding data column to file list view, this displays data of the current order_by. 2023-11-02 16:56:05 -06:00
Roland
f9f71fa908
Searching all zips when searching for Software. 2023-11-02 15:31:13 -06:00
Roland
472846038b
Auto-submitting global search input, but debouncing with 1 second. 2023-11-02 15:24:07 -06:00
Roland
badfe691de
Using css ellipsis for download rows. 2023-11-02 10:07:49 -06:00
Roland
38cddeff98
Adding software file filter. 2023-10-29 21:26:35 -06:00
Roland
15687e534a
Moving data from models to FileGroup.
* Ebook.creator -> FileGroup.author
* Video.viewed -> FileGroup.viewed
* Video.duration -> FileGroup.length
* Video.url -> FileGroup.url
* Archive.url -> FileGroup.url
2023-10-29 17:31:54 -06:00
Roland
ae9d188391
Adding FileGroup.published_date/author/etc. Creating script which converts a Singlefile to an Archive FileGroup files.
Parsing SingleFile datetime using dates.strpdate.

Adding timezone when no timezone info is available.

Installing python-dateutil to assist in handling diverse datetime strings.

Reworking refresh progress to use idempotency to display more accurate progress.  Only logging that files have been invalidated if they used to be indexed.

Clearing archives to force re-index.

Ignoring 'Unknown' author from PDF metadata. Using file_title when pdf metadata title is empty.
2023-10-23 18:09:08 -06:00
Roland
619a88c429
Reworking default/empty/error states for all pages. Null is pending, undefined is error. 2023-09-07 20:13:36 -06:00
Roland
36083200e6
Creating Video.ffprobe_json column which stores ffprobe information about the video file. Creating find_corrupt_videos.py script which automatically re-downloads videos which do not have both video and audio streams. 2023-08-03 21:40:25 -06:00
Roland
59c4ab2502
Replacing icon property when icon has been requested.
Creating <APIButton/> which handles success/failure/pending during API calls.
2023-07-21 16:52:15 -06:00
Roland
03a2df217d
Zim/Kiwix (Wikipedia) feature
* A new zim module has been added, this module allows several things to interact with Zim files.
  * Allow searching of Zim files using libzim.
  * Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
  * Creation of a new zim docker container which allows viewing of the Zim file contents.

* This commit also adds move/rename/mkdir features in the File Browser.
  * File tags are preserved when moving/renaming.
2023-07-10 21:30:40 -06:00
Roland
387282b445
Adding my public gpg key.
Moving upload destination input out of drop Segment.

Disabling uploads when downloads are disabled.

Inverting labels for checkboxes in inverted modal.

Deleting file_groups which no longer contain the exclusive primary path.

Adding debug logging for all API errors.

Checking that upload output is not a directory.

Expanding nginx request size.

Adding docs to upload endpoint.

Using dot-circle icon for more image mimetypes.

Using Form.Field to make upload destination required.

Adding border to inverted Segments.

Many files can be uploaded in chunks.

Files can be uploaded, progress is displayed.

Showing white border around share QR code.

Displaying WROLPi colors for download icons.

Creating button to clear DirectorySearch.

Excluding channel/domain directories so other directories can be returned.

Ordering search results.

Ignoring when no channel is found only if we have a destination.

Limiting directory search results.

Displaying categories when searching destination directories.

Handling when live_chat.json file exists.

Allowing refresh of directory directly.

Logging which download was successful.

Removing timeout.

Creating index for directory names.

Sleeping to catch cancel.

Arbitrary files can be downloaded using the new FileDownloader.
2023-04-28 13:56:32 -06:00
Roland
8faac3f3a1
Handling when file may not have title for headline. 2023-04-12 18:25:38 -06:00
Roland
29e562c520
Headlines can be displayed for file search results. 2023-04-12 18:09:10 -06:00
Roland
f33dc31111
Preventing double-search when order is changed on page switch. 2023-04-12 16:09:09 -06:00
Roland
a3321cf2fa
Adding Counting step to file refreshing. 2023-04-10 16:28:10 -06:00
Roland
d84e9726fe
Displaying total groups in modeling progress. 2023-04-10 15:56:25 -06:00
Roland
1016e32846
Counting files before refresh to display more helpful progress. 2023-04-10 15:48:25 -06:00
Roland
0b0e6346a7
Fixing bug where file search would issue twice on load. 2023-04-09 14:34:33 -06:00