An unknown URL (e.g. /archive instead of /archives) had no matching route,
so react-router rendered the root errorElement in place of Root. That put
PageNotFound outside ThemeProvider, and its Mantine Box threw
"MantineProvider was not found", leaving a blank page.
- A catch-all route renders PageNotFound inside Root, with the navigation
bar, footer and theme.
- The root errorElement brings its own ThemeProvider, so it can render when
Root itself fails.
- App exports its routes so a test can mount them in a memory router.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Repos keeps local mirrors of git repositories, cloned over https and kept up to date on a schedule.
- A Repo is a Collection (kind='repo') plus a `repository` row. The new `git` downloader clones it, then updates it
(weekly by default, or Never: the clone is kept as it is and only updated on request).
- Repos are saved in `repos/<tag>/<name>` (the Repos Directory, ignored by default) or a directory the user chooses.
An existing clone can be imported where it is. repos.yaml is the source of truth.
- A GitHub-like page shows each repo's README, files, and history, with a ZIP download. READMEs are searchable
under the global search's Other tab.
- git is hardened against hostile repos and configs: https only, pinned addresses with no private addresses (for the
repo and its submodules), sanitized configs, no hooks or links, and only clones WROLPi made or imported are
updated.
- Tagging a Repo moves its clone as one directory. Repo changes wait for repos.yaml to be imported at startup.
- The file mover groups unindexed files with no primary file as one FileGroup, and never replaces a file at the
destination. A failed tag move restores the Collection's tag. A Collection's directory cannot leave the media
directory.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* bookmarks: user bookmarks in the navigation bar, stored in bookmarks.yaml
A "Bookmarks" tab between Calculators and Flasher opens on hover or click and
lists the user's bookmarks, with directories as flyout submenus (Menu.Sub).
The mobile hamburger shows directories as labelled groups instead. The tab
always shows, with an "Edit bookmarks" item, so the feature can be found
before any bookmark exists.
Bookmarks live in a new config-only bookmarks.yaml: a nested tree where a
node with `children` is a directory and a node with `url` is a bookmark.
Every node carries an integer id so the API can rename, move and delete
without relying on names. A URL may be a path on this WROLPi, a port on this
host (`:8096/` or `http://:8096/`, resolved against whatever hostname the
browser used, so it works over LAN IP, mDNS and the hotspot alike), or an
absolute URL. Each bookmark chooses whether to open in a new tab.
The editor at /more/bookmarks follows the playlist items table: numbered rows,
up/down/edit/delete, and inline "Add a bookmark" / "Add a directory" forms
with a Location picker. Deleting a directory asks first, since its contents
go with it. The menu and the editor are in the component gallery.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* bookmarks: mark More current when the editor is folded inside it; rem indents in the mobile menu
Found on the QA Pi at a width where Bookmarks overflows into More: the More
button carried no mark on /more/bookmarks, because the bookmarks link has no
`to` for isLinkActive to match. isLinkActive now knows the bookmarks link owns
its editor page, and the Bookmarks tab uses the same rule.
The mobile menu's directory labels sat a few pixels left of the labels around
them: the indent was in em, and a Menu.Label is set smaller than a Menu.Item.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* bookmarks: parse URLs instead of matching a prefix; validate the whole tree on every write
The URL contract was checked by prefix on both sides, and a prefix is not a
form: `//evil.com` starts like a path and `:8096@evil.com` starts like a port,
and a browser takes both off this host. A path is now a single leading slash
that resolves to this origin, a port form is an optional scheme, `:`, digits
and then only a path, query or fragment (no `@`, no backslash, no control
characters), and anything else must parse as http(s). The same rules live in
wrolpi/bookmarks.py and BookmarksContext.js, with the same bypass strings in
both test suites.
The config's dataclass validator only saw that `bookmarks` was a list, so a
hand-edited file or the generic config API could store a `javascript:` URL
that the bookmarks API would refuse, and the nav rendered it. The config now
validates the tree itself on every write and import: URL forms, exactly one of
url/children, unique integer ids, boolean new_tab, and no cycles (a YAML
anchor can build one). An invalid file is not imported and, since it was never
successfully imported, is never overwritten.
On the page, a stored URL that does not resolve is rendered as text, never as
an href. A refused add keeps what was typed instead of clearing the form, and
an edit whose move is refused still refreshes the nav (the rename was
committed) and keeps the modal open. An IPv6 host is bracketed before a port.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* help: document Bookmarks
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
ControllerPage was the final call site; Theme.tsx then had nothing left to wrap,
so it drops from 861 lines to 338 -- the provider that resolves the theme, stamps
`data-theme` and `data-media-filter`, persists both and hands Mantine our tokens.
Everything below that was Semantic wrappers.
The dark-mode compatibility props went with them. `i`, `s`, `t` and `inverted`
existed so unmigrated components could borrow Semantic's dark treatment, and
they carried hardcoded greys -- #1B1C1D, #dddddd, #eeeeee -- which is exactly
what the token themes replaced. Three call sites in Common.js were still
injecting the #eeeeee one into links.
Then `semantic-ui-offline/semantic.min.css` came out of App.js. That took more
with it than expected, and looking at the running app is the only reason it was
caught:
Every link lost its colour and gained the browser's underline and visited
purple. The dashboard's load averages and section headings were underlined
and purple. Links now take `--blue`, which remaps per theme, and underline
only on hover -- many of them wrap cards, posters and icons.
`box-sizing: border-box` disappeared, which silently changes the size of every
padded element in the app. Semantic's reset had been providing it.
The navigation bar overflowed a 1000px viewport with `nowrap` and cut "Admin"
off the right edge -- the one link a user needs to reach the settings that fix
a broken WROLPi. Semantic's menu wrapped; ours does now too.
The gzipped CSS bundle is 47.5 kB, and one call site was still leaning on
Semantic's grid classes -- now a flex column, since all three of its callers
pass library components.
The ratchet stays rather than being deleted. At zero it asserts four things: no
file imports Semantic UI, none calls its toast library, none leans on its
stylesheet, and no file reaches back through Theme.tsx. It can go when the
packages are uninstalled.
Theme.test.js is deleted, but not before porting the two behaviours it
protected: that Button forwards a ref to the real DOM element (a class-component
ref once crashed the Control page, because portals call `node.contains`), and
that the progress percentage stays readable on every bar colour.
Phase 2, part 4. Baseline 50 -> 15.
Migrated: all eight calculators, the rest of inventory and collections, the
Map/Zim/Docs cluster, Videos, Files, Search, Archive, DashboardPage,
DatesSelector, HelpModal, Events, Tags, Flasher's neighbours, admin Settings
and Status, Download, and App.
Also the three shared modules everything imports: api.js and customHooks.js
swap the Semantic toast library for the library's shim (same call shape), and
useForm.js renders the library's Message. useForm accepts both the old
`{content, header, positive|negative}` shape and the new `{kind, title}`,
because the pages building those objects migrate separately.
Two library gaps closed, both found by call sites needing them:
ActionInput is a forwardRef now. Its ref reaches the input rather than the
wrapper, because callers use it to focus or select the field -- a ref to the
surrounding div would do neither, which is why one call site had resorted to
hand-copying the component's markup.
Header takes a `color` token name. Three separate call sites had reached for
an inline style to colour a heading that carries meaning ("3 items to add"
against "2 to remove"), which is how a hex ends up in markup.
Two files needed repair after being interrupted mid-edit: MapViewer had a
Segment left in its error branch, and VideoPlayer had lost the useContext
import its own code still used. Neither jest nor tsc noticed; only
`npm run build` did, which is the whole reason that check exists.
Fifteen files remain: Archive, BulkTagModal, Channels, Docs, FileBrowser,
FilePreview, Flasher, Nav, Playlists, Theme.tsx, Upload, VideoPlayer, Zim,
ControllerPage, admin/Downloads. Theme.tsx is deliberately last -- the
unmigrated files still import its Semantic-backed wrappers, so it can only go
once they have.
The gallery at /theme-sample is now a normal route rather than a development
one, and Settings links to it, so a user can see what a theme actually looks
like before committing to it. Every new component belongs in the gallery: it is
how the design gets reviewed, and how a component that only looks wrong in one
of the four themes gets caught.
Settings gains a Theme section with the new ThemePicker, which explains what
each theme is for - notably that night mode filters videos, documents and maps
to red as well, not just the interface.
Theme names and picker options move to themes/names.ts so the component library
can read them without importing components/Theme.tsx, which still pulls in
Semantic UI. Theme.tsx re-exports them, so existing call sites are unchanged.
The navigation bar's compact picker now reads the same list, so the two cannot
drift apart.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds src/components/ui: buttons, messages, labels, progress, status, tables,
form controls, panels, cards, statistics, modals, tooltips, tabs, and icons.
Everything resolves its colors through theme tokens, so no component branches
on the current theme, and Semantic UI keeps rendering the app meanwhile.
Mantine is bridged rather than restyled: its own CSS variables are pointed at
our tokens. Four details were not obvious, and each caused a visible defect
in night mode before being fixed:
- Structural colors must go in the resolver's light/dark blocks. Mantine
emits its values under [data-mantine-color-scheme], which outranks the
:root block that `variables` becomes, so its grays were winning.
- The dark and gray palettes are mapped shade by shade to tokens. Mantine
reads them directly for table borders, zebra stripes and input chrome;
they can be neither flattened nor left alone.
- Mantine writes --button-* as inline styles, which no stylesheet can
outrank, so the dashed danger treatment sets real properties instead.
- Switch colors are written onto inner elements, so they need scoped CSS
rather than a variable on the root. Adds a --knob token for control
handles, because Mantine's default thumb follows theme.white, which is
near-black in the dark-based themes.
An audit of every painted pixel in the /theme-sample gallery now finds no
non-red channel in night mode.
Icons come from @tabler/icons-react with the 102 Semantic names the app uses
mapped to components; they stroke with currentColor, so they follow the theme
without any rule of their own. Toasts keep the react-semantic-toasts-2
signature so those call sites migrate by changing an import.
TypeScript 5 is required: Mantine's declarations use `const` type parameters,
which 4.9 cannot parse (Mantine 7 is the same, so downgrading is no escape).
That needs two npm overrides to keep clean installs working, both verified
against a Docker build: react-scripts declares peerOptional typescript ^3||^4,
and fork-ts-checker 6.5.2 assigns typescript.performance.mark, which TS 5
made getter-only - crashing the dev server's type checker on every rebuild.
semantic-baseline.test.js ratchets the removal: it fails if a file not already
listed imports Semantic UI. An ESLint rule would have printed 75 warnings per
rebuild, and CI unsets CI= for its build step, so it would guard nothing there.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every color now resolves through CSS custom properties on
html[data-theme=...], so a theme is a table of values rather than code.
Four themes ship: light, dark, night (red hues only, to preserve dark
adaptation off-grid) and amber (monochrome amber terminal).
Fonts are exposed as two roles, --font-body and --font-mono, which a theme
assigns a bundled face to; amber uses the mono face. Faces must be
self-hosted, never fetched from a CDN, so the app keeps working offline.
Night mode also filters media that cannot consume tokens - video, images,
PDFs, embedded pages, canvases - through an SVG color matrix that maps
luminance onto the red channel. Tag such an element with `.media`; a filter
on an app-level ancestor would create a containing block and break fixed
positioning.
ThemeProvider persists the choice and index.html stamps data-theme from
localStorage before the bundle loads, so the first paint already matches the
saved theme on a slow connection. The OS preference is followed only while
"system" is selected; night and amber are explicit choices. The nav toggle
becomes a five-item picker.
Semantic components still render themselves, so only the shell is
token-driven for now; the inverted props stay in place until each component
is migrated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Browser-based ESP32/ESP8266 flasher (esptool-js + Web Serial); nothing is
uploaded to WROLPi, the flash is written straight to the device over USB.
Firmware sources (themed, tabbed): a saved set, the WROLPi media directory,
or a local file. Local files can be dropped anywhere on the page, which jumps
to the "Add from computer" dropzone. The media picker searches .bin files by
path and can detect the connected chip to filter firmware to matching images.
- Indexed file_group.suffix + path filter for fast .bin search (alembic
migration backfills via split_path_stem_and_suffix)
- modules/flasher: chip detection from ESP image headers, /search endpoint,
and flasher.yaml saved configurations (/configs CRUD)
- Flash progress modal with ETA, connect timeout, boot-mode hint, auto-
scrolling logs
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the DB-backed inventory module with a config-only design and build out
food-storage tooling around it.
Inventory (config-only, no database)
- Each inventory is a YAML file at config/inventory/<slug>.yaml, loaded once
into memory and the single source of truth. New reusable MultiFileConfig
(wrolpi/common.py) manages a directory of per-entity files.
- Inventory types (food/fuel/tool) seed a customizable field schema
(text/number/quantity/date/select/location/calories). Unit math/aggregation
moved to the frontend (mathjs); Pint dropped from the backend.
- Minimal API: GET / (all inventories full), POST /, PUT /<slug> (whole-
inventory save with version-based optimistic concurrency; 404 for missing),
DELETE. Legacy inventories.yaml / old tables migrate to per-inventory files
(pound->lb), and an alembic migration drops the tables.
Frontend
- New app/src/components/inventory/ package: spreadsheet-style table with
Tab/Enter keyboard entry, sortable columns, expired-row highlighting,
customizable field editor, and pooled cross-inventory location autocomplete.
- A single useInventories store loads everything once and persists whole
inventories optimistically; list/items/summary/locations derive client-side.
- Summary groups items (default by Category), totals size x count, shows a
calories column, and embeds a ration estimate (how long the stock lasts).
Food catalog
- Shared, editable catalog (config/inventory/catalog.yaml) seeded from
catalog_defaults.yaml (~60 generic items, total calories per package) with
merge-keep-edits + tombstones. GET/PUT /api/inventory/catalog, a management
modal, and entry-row pre-fill of category/size/calories from a name match.
Calculators
- Generic Food Storage planner (like the Water calculator): household +
editable per-person/year amounts -> total to store over a chosen duration,
with women/children weighted by DGA calorie shares and an imperial/metric
toggle.
Also fixes ConfigFile.save() to accept nested file_names. Backend pytest and
frontend jest suites updated/added throughout; alembic check clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add Playlists: ordered, mixed-type collections
A playlist (Collection kind='playlist') is a manually-curated, ordered list that
mixes item types. Squash of the playlists feature branch (24 commits).
Data model & API:
- CollectionItem is polymorphic via item_kind: 'file' (FileGroup), 'zim'
(zim_id + zim_entry), or 'url' (link to anything the WROLPi browser opens,
e.g. /map?lat=&lon=&z=). CHECK constraint enforces exactly one kind; per-kind
unique constraints; migration a7b1c2d3e4f5.
- Endpoints: create playlist; add/remove/reorder items (idempotent adds: 201 new,
200 existing); GET returns the ordered items. All mutations wrol_mode_check'd.
Config (source of truth):
- playlists.yaml (PlaylistsConfig) stores each playlist's metadata AND ordered
items (file by relative path, zim by path+entry, url inline); round-trips a
DB wipe. Supports backup import (DB_CONFIG_FILE_NAMES).
- The redundant generic CollectionsConfig (collections.yaml) is deleted:
channels/domains own their configs, authors/subjects are re-derived.
On-disk sync (sync_playlists_directory switch):
- Each playlist materializes into the configurable Playlists Directory
(playlists_destination, Settings UI input, excluded from indexing) with
position-prefixed entries: NNNN_<file> hard links (copy fallback across
filesystems) and NNNN_<title>.html stubs for zim/url items.
- Tagged playlists are namespaced like channels: <playlists>/<tag>/<name>;
a tag change moves the directory and prunes the empty old tag dir.
- Deletion safety: managed files are only removed when they are our marked
stubs or still have another hard link (st_nlink > 1) -- never a last copy.
Stray loose files at the playlists root are removed; stub URLs are
scheme-allowlisted (no javascript:/data:).
Frontend:
- /playlists list page (CollectionTable, name filter, New Playlist modal, Tag
column) between Files and Zim in the nav.
- View-only detail page (Name link) and edit page (Edit button): themed table
with model icons/colors + poster thumbnails, items link to their model page
(video/archive/doc) or preview, up/down reorder, remove, add-URL form,
single-tag selector; all editing disabled in WROL Mode.
- Reusable green "Add to Playlist" button on Video/Archive/Doc detail pages,
the FilePreview toolbar, Zim search results, and Map pins (added as a /map
URL item with a map-marker icon).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Playlist edit page parity: rename, custom directory, tag modal move suggestion
Match the ChannelEditPage/DomainEditPage UX on the playlist edit page:
Back/View buttons, an edit form (name, description, directory), and the
shared CollectionTagModal with the violet "Tag" button. The tag modal now
suggests moving the playlist to its managed tag location
(playlists/<tag>/<name>); accepting the suggestion stores directory=None so
the sync keeps auto-managing the location. Tag buttons on the Channel and
Domain edit pages are now violet to match.
Backend: update_collection learns rename (with per-kind validation and
duplicate checks) and playlist custom directories (cleanup of the abandoned
directory, config round-trip, commit-before-switch to avoid the background
sync racing pre-commit state). tag_collection's duplicated untag/tag
branches are merged, and the per-kind config-save dispatch is consolidated
into _activate_config_save().
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Deduplicate Collection error handling and Tag creation
Delete the duplicate UnknownCollection in wrolpi/collections/errors.py and
re-export the canonical one from wrolpi/errors.py, which carries
status_code=NOT_FOUND (the local copy inherited 500). Replace the repeated
query/raise pattern in reorganize.py with Collection.find_by_id.
Add Tag.get_or_create_tag and use it for collection tagging; flush the new
Tag before returning so callers can read .id immediately -- previously
tagging a collection with a brand-new tag name set tag_id=None. Regression
test included.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Create a new Docs module that consolidates PDF and eBook handling into a
unified system with subjects and authors support. This replaces the
separate PDF/eBook modelers with a single Doc model backed by an
alembic migration (rename ebook to doc).
Key changes:
- New modules/docs package with models, API, extractors, and lib
- Subject and author metadata extraction with case-insensitive
normalization for cleaner PDF metadata
- Frontend Docs pages with AuthorsPage, SubjectsPage using
CollectionTable, and doc card deletion
- Global search integration for subjects and authors
- Generic collection item count support
- Frontend file card metadata adjusted to match collection sort
- Bump yt-dlp version
* Batch reorganization is supported for both Domain and Channel Collections.
* Batch reorganization happens when any Collection does not match the respective file format. For example, a Channel Collection file format that does not match the `yt_dlp_options.file_name_format` will show `needs_reorganization`.
* Individual Collections can always be reorganized. The reorganization is a little more aggressive. Useful for when some files may have been missed in the batch reorganization.
Storing video settings in session storage to help user start downloads consistently.
Applying channel tag name to new channels when download videos or channels.
Initializing map tiles when importing and resetting map.
* A new zim module has been added, this module allows several things to interact with Zim files.
* Allow searching of Zim files using libzim.
* Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
* Creation of a new zim docker container which allows viewing of the Zim file contents.
* This commit also adds move/rename/mkdir features in the File Browser.
* File tags are preserved when moving/renaming.
Tags can be created and deleted in UI.
Adding tag_file using id to avoid session error.
Using UTC in test.
Matching tag_names style in test.
Tag icon is displayed on file rows.
Archives, Videos and Files can be filtered by their tags.
Tags can be added and removed from UI.
Tags are displayed in VideoPlayer.
The tags of a file are returned in it's JSON.
Video favorites are converted to Favorite tag on channel config import.
Updating tests to match frontend usage.
PDF creator and modification time are displayed.
Reworking file search to use Video.__json__ for complex video relations.
Archives can be viewed.
Removing ebook.title.
Reworking alembic migration to be an upgrade rather than a replace.
Skipping test on circleci.
Clearing out video caption index only if extract_subtitles if off.
Channel tests pass.
Some video tests pass.
Missing videos can be found. A video can be downloaded.
Orphaned files can be found.
Videos can be tagged.
Videos can be searched.
Videos can be modeled.
Converting some tests to use FileGroup.primary_path.
Adding archive and domains to alembic migration.
Testing cascading Archive titles.
Archives now uses FileGroup.
archive.lib tests pass.
Archive is partially converted to FileGroup.
Calculating FileGroup.primary file on refresh, rather than on select.
wrolpi tests pass.
Using FilesType to convert between str/pathlib.Path for FileGroup.files.
Using list of files in FileGroup.files instead of dictionary.
Storing files in dedicated FileGroup.files to avoid overwriting them.
Adding comment explaining CASE.
Not overwriting FileGroup.data on refresh if no change has been detected.
Removing File model in favor of FileGroup.data.
Adding typing to Files methods.
Files are indexed using the primary mimetype of the FileGroup.
files test_lib tests pass.
Removing file_group_file in favor of simple one-to-many relationship.
PDF files are indexed during refresh.
Adding Tag and TagFile to tag files with specific tags.
Only iterating when full_stem_ids has contents.
Testing that FileGroups are related to their Files.
Files and file_groups are created and linked.
Bump webpack from 5.75.0 to 5.76.1 in /app (#160)
Bumps [webpack](https://github.com/webpack/webpack) from 5.75.0 to 5.76.1.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.75.0...v5.76.1)
---
updated-dependencies:
- dependency-name: webpack
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Moving FilesRefreshButton and DirectoryRefreshButton out of browser file.
Moving file fetching tino FileBrowser.
Removing sub-folders from Files display when any parent folder is clicked.
Using closed captioning for vtt/srt file icon.
Using my own react-semantic-toasts-2 to use toasts in Node 18.
Fixing bug where more than the last page was displayed.
Upgrading yt-dlp.
Fixing sub-folder file selection.
Adding rowKey to SortableTable to give React a key for each row.
Renaming key to path.
Removing duplicate urls.
Unifying file modals.
Displaying modals for image and text files.
Handling refresh loading in FileBrowser. Creating refreshing_directory flag.
Removing accidental commit.
Sorting by url_count.
Using SortableTable on Domains page.
Using SortableTable for Channels page.
Created new FileBrowser which allows browsing of files in the media directory.
Using Node 18 in pi-gen.
Creating new FileBrowser to replace outdated react-keyed-file-browser.
Returning mimetype with file list.
Do not display 'Do not download' on Channel download in Dashboard.
Allowing null channel download frequency so user can stop downloading a channel.
Bumping version for Node 18.
Using my own semantic-ui-offline to avoid pulling Google fonts.
Upgrading app docker container to use Node 18. This requires the removal of several npm packages. File browser will require re-write. It now pulls fonts from Google, this will need to be fixed.
Testing DownloadsToggle click.
Testing trinary states of DownloadsToggle.
Installing @testing-library/user-event for testing App.