Commit graph

657 commits

Author SHA1 Message Date
Roland
898da37d47
Fixing bug where zim articles were not aligned in modals. 2026-06-11 19:46:34 -06:00
lrnselfreliance
9a5d1f1630
Add ordered collections (playlists): polymorphic items + item API (#475)
* Add Playlists: ordered, mixed-type collections

A playlist (Collection kind='playlist') is a manually-curated, ordered list that
mixes item types. Squash of the playlists feature branch (24 commits).

Data model & API:
- CollectionItem is polymorphic via item_kind: 'file' (FileGroup), 'zim'
  (zim_id + zim_entry), or 'url' (link to anything the WROLPi browser opens,
  e.g. /map?lat=&lon=&z=). CHECK constraint enforces exactly one kind; per-kind
  unique constraints; migration a7b1c2d3e4f5.
- Endpoints: create playlist; add/remove/reorder items (idempotent adds: 201 new,
  200 existing); GET returns the ordered items. All mutations wrol_mode_check'd.

Config (source of truth):
- playlists.yaml (PlaylistsConfig) stores each playlist's metadata AND ordered
  items (file by relative path, zim by path+entry, url inline); round-trips a
  DB wipe. Supports backup import (DB_CONFIG_FILE_NAMES).
- The redundant generic CollectionsConfig (collections.yaml) is deleted:
  channels/domains own their configs, authors/subjects are re-derived.

On-disk sync (sync_playlists_directory switch):
- Each playlist materializes into the configurable Playlists Directory
  (playlists_destination, Settings UI input, excluded from indexing) with
  position-prefixed entries: NNNN_<file> hard links (copy fallback across
  filesystems) and NNNN_<title>.html stubs for zim/url items.
- Tagged playlists are namespaced like channels: <playlists>/<tag>/<name>;
  a tag change moves the directory and prunes the empty old tag dir.
- Deletion safety: managed files are only removed when they are our marked
  stubs or still have another hard link (st_nlink > 1) -- never a last copy.
  Stray loose files at the playlists root are removed; stub URLs are
  scheme-allowlisted (no javascript:/data:).

Frontend:
- /playlists list page (CollectionTable, name filter, New Playlist modal, Tag
  column) between Files and Zim in the nav.
- View-only detail page (Name link) and edit page (Edit button): themed table
  with model icons/colors + poster thumbnails, items link to their model page
  (video/archive/doc) or preview, up/down reorder, remove, add-URL form,
  single-tag selector; all editing disabled in WROL Mode.
- Reusable green "Add to Playlist" button on Video/Archive/Doc detail pages,
  the FilePreview toolbar, Zim search results, and Map pins (added as a /map
  URL item with a map-marker icon).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Playlist edit page parity: rename, custom directory, tag modal move suggestion

Match the ChannelEditPage/DomainEditPage UX on the playlist edit page:
Back/View buttons, an edit form (name, description, directory), and the
shared CollectionTagModal with the violet "Tag" button.  The tag modal now
suggests moving the playlist to its managed tag location
(playlists/<tag>/<name>); accepting the suggestion stores directory=None so
the sync keeps auto-managing the location.  Tag buttons on the Channel and
Domain edit pages are now violet to match.

Backend: update_collection learns rename (with per-kind validation and
duplicate checks) and playlist custom directories (cleanup of the abandoned
directory, config round-trip, commit-before-switch to avoid the background
sync racing pre-commit state).  tag_collection's duplicated untag/tag
branches are merged, and the per-kind config-save dispatch is consolidated
into _activate_config_save().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Deduplicate Collection error handling and Tag creation

Delete the duplicate UnknownCollection in wrolpi/collections/errors.py and
re-export the canonical one from wrolpi/errors.py, which carries
status_code=NOT_FOUND (the local copy inherited 500).  Replace the repeated
query/raise pattern in reorganize.py with Collection.find_by_id.

Add Tag.get_or_create_tag and use it for collection tagging; flush the new
Tag before returning so callers can read .id immediately -- previously
tagging a collection with a brand-new tag name set tag_id=None.  Regression
test included.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 19:46:34 -06:00
lrnselfreliance
e018616daf
Warn in the navbar when a drive's SMART health is failing. (#471)
A failing SMART self-assessment predicts imminent drive failure, so it
deserves a prominent warning.  useDriveHealth() reads the assessment
already present in smart_stats (pySMART reports PASS / FAIL / null) and
flags any drive whose assessment is FAIL.  Nav.js renders a red warning
icon in its own always-on slot — like the power warning, and unlike the
single-slot priority chain where a transient high temp could hide it.

Frontend-only: the assessment field is already in the cached payload.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 19:46:33 -06:00
lrnselfreliance
1bd102c768
Warn in the navbar when a hard drive runs hot (#470)
* Warn in the navbar when a hard drive runs hot.

The navbar already warns about high CPU temperature; add the equivalent
for drive temperature, since an overheating disk risks data loss and a
user running an off-grid library won't be watching for it.

- controller/lib/smart.py: build_smart_stats() returns the per-drive list
  plus warning thresholds (55C warn / 65C critical) so the frontend stays
  dumb. Reading SMART wakes a spun-down drive, so drives in standby are
  detected with `smartctl -n standby` (which works through the USB-SATA
  bridges where `hdparm -C` only reports "unknown") and NOT read; their
  last-known reading is reused instead.
- controller/lib/status_worker.py: collect SMART on a slow 60s cadence
  (cached on app.state) rather than the 5s status loop, so the poll does
  not keep USB drives awake. The result flows into the same cached payload
  via a new collect_all_status(smart_stats=...) kwarg.
- Frontend: useDriveTemperature() reports the hottest drive; Nav.js shows
  a yellow/red `hdd` icon with a popup, mirroring the CPU warning.

Verified on hardware: smart_stats reports real temps and SMART runs once
a minute, not every 5s.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Address Greptile review on HDD temperature warning.

- _drive_is_asleep also treats SLEEP mode as asleep, not just STANDBY.
  smartctl -n standby exits "Device is in SLEEP mode, exit(2)" for the
  deeper power mode; matching only "STANDBY" meant a SLEEP drive was read
  and woken — the exact failure the check prevents.  (TDD: added a failing
  SLEEP test, then fixed.)
- useDriveTemperature uses ?? instead of || for the temperature fallback
  to express "fall back only on null/undefined".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 19:46:33 -06:00
Roland
9d6eb0873e
Controller can unmount foreign mounts; protect the primary drive.
The /api/disks/unmount endpoint only removed entries from fstab.yaml and
asked the reconciler to unmount, but the reconciler deliberately never
unmounts paths it has not claimed.  Foreign mounts (udisks2 desktop
automounts, manual mounts) therefore could not be unmounted from the UI,
failing with a confusing "check journalctl" error.

- Fall back to a direct umount(8) when a reconcile leaves the requested
  mount point live.
- Reject unmounting the primary WROLPi drive (/media/wrolpi) with a
  clear 400, normalized to catch path traversal.
- Require the normalized mount point to be strictly under /media/
  before any direct umount, since the fallback runs as root.
- Show "System" instead of an Unmount button for /media/wrolpi in both
  the React UI and the Controller's simplified UI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 19:46:30 -06:00
Roland
5497e1eb61
Fixing flaky domain search test. 2026-05-31 13:09:20 -06:00
Roland
345b69b7d7
chore: address Greptile P2 review comments on PR 449
- In get_effective_video_settings: on unparseable sleep_requests, delete
  the key instead of forcing 0.0. This lets the global config default
  (already present from _extract_global_inheritable) take effect, as
  suggested by Greptile.

- In handleInputEvent: add explicit `value !== ''` guard before the
  isNaN check. Prevents silent NaN in form state for empty number
  fields (isNaN('') === false in JS).

Both changes are strict improvements with no behavior change for
valid inputs.
2026-05-31 13:02:44 -06:00
Roland
9cc5c15e32 fix: coerce sleep_requests and numeric settings from string form values
`sleep_requests` (and other numeric inheritable video download settings)
could arrive as strings ('' or '0.75') from the frontend NumberInputForm
and DownloadRequest bodies. These were persisted into Download.settings
(JSONB) and later triggered:

    if effective['sleep_requests'] > 0:
    TypeError: '>' not supported between instances of 'str' and 'int'

in VideoDownloader.execute_download (and similar sites).

Root cause: DownloadSettings.__post_init__ only filtered None/[] and did
no type coercion, even though the dataclass declared Optional[float]/int.

Changes (minimal targeted fix):
- Add _coerce_optional_number helper and apply it to all numeric fields
  in DownloadSettings.__post_init__ (the existing sanitization boundary).
- Centralize legacy string coercion for persisted data in the single
  read chokepoint: get_effective_video_settings.
- Simplify VideoDownloaderConfig.sleep_requests getter.
- Frontend: use parseFloat for type=number inputs + null defaults for
  sleep_requests (prevents '' from ever being submitted).
- Add focused regression test for string coercion path.
- Revert unrelated change to string field filtering behavior.

No NaN/Inf speculation, no duplicated try/float guards at every call site,
no behavior change for non-numeric settings.
2026-05-31 13:02:31 -06:00
Roland
342374b97f
Guard preset lookup, drop dead Media import and volume*1 no-op
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
Address Greptile review on PR #447:
- Fall back to the minimum preset when an unrecognized/stale
  water_calculator_preset is read from localStorage, so a bad persisted
  value can no longer crash the component on mount.
- Remove the unused Media import.
- Simplify the metric branch of waterWeight (volume * 1 -> volume).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 18:07:01 -06:00
Roland
d998bb93a1 Adding realistic lifecycle test. 2026-05-28 18:06:48 -06:00
Roland
d3e57ea894 Water Storage calculator: rate guardrails, persisted duration, extra demand
Addresses calculator review feedback:
- Add an "Additional daily demand" (Extra) field for pets, livestock,
  guests, or sick household members not covered by the per-person rates.
- Make the preset choice explicit buttons with a note that switching
  resets rates to the preset defaults.
- Clamp rate and extra inputs to non-negative via a shared helper that
  preserves in-progress decimal entry (so "1.5" is typeable).
- Persist the duration slider to local storage (with an out-of-range
  guard), matching the people-count/metric/preset persistence.
- Relabel "Total Weight" as "Water Weight" to avoid implying it includes
  container tare weight.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 18:06:48 -06:00
Roland
c63fece967 Add Water Storage calculator (people → duration → containers)
New offline prepper calculator under Calculators:
- Men/women/children/pregnant-nursing counts with editable per-person
  daily rates (drinking + hygiene/dish use).
- Minimum/Comfortable presets grounded in CDC/FEMA (1 gal/person/day
  floor, store 2 weeks) and National Academies adequate-intake figures;
  Pregnant/Nursing adds the CDC/Ready.gov +0.5/+1 gal. Sources cited in
  the Daily Use info popup.
- Duration slider on a clean ladder (days → weeks → months → years) so it
  never lands on awkward values like "68 days".
- Containers Needed table listing every container (IBC tote → 500 mL
  bottle) with capacity and count + spare for the total volume.
- Total water and total weight (8.345 lb/gal, 1 kg/L).
- Imperial/metric, preset, and people counts persist via local storage.

Pure functions (dailyDemand, totalWater, containersNeeded, waterWeight,
formatDuration) covered by 21 unit tests. Registered in the calculator
list and global app search.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 18:06:48 -06:00
Roland
2e950b85ad
Drop spammy console.debug from Drive Ratio render/keystroke paths
Address Greptile review: remove the per-render and per-keystroke console.debug
calls (they fired on every React render and every field edit). Keep the single
strategic solveTrain log, which fires only on a settled (debounced) solve.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 16:19:21 -06:00
Roland
4e25459d50 Add Drive Ratio calculator to global search apps
Add a SUGGESTED_APPS entry so the Drive Ratio calculator shows under "Apps"
in the global search, matching the other calculator entries.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 15:17:50 -06:00
Roland
c38f90aae8 Add Drive Ratio calculator (pulleys, gears, sprockets)
A new "Drive Ratio" calculator under /more/calculators for belt, gear, and
chain-sprocket drives, supporting a compound train of 2–6 wheels on a single
shared belt/chain (constant size·rpm across the train).

- Pure, unit-tested math: size·rpm solver, ratio, torque, surface/belt speed,
  belt length, sprocket pitch diameter, chain length, gear pitch diameter.
- Mode toggle (Pulley/Gear/Sprocket) with imperial/metric units; torque in
  lb·ft or N·m.
- Add/remove wheels (2–6), each with a colorblind-friendly color shared between
  its inputs and the schematic SVG diagram (Paul Tol bright palette).
- Dependency-free inline-SVG diagram: colored wheels threaded by one belt/chain,
  or meshing gear teeth; auto-fitting viewBox.
- Per-mode details: belt/chain length and center distance (2-wheel only, with an
  info popup), per-wheel pitch diameters, and constant train speed.
- Guards against impossible geometry (wheels overlapping) and ideal-torque note.
- Integer teeth enforced in gear/sprocket modes; pulley diameters stay continuous.

Also fix Theme.Header dropping the theme's text color when a caller passes a
`style` prop (now merges {...t.style, ...props.style}).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 15:17:50 -06:00
Roland
cd8792f697
Warning users when files exist in mount location. Users may have accidentally downloaded files into the media directory, we want to warn them.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-05-14 17:02:52 -06:00
Roland
da340a00b3
Displaying warning when no drive is mounted. 2026-05-14 16:16:18 -06:00
Roland
86b7782361
Adding toggle to skip already downloaded archives and files.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-05-05 14:48:35 -06:00
Roland
ba63170e55
Consolidating deletion endpoints.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-05-03 19:14:30 -06:00
Roland
09a967535b Models like Archive/Video/Doc can be deleted, even if they are tagged. User is shown a warning and must confirm deletion of tagged files. 2026-05-03 19:14:15 -06:00
Roland
513ae52a6e
Using style on extension page description.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-05-03 13:57:33 -06:00
Roland
a9a460c54c
Creating browser extension to facilitate starting downloads. Chrome/Brave and Firefox supported.
The extension is hosted in a sibling repo and
is distributed off-store. Each WROLPi release ships the prebuilt artifacts
under wrolpi/static/extensions/ and serves them at /api/extensions/<file>:

  - wrolpi-chrome.zip       Load-Unpacked sideload for Chrome / Brave / Edge
  - wrolpi-firefox.xpi      Mozilla-signed (unlisted self-distribution),
                            one-click install in any Firefox
  - versions.json           Per-browser version metadata for the install page

Backend (modules/archive/api.py, wrolpi/root_api.py):

  - POST /api/archive/upload accepts an optional repeated `tagNames`
    multipart form field; the upload switch handler applies tags to the
    new FileGroup via FileGroup.set_tags(). Unknown tag names are auto-
    created inside a per-name SAVEPOINT so concurrent inserts on the same
    novel name don't abort the transaction (Tag.name has unique=True).
  - GET /api/extensions returns metadata (per-file availability + size +
    versions.json contents) for the install page.
  - GET /api/extensions/<filename> serves the binary, whitelisted by
    filename. Firefox .xpi is served as application/x-xpinstall WITHOUT a
    Content-Disposition: attachment header, so Firefox triggers its
    add-on install handler instead of saving the file.

Frontend (app/src/):

  - New /admin/extension page (linked from AdminRoute) detects the user's
    browser via navigator.userAgent and renders an install card per
    Firefox / Chromium / Other. Pulls metadata from /api/extensions to
    show file size and version, and gracefully reports 'Not yet
    available' when binaries are missing.
  - Dashboard's Getters component supports a deep link from the
    extension's 'Configure Download' button:
      /?downloader=<archive|video|...>&download_url=<u1>&download_url=<u2>
    On mount, the matching downloader form auto-opens with URLs pre-
    filled. Query params are cleared on close so a refresh doesn't
    re-open the modal.
  - DownloadMenu and the per-downloader forms accept an `initialUrls`
    seed via the existing `download` prop pattern, joined with newlines
    for the `urls` textarea (and first URL for single-URL forms).

Behavior is fully backwards compatible: pre-existing API callers can
still POST to /api/archive/upload without `tagNames`, and the
DashboardPage modal continues to open via the existing button click
when no deep-link query params are present.
2026-05-03 11:48:50 -06:00
Roland
ae460af8d6
URLs can be dropped onto download textarea.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-05-02 13:35:08 -06:00
Roland
7bf1c80847
Fixing bug where pagination would show an extra link that would load no results.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-04-30 17:40:26 -04:00
Roland
6b12e97cf0
Changing model file icon to teal tube.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-04-15 14:04:07 -06:00
Roland
5dd5933451 Adding 3mf file viewer to FilePreview. 2026-04-15 14:03:56 -06:00
Roland
575a1b2a80
Reworking file preview buttons to be consistent across screens. 2026-04-15 13:18:45 -06:00
Roland
29133b1e56
Reworking StlViewer size so that it is no longer cropped by the window. 2026-04-15 12:53:27 -06:00
Roland
66cecce5e6
Adding Doc filter button so users can search just PDFs/EPUB/etc. 2026-04-14 15:35:35 -06:00
Roland
9468448a54
Modifying Video/Channel/RSS Downloaders so that they can download videos, but audio only. These audio files are grouped under their channel directories. 2026-04-14 15:33:01 -06:00
Roland
8fb9e35aef
Displaying confirmation modal when deleting tagged files, rather than just refusing to delete them. 2026-04-14 11:46:50 -06:00
Roland
8a75cf9057 Linking to specific page or section of an EPUB/PDF when user searches. 2026-04-13 16:47:02 -06:00
Roland
c726654de9
Map search added to global search. Preview cards are displayed for all results in new Map tab.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
Map viewer has simple search.

Map search databases can be downloaded from CDN, or built locally. Map searches can be built offline, but will not have enriched things like state names downloaded from Wikidata.
2026-04-13 12:17:08 -06:00
Roland
74a4198f8d
Not using flexbox when measuring navbar item widths so that Admin link is not cropped.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-04-10 15:27:15 -06:00
Roland
1294b14c10
Adding button to bulk tag file groups for collection items. Always displaying selection column on collection page tables.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-04-09 16:18:26 -06:00
Roland
4b34d37838 Navigation links automatically hide based on screen width, rather than hard breaks. 2026-04-09 16:17:38 -06:00
Roland
a8b756b01a
Deleting archives by file group id, rather than archive id. 2026-04-09 15:29:28 -06:00
Roland
a720818625
Removing Controller link in favor of the button in services. 2026-04-09 15:03:28 -06:00
Roland
e0324ce0fc Migrating Controller from port 8087 to 80 so that it is more discoverable. Reformatting Controller dashboard to make header at the top. Reformatting navigation bar to handle the new links which prevented Admin from being visible in some screens. 2026-04-09 15:01:05 -06:00
Roland
3a21f7b23f
Pressing enter while only one tag is filtered will apply it. 2026-04-04 18:08:33 -06:00
Roland
169d2d37f0
Adding more dashboard unit examples. Adding tests for unit conversion. 2026-04-04 17:56:08 -06:00
Roland
bbc23dbf90
Adding map default view option to the map context menu. 2026-04-04 14:56:25 -06:00
Roland
b8a532c8d9
Key changes:
- Add MapLibre GL JS viewer with light/dark themes, terrain layer,
  and embedded static map page for coordinate sharing
- Add map pin support with persistent map_pins.yaml config
- Add map region subscriptions and downloader with progress tracking,
  sourcing pre-extracted PMTiles from a cold CDN
- Add map export to PNG and GPS coordinate copy via context menu
- Add MapViewer React component alongside reworked Map page
- Add map catalog and region-building script (build_map_regions.py)
- Remove OSM import/render scripts, leaflet files, Apache configs,
  renderd service, and osm-map-dumper Docker container
- Update Caddyfile to serve PMTiles and map static assets
- Update build/install/repair scripts for Protomaps dependencies
- Drop map_file table, add/drop map_subscription table (alembic)
- Adding button to skip primary drive mounting during onboarding.
2026-04-04 13:48:19 -06:00
Roland
0a5fe7eebe
Displaying 7z/rar preview when user previews those archives.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-03-31 12:34:42 -06:00
Roland
ce0cdd3a2e Samba is installed and can be configured in the Controller. 2026-03-29 10:19:28 -06:00
Roland
bf5b787fb3
Unifying Files tab appearance for model pages.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-03-28 14:19:09 -06:00
Roland
2943b2a9c8
Adding buttons to DocPage so that user can select which version of the same document to view, for example a PDF or EPUB of the same document. 2026-03-28 13:52:01 -06:00
Roland
53f4e29b64
Updating controller hotspot and bluetooth toggle commands with real paths for RPi. 2026-03-28 10:35:07 -06:00
Roland
1c55745884
Adding bluetooth radio toggle to Controller. Writing controller.yaml to media drive on startup, if it doesn't exist. 2026-03-28 10:09:58 -06:00
Roland
9e80683b9a
Adding model dimensions to STL preview.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-03-27 17:59:58 -06:00