* The backup drive keeps deleted files for 30 days, then removes them
The fifth part of replacing the sync. A backup drive never deleted anything, so it grew forever and kept every file
the media directory had ever held. Now, like a secondary:
- A file deleted from the media directory, or the old version of one that changed, is moved to a dated folder in
.backup-trash on the backup drive. Folders older than backup_delete_after_days (sync.yaml; 30 by default, also
for drives set up before this; none never deletes) are removed, before and after each backup, and the oldest go
sooner when the drive is nearly full.
- A backup refuses a media directory that is not mounted, or has no config/wrolpi.yaml: copied with deletes, an
empty folder would move the whole backup drive to the trash.
- The Backup Drive panel has the same "Delete after N days / Never delete them" control as the Sync panel, now one
component, and refuses an empty number of days.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A nearly full drive keeps the trash it just filled, and makes room for the database
Review found, in the backup drive's trash and in the secondary's sync, which share its shape:
- Below 1 GiB free, the purge after a copy removed the newest trash folder too. Moving a file to the trash is a
rename, which frees nothing, so on a nearly full drive a deleted file was gone at the end of the same run instead
of 30 days later. The newest folder is now only removed by age (0 days still deletes at once).
- The purge made room only down to 1 GiB, so a database larger than the free space failed to be copied every time
while the trash was full. Before the database is written (the backup's snapshot, the secondary's copy of its
primary's), the trash now makes room for the whole database.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* The old sync engine is removed
Part 4 of replacing the sync. Since the previous part a secondary copies its primary with rsync, run by the
Controller, so nothing used the HTTP engine any more:
- the manifest (/api/sync/manifest), the download of a FileGroup's files over HTTP, the copy and import of each
synced config, the moving of a replaced file's links, the request, cancel and tick of the old sync;
- the API lock that answered 503 SYNCING while that engine ran, its Event, the sync_running flag, the old status
record in /api/status and its schema, and the frontend's handling of 503 SYNCING;
- their tests.
What a primary says about itself (/api/sync/info), the secondary's settings (its primary, the folders it does not
copy, Test Connection, the folder chooser), taking the primary's library settings when the API starts, and the
daily window of the backup drive stay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* The primary's sync info no longer reports the largest FileGroup id
Only the old engine read it. Review also found a docstring that still spoke of a manifest.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A sync is announced again
Since the Controller runs a sync, nothing sent its events: the old engine did, and the Controller cannot send the
API's events (the API is stopped for most of a sync). The API's loop now reads the Controller's record of the last
sync and sends one event per finished sync: "Synced N files from the primary" (with the files moved to the trash,
or a warning) or "Sync from the primary failed". As before, a sync that changed nothing is not announced (within
the window that is every 15 minutes), nor one that was stopped on purpose. What was announced is kept in
config/.sync, which a sync never touches, so the restart every sync makes neither repeats nor loses one.
Found by review, and by the dead-code check, which saw the two event methods unused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* A secondary syncs by rsync: the Controller runs it on its schedule, and both pages follow it
The third part of replacing the sync.
- The Controller syncs within the daily window (every 15 minutes after the last sync finished, in WROLPi's
timezone), as the API did. The API's loop no longer runs the old engine, and its /api/sync/run and
/api/sync/cancel ask the Controller.
- The Sync page and the overlay that covers the app read the sync's status from the Controller, which is the only
one that can answer while the API is stopped. The overlay is also shown when the settings cannot be read, as
happens while the API is stopped. Sync Now and Stop go to the Controller.
- The Controller's fallback page, where Caddy sends the browser while the API is down, shows the sync's progress
and a Stop button.
- New on the Sync page: how long files the primary deletes are kept (30 days by default, or never), and a note that
an excluded folder's files are still listed on a secondary.
The old engine's code is removed in the next part.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Remove SyncRunRequest, unused since a sync is asked of the Controller
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* In Docker, a sync's progress moves while a large file is copied
Docker's logs hold a line until it ends, and rsync's progress for one large file is a single line rewritten with
carriage returns, so the page showed 0% until the file was done (hours, for a large map). The rsync helper's
output is now read by attaching to it, before it is started so that nothing is missed.
Found by a sync between two Docker WROLPis.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A sync is recovered before the schedule can start another, and an empty days value is refused
Review found:
- The schedule and the recovery of an interrupted sync started side by side when the Controller started. The state
an interrupted sync leaves is due, so inside the window a new sync could start while the recovery restarted the
API and wrote over the new sync's status. Every tick of the schedule now recovers first, and only a status of
"running" with no sync holding the lock counts as interrupted. A sync records its outcome before it frees the
lock, so a tick between the two cannot mistake it for an interrupted one.
- The Controller may start before the drive is mounted. The empty record it read then was kept and saved over the
real one, losing the last successful sync. The state file is now read again until it can be.
- A cleared number of days was saved as 0, which empties the trash at the next sync. It is now refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* The Controller can make a secondary a copy of its primary with rsync
The second part of replacing the sync. The Controller runs it, because the secondary's API is stopped while its
library is replaced: checks, a database snapshot on the primary, stop the API, rsync every file from the primary's
daemon (ignored directories included), fetch the snapshot and put it in the database's place, start the API, purge.
- Files the primary deleted or changed are moved to a dated folder in .sync-trash, and removed after
delete_after_days (a new sync.yaml setting, 30 by default; none never deletes), or sooner when the drive is nearly
full.
- A primary of another version or database migration, or with its media unmounted, is refused before anything is
touched. The database is replaced last, so a sync that stops earlier leaves one that works.
- On a Raspberry Pi and Debian rsync runs as the wrolpi user; in Docker in a helper container made from the api
container. A Controller that starts after an interrupted sync starts the API.
- When a secondary's API starts it takes the library settings of the primary's wrolpi.yaml, which the sync left.
Reached only through the Controller's /api/sync/run for now; the existing sync and its schedule are unchanged.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
* The sync copies no link that leaves the library, and root follows none in the media directory
Security review of the Controller's sync:
- An exclusion that ends in a slash matches only a directory. A file or link named config/ssl, tags or
config/backup on the primary would have replaced this machine's directory of that name. The exclusions now match
whatever has the name.
- Links were copied as they are. One that points outside the primary's library would, on the secondary, point at
the secondary's own files, which the web server then serves. Only links that stay inside the library are copied,
and no device files.
- The Controller is root, in a media directory that belongs to another user. It no longer writes, reads, replaces
or purges through a symbolic link there.
Found on the way: the directory the Controller made for the incoming database belonged to root, so rsync, which
runs as the library's owner, could not have written into it. It now belongs to the media directory's owner.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
* A sync ends when its primary stops answering, and makes room before it copies
Review found:
- rsync waits forever by default. A primary that lost power or its link during a copy left the secondary's API
stopped until someone cancelled. rsync now gives up after 30 seconds without a connection and 5 minutes without
data, which ends the run and starts the API.
- The trash was only purged after a run that succeeded. The trash is what fills the drive, and a full drive fails
every run, so it never got its turn. It is now also purged before the copy.
- A value in sync.yaml that could not be used raised after the run lock was taken and before the handlers that
release it, leaving "A sync is already running" until the Controller restarted.
- The playlists directory left out of the copy was this machine's, not the primary's, whose hard links would have
arrived as full copies. The primary's wrolpi.yaml is now fetched before the copy, and both are left out.
- A primary whose media directory is elsewhere is refused: its database names every file by its full path.
- In Docker rsync is its own container, which outlives a Controller that stops. A Controller that starts after an
interrupted sync now removes it before it starts the API.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
The Celsius/Fahrenheit/Kelvin converter is removed from the Calculators
page and from the search suggestions.
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
A file of features often carries a basemap's layers too (roads, water, buildings, place names).
Drawn over the basemap they double every road and label. Those layers, by the names of the
Protomaps and OpenMapTiles schemas, are now off until they are checked; every other layer is on
as before, and a choice already made is kept.
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
An overlay drawn with a style of its own names icons, which needed the sprite sheet that style was made for. The map now ships icons for overlays, the Maki and Temaki sets (both CC0), and always draws those, in both viewers.
The sprite sheet is the translation: its index lists every icon under its own name, under each other name it answers to (modules/map/icons/aliases.json), and in both spellings of each. A name with no icon is drawn as a plain dot. The icons are signed distance fields, so a style's icon color and halo apply.
A pattern is not an icon: what it filled (a fill, a line or an extrusion) is drawn in a translucent color, where removing a pattern used to leave MapLibre's opaque black.
A sprite sheet beside a map file is no longer used, served or reported. wrolpi/scripts/build_map_icons.py builds the sheet from the SVGs; the sheet is committed.
The standalone viewer (port 8084) drew basemaps only: a map file which is
not a basemap was skipped. It now offers each overlay in its layer panel,
hidden until turned on, with a checkbox per layer, draws it with the style
beside it when there is one, and shows a feature's attributes on a click.
The overlay logic is the map viewer's own. It moved to dependency-free
modules (app/src/map-shared) which a browser can load as they are; the
standalone viewer, which has no build step, loads copies of them from
modules/map/static/js. A test fails while a copy differs from its module
(`npm run sync:map-shared` makes the copies). What differs between the
viewers is passed in: where a map file and its style are fetched from, and
whether a layer can be checked before MapLibre is given it.
The standalone viewer cannot check a layer first, so it adds an overlay's
layers to the map one at a time once the style has loaded: MapLibre refuses a
whole style over one layer it cannot draw, but only that layer when it is
added by itself.
The API also names a style by its path within the map directory, which is
what the standalone viewer is served.
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
The contour label layer named no font, so MapLibre asked for its default
("Open Sans Regular,Arial Unicode MS Regular"), which the map does not ship.
The request failed and the labels were drawn with glyphs the browser made up
locally. The layer now names Noto Sans Regular, in the map viewer and in the
standalone viewer.
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
When a MapLibre style (and optionally a sprite sheet) sits beside a PMTiles file, the overlay is drawn with it instead of the generic colors: `<name>.style.json` and `<name>.sprite.json/.png`, or `style.json` and `sprite.json/.png` shared by the files of a directory.
Only the style's layers which draw that file are used; its sources, sprite and glyph URLs never are. Fonts are replaced by the ones the map has, images are pointed at the overlay's own sprite (as strings, expressions or property functions), and each layer is validated so one the map could not draw is skipped instead of stopping the map.
The sprite is served by the API, which falls back to the single resolution sheet when there is no double resolution one.
Any PMTiles file under the map directory, at any depth, can be shown. The API reads each file's header and metadata and says what it is (basemap, terrain, overlay, unreadable). Overlays are offered in the layer control under the name the file gives itself, hidden until turned on, with a checkbox per vector layer, and are drawn generically.
Map files are named by their path within the map directory everywhere, and unfinished downloads are no longer listed. A file can be ignored from the Manage page; ignored files are kept in a new maps.yaml config.
PMTiles metadata is not decompressed past its size limit, and the standalone viewer encodes map file paths.
On the search page, choosing Any Tag without a search string left the
Dashboard showing: the page, the Files results and the Files count all waited
for a search string or tag names.
Zim search has no "any tag" search, and its tag picker's Any button left no
filter applied, so the picker no longer offers it. Zim and Other counts are
only requested when there is a search string or tag names to count by.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Search results sort as the filter dialog says
Docs sorted ascending orders backwards and fell back to id order for every
descending order (`-size`, `-published_datetime`); it now takes the same `-`
prefixed orders as the other searches and rejects unknown ones with a 400.
Archives and Docs searches without an order now put the most relevant results
first, as Videos does and as the dialog shows.
Rank has no direction in the dialog: choosing it sends `rank` (most relevant
first) instead of inheriting a descending `-rank` (least relevant first).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* An old -rank link becomes rank when the filter dialog closes
The dialog shows Rank without a direction, but it seeded its draft from the URL
unchanged, so Done left a saved `-rank` (least relevant first) in place.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The filter modal on these pages offered "Any Tag", but only the Files search
sent it; the module searches had no `any_tag` field. Docs also matched any of
the listed tag_names, while every other search requires all of them.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* A secondary WROLPi copies its primary's library over HTTP
A WROLPi with role "secondary" pulls from its primary: a manifest of every
FileGroup's files (/api/sync/manifest), then each missing or changed file
from the primary's Caddy /media/ (resumable, written to a hidden partial and
renamed with the primary's mtime), a refresh, then the library configs,
imported as the primary wrote them. wrolpi.yaml contributes only its library
settings. Files deleted on the primary are kept.
A secondary never saves the synced configs and never downloads. Its API
answers 503 SYNCING while a sync runs, and every page warns that it is a
copy. It syncs in a daily window or on "Sync now"; WROL Mode stops it.
On a Pi 4 (no AES instructions) the client uses TLS 1.2 ChaCha20, which
pulls ~110 MB/s from Caddy versus ~41 MB/s for rsync over ssh.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Sync has its own /admin tab; a sync's reads from the primary are bounded
The sync settings and progress move from Settings (and the component
gallery, which is for library components) to /admin/sync.
A download never writes past the size in the primary's manifest, and configs,
listings, and manifest pages are capped. A primary address may not carry a
user name or password (it is logged and shown). While a sync runs, the API
answers only the exact requests that show its progress or stop it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A sync keeps the secondary's replaced configs in today's config backup
The first copy of the day is kept, like a save, so a config backup import
undoes a sync's config changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Docker's Caddy logs every request
`docker compose logs -f web` shows each request with its status, size and
duration, e.g. a secondary's /media/ downloads during a sync.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A sync reports its progress as it downloads, and refuses quietly
The status updated only when a page of 500 FileGroups finished, so a page of
large files looked stuck; it now updates every 2 seconds. Requests refused
while syncing are answered by the middleware instead of raised, so the UI's
polling no longer logs an error each time. The primary's version is stored
without its trailing newline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A WROLPi can back up to a second drive; sync settings live in sync.yaml
The Sync tab's Backup Drive copies the media directory to a second mounted
drive with rsync during the sync window (hourly), or on "Back Up Now".
Nothing on the backup drive is deleted. The Tags and playlists directories
are left out (WROLPi rebuilds them from the configs), and the database is
written as an SQLite snapshot rather than copied as files, so the drive can be
mounted as the media directory and start. A backup is written only while the
chosen drive (by UUID) is mounted where it was chosen, and is not the media or
system drive; rsync is stopped if the drive disappears. Not in Docker, which
cannot reach other drives.
Role, primary, window and backup drive move from wrolpi.yaml to sync.yaml:
they belong to this machine, and are never copied from a primary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* The Sync tab shows the configured backup drive
getSettings copies each setting by name and did not list the backup drive,
so the Sync tab showed "None" while one was configured. It also dropped
`version`. A test now fails for any setting the API sends that getSettings
drops.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A backup can go to a folder on its drive, and remembers its last success
The Backup Drive panel takes a folder on the chosen drive. The drive is
checked through the folder's mount point: the folder is written only when the
drive with the saved UUID is mounted there, so a folder left on the system
drive by an unplugged drive, or on another drive mounted inside the chosen
one, is refused. A missing folder is created once the drive is checked.
When the last backup succeeded is saved in sync.yaml, so it is known after a
restart. "Stop Backup" is disabled while the database snapshot is written (it
runs to the end), and a finished backup clears the request to stop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* The backup drive has its own window
The sync window was shown in the Sync panel but also scheduled the backup
drive. The Backup Drive panel now has its own window (sync.yaml
backup_window_start/end), saved with the drive; the sync window is shown,
and used, only for a secondary's sync.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Sync and the backup drive can each exclude folders
A secondary or a backup drive may be smaller than the primary's library.
Each Sync tab panel has its own excluded folders in sync.yaml, chosen in
a folder browser: a secondary browses its primary's folders (through
Caddy), the backup drive browses this WROLPi's media directory. Excluded
folders are never copied; what was copied before is kept.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* The folder browser does not offer lost+found
Every ext4 drive has a root-only lost+found folder; a sync and a backup
never copy it, so it is not offered for exclusion.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A sync never writes into config or tags, whatever their case
On a media drive that ignores case (exFAT, NTFS, macOS), a manifest path
like Config/wrolpi.db is the real config folder. The protected and
unlisted top-level folders are now compared case-insensitively in the
manifest, in resolve_destination, in excluded-folder validation, and in
both folder listings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A backup writes the database snapshot only when the database changed
Every backup rewrote the whole database snapshot (2.5 GB on a busy
WROLPi), even on the window's idle hourly runs. The snapshot is now
skipped when the live database (its size and mtime, and those of a
non-empty WAL) and the drive's copy are both unchanged since the last
snapshot. The record is in memory, so the first backup after a restart
writes it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Remove the unused save_sync_config switch
Nothing activated it; SyncConfig saves directly. The test server's
log_message override is called by http.server, so it goes in the
vulture baseline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* The Settings upgrade can switch to a chosen branch
The Upgrade panel gains a Branch input: a dropdown listing the current
branch, master, and release that also takes a typed branch name.
/api/upgrade/start accepts an optional {"branch": ...}; without it the
current branch is upgraded as before. A branch is written into the systemd
env file and passed to `upgrade.sh -b`, so only plain git branch names are
accepted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* The Controller upgrade refuses a branch that is not a plain branch name
The Controller's upgrade script wrote its branch straight into the systemd
env file, so a newline in it added a variable, and a leading dash became an
upgrade.sh option. It now accepts the same branch names as the API, and no
script parameter may span lines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* STL files preview with three's own STLLoader
react-stl-viewer inlines a copy of three-stdlib 2.17.2's STLLoader, which reads
ASCII STL text with LoaderUtils.decodeText. three removed that in r175, so
every ASCII STL preview failed. The three-stdlib override never reached that
copy, and react-stl-viewer has had no release since January 2024.
The 3MF preview's three.js viewer now previews STL files too, loading each
format with the loader that ships with three, so the loader always matches the
installed three. react-stl-viewer and the three-stdlib override are removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* An e2e spec previews ASCII and binary STL files in a real browser
The STL preview only fails in a browser with WebGL, so the spec opens both
encodings on the live stack and checks the dimensions line appears without a
load error. fb:reset can now write binary files ({base64: ...}).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* STL previews stand Z-up models upright
STLs are Z-up (the 3D printing convention) and three is Y-up, so the
preview rotates STL geometry to stand the model upright, as react-stl-viewer
did. Model Z reads as Height again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 3MF previews stand Z-up models upright too
3MF is Z-up like STL, so the rotation moves from the STL loader into the
shared viewer and applies to every model. Model Z reads as Height.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* File previews show their buttons without scrolling
.preview-fit reserved a fixed 18.75rem for the rest of the preview modal,
which was short: every framed preview overflowed a little, and a 3D model's
dimensions line pushed the action buttons below the fold. The preview modal
now fills its height as a flex column, and the .preview-fit takes whatever
height is left. Previews without a .preview-fit keep their content's height.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* 3D previews frame the model and light it brightly
- The camera fits the model's bounding sphere into the narrower field of view,
from a raised three-quarter front view, instead of backing off 2.2x along Z.
- A hemisphere light and a headlight that follows the camera (from a little
above and left) replace lights whose intensities were set before three's
r155 lighting change and read dim.
- Geometry without normals (3MF) gets creased normals, so flat faces no longer
shade into their neighbors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* A directory rename is finished by the file worker, not the request
A directory rename queued a move, waited for it, then removed the old
directory and updated nested Collections itself. A rename that outlasted
Sanic's RESPONSE_TIMEOUT (60 s) was cancelled while waiting: the move still
finished, but the old directory and its emptied subdirectories were left
behind, nested Collections kept their old directories, and the browser was
told the rename failed.
The move job now finishes the rename (nested Collections, then removing the
old directory, empty directories only), and the request returns once the
move is queued. A failure is reported by the job's event. An empty
directory can be renamed (it raised before). The worker status carries
`rename_from`, so the Files page progress bar labels the rename, and the
file browser fetches again when the file worker is done.
A directory rename no longer queues a refresh of its parent: the file
worker runs the newest task first, so it scanned the directory before the
rename.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A queued rename returns its job, and the file browser waits for it
Review fixes:
- The cancellation test patched `wait_for_job`, which this path no longer
calls, so it cancelled nothing. It now asserts the invariant: the
request only queues the rename (it never waits, and nothing has moved
when it returns), and the file worker alone finishes it.
- The file browser inferred the end of a rename from the file worker's
polled status. A short job can start and finish between two polls, and
the first idle poll (before the job started) spent the "fetch once"
token, so the table could keep the old name. POST /api/files/rename now
returns `{"job_id": ...}` (null when the rename is already done), GET
/api/files/jobs/<job_id> reports pending/complete/failed, and the file
browser polls that job before fetching.
- useBrowseFiles ignores a fetch that settles after a newer one, so a
listing requested before a rename cannot replace the one after it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
react-stl-viewer pins three-stdlib 2.17.2, whose STLLoader decodes ASCII STL
files with three's LoaderUtils.decodeText. three removed that function, so
every ASCII STL preview failed while binary STLs (which never decode text)
still worked. Override three-stdlib to 2.36.1, which ships its own decodeText.
Jest can now load three: a small transform converts three/three-stdlib ES
modules to CommonJS without babel-preset-react-app, whose class-properties
plugin rejects three's static class blocks at our pinned Babel version.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Legacy inventory items stored date fields as Unix epoch seconds. The table showed the raw numbers, and sorting and expired highlighting did not treat them as dates.
- Frontend parses date values as YYYY-MM-DD or legacy epoch seconds; display, mobile view, export, search, sort and the expired check all use calendar dates with no timezone shift.
- The legacy migration converts epoch seconds to YYYY-MM-DD, and import repairs already-migrated files once.
- Jest runs with a pinned timezone; the expiry tests pass under any process timezone.
Closes#41
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Distrust compromised eMudhra intermediates in certifi and Deno (#854)
* Distrust compromised eMudhra intermediates in certifi and Deno
The private keys of eight eMudhra intermediate CAs can be recovered from
their certificates, so anyone can mint a trusted certificate for any host.
* Require certifi>=2026.10.19; yt-dlp, requests and curl-cffi use its bundle,
and 2026.10.19 removes the emSign ECC Root CA - G3 root they chain to.
* Set DENO_TLS_CA_STORE=system for the API (systemd + Docker) and the archive
container so Deno follows the apt-patched system store instead of roots
that only change with a Deno upgrade.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Keep Deno's bundled roots in the archive container
browserless/chrome:1.61.1 is Ubuntu 22.04 whose newest ca-certificates is
from 2023, older than Deno 2.8.3's bundled roots. The system store only
becomes the better choice once the archive base image is upgraded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* build(deps): bump dompurify from 3.4.16 to 3.4.17 in /app
Bumps [dompurify](https://github.com/cure53/DOMPurify) from 3.4.16 to 3.4.17.
- [Release notes](https://github.com/cure53/DOMPurify/releases)
- [Commits](https://github.com/cure53/DOMPurify/compare/3.4.16...3.4.17)
---
updated-dependencies:
- dependency-name: dompurify
dependency-version: 3.4.17
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: lrnselfreliance <33263708+lrnselfreliance@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Download and Upload are disabled until a full refresh completes, and a
fresh install never had one until the user found the refresh link.
Startup now queues it, and the dashboard says why Download and Upload
are disabled meanwhile.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Transcode, job, and Kiwix notifications send no URL, but every user notify
toast said "Click here to view the shared page" and opened /null on click.
Only a notification carrying a URL is followable now.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
An unknown URL (e.g. /archive instead of /archives) had no matching route,
so react-router rendered the root errorElement in place of Root. That put
PageNotFound outside ThemeProvider, and its Mantine Box threw
"MantineProvider was not found", leaving a blank page.
- A catch-all route renders PageNotFound inside Root, with the navigation
bar, footer and theme.
- The root errorElement brings its own ThemeProvider, so it can render when
Root itself fails.
- App exports its routes so a test can mount them in a memory router.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* app: a download frequency of "Once" no longer blocks submission
useForm treated any falsy value as a missing required field. "Once" is
frequency 0, so the Channel/Playlist, RSS, and Zim download forms never
became ready and the Download button stayed disabled. Only null,
undefined, '', false, and NaN count as missing now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* downloads: editing to "Once" stores a once-download, not frequency 0
Once-downloads have a NULL frequency. A 0 frequency counted as recurring
(frequency != None) but could never be scheduled (not frequency), so an
edited download stayed on the recurring list and could break renewal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Resume EPUBs and comic books where the reader left off
The EPUB viewer (epub.html) accepts `?cfi=` and posts its location and
progress to the page that embeds it on every page turn. Progress comes from
the chapter and the page within it, rather than book.locations.generate(),
which stalls a Raspberry Pi on a large book. A search result's `loc` still
wins over the saved position.
The comic viewer opens at the saved page and saves each page shown.
Both resume on the doc page and in the file preview. The resume point is
decided once per opened file, so saves made while reading do not change the
EPUB viewer's URL and reload it. The doc page tracks only the FileGroup's
primary file; another format beside it is not tracked.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* EPUB/comic progress: never save a place the server would discard
The server treats a position at the start (the cover, under 1%) as not
started, and replaces the saved place with progress 0. Both viewers
reported the place they opened on, so opening a finished book un-finished
it, and peeking at the cover erased the reader's page.
- The comic viewer does not save the page it opens on, nor the cover; a
peek at the cover keeps the last page saved.
- The EPUB viewer posts nothing at the start of the book.
- Progress before the end is kept between 1% and 94% (resumableProgress), so
early pages of a long book and the last pages before the end keep their
position. Only the last page (comics) or the end (EPUB) is finished.
- Non-finite EPUB progress is ignored.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Resume videos and audio where the user left off; show progress on cards
Videos and audio resume where the user left off, on the video page and in
the file preview, and keep playing. `?t=` starts at an explicit time
instead.
useFileProgress saves how far the user is through a file: every 15 seconds
while it changes, and a final save (a beacon, which survives the page
closing) on pause, end, hiding the page, leaving it, or switching files.
Each save is also kept in sessionStorage; on open the newer of that and the
server's position wins, so coming straight back resumes exactly even if the
final save is still in flight.
Cards show a progress bar along the bottom of their poster. The dashboard's
Recently Viewed Files gains All / Continue; Continue uses the new
`in_progress` file search filter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Progress review fixes: in_progress totals, latest dashboard request, preview viewed
- The in_progress search filter skips the uncounted fast path and is part of
the totals cache key, so Continue never reports All's (or the library's)
count.
- The dashboard's Recently Viewed hook applies only its latest response, and
clears the list when switching All/Continue until the new files arrive.
- POST /api/files/file returns `viewed`, so a preview can tell a file
finished elsewhere from an older position this tab holds.
- The progress hook tests restore Blob, fetch and sendBeacon.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
GitHub Actions runs every job CircleCI did (#789), so CircleCI only
duplicated them. The README badge now shows the backend tests on master.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Opening a doc on its page never set FileGroup.viewed, so docs were missing
from Recently Viewed. GET /api/docs/:id now marks the doc viewed, with the
same skip_viewed parameter as archives and videos.
The file preview marked a file viewed more than once: opening from a
?preview= URL sent two tracked requests, and every tag change re-fetched the
file with tracking on. The fetches now skip tracking, and the preview marks
each opening viewed once.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Count Channels with any of the Tags in the Other search estimate
The Other tab lists Channels with any of the selected Tags, but its estimate
only counted Channels with the first Tag, so the two disagreed when several
Tags were selected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Search Channels by name in the Other search tab
The Other tab only listed Channels with the selected Tags, so typing a
Channel's name found nothing. Channels now match the search text by name,
and the Tags when there are both. The list and its estimate share one query.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add Domains to the Other search tab
Domains match the search text by name and/or any of the Tags, like Channels.
The list and its estimate share one query.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add Playlists to the Other search tab
Domains and Playlists are Collections, so the Other tab searches both with
the existing /api/collections/search (by kind) instead of a Domain-only
endpoint. The search and the estimate share search_collections_query. The
search now loads each Collection's Downloads up front instead of once per
Collection.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Repos keeps local mirrors of git repositories, cloned over https and kept up to date on a schedule.
- A Repo is a Collection (kind='repo') plus a `repository` row. The new `git` downloader clones it, then updates it
(weekly by default, or Never: the clone is kept as it is and only updated on request).
- Repos are saved in `repos/<tag>/<name>` (the Repos Directory, ignored by default) or a directory the user chooses.
An existing clone can be imported where it is. repos.yaml is the source of truth.
- A GitHub-like page shows each repo's README, files, and history, with a ZIP download. READMEs are searchable
under the global search's Other tab.
- git is hardened against hostile repos and configs: https only, pinned addresses with no private addresses (for the
repo and its submodules), sanitized configs, no hooks or links, and only clones WROLPi made or imported are
updated.
- Tagging a Repo moves its clone as one directory. Repo changes wait for repos.yaml to be imported at startup.
- The file mover groups unindexed files with no primary file as one FileGroup, and never replaces a file at the
destination. A failed tag move restores the Collection's tag. A Collection's directory cannot leave the media
directory.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Preview markdown files formatted, with a raw toggle
Markdown files are typed text/markdown (libmagic reports them as text/plain, or
as a programming language when they hold code) and keep the text indexer, so
they stay deep-searchable. Existing files switch type on their next refresh.
The file preview renders markdown with markdown-it into the page, so it follows
the theme tokens and the night media filter. A Formatted/Raw toggle shows the
source instead, and the choice is remembered. Relative links and images
resolve against the file's directory; URLs that climb out of /media/ are
dropped. Raw HTML in a file is escaped rather than parsed, and the output is
also sanitized with DOMPurify.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Do not load images from other sites in the markdown preview
An image from another site (http:, https:, or protocol-relative //) loaded as
soon as the preview opened, telling that site who was reading the file and
when. Such an image now becomes a link labeled with its alt text, so the
reader chooses whether to follow it. Inside a link (a README badge) it becomes
the outer link's text, since links cannot nest. Media and embedded data:
images still load.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Add a Solar calculator with offline NASA POWER data
Estimates a solar array's typical-year production from a location and the
panels; everything else has defaults under an optional "Fine-tune" section.
The model works from 12 monthly averages: Klein's average day per month, the
Erbs monthly diffuse fraction, Collares-Pereira & Rabl / Liu & Jordan hourly
split, HDKR transposition for fixed, single-axis and dual-axis arrays, then
PVWatts' ASHRAE incidence loss, SAPM cell temperature, DC power, system
losses and optional inverter. "Best for the year" and "Best for winter" sweep
the tilt.
Sunlight, diffuse, albedo and temperature come from NASA POWER's 2001-2020
climatology on a 1 degree grid, packed into app/public/data (3.1 MB, ~1 MB
gzipped) by scripts/build_solar_climatology.py and fetched only when the
calculator opens. Users can replace it with their own monthly values.
Every setting lives in the URL, with defaults left out, so the Share button
shares the whole configuration.
Against PVWatts v8 for a 1 kW fixed array: Phoenix +0.6%, Seattle +8.8%,
Anchorage +8.0% annually; these are recorded as tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Open the Solar calculator from the map, pins and current location
The map's right-click (or long-press) menu gains "Calculate Solar
Performance", and each saved pin a sun button; both open the calculator at
that location. A button beside Latitude fills in the browser's current
location. Locations are rounded to 4 decimals, and longitudes from a map
panned around the world are wrapped back to -180..180.
The map removed lat/lon/z from the URL when it closed, even after navigating
to another page, which stripped the calculator's location from its URL (and
from links shared with the Share button). It now only cleans up while the
URL is still a map page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add off-grid sizing to the Solar calculator
A new "Off-grid sizing" tab sizes a battery system for the darkest month
from a list of devices (watts x hours per day):
- Battery bank for a number of days without sun, for LiFePO4 or lead-acid,
at 12, 24 or 48 V. Outdoor lead-acid is enlarged for the cold; outdoor
lithium warns against charging below 0 C.
- Panels needed, from the worst month's DC production per kW at the chosen
angle, with a button to apply the count. A PWM controller's loss (battery
voltage over the panel's Vmp) raises the count.
- Charge controller current, and the string voltage on the coldest morning
against the controller's limit, since cold panels exceed their rated Voc.
The coldest morning defaults to 25 C below the coldest monthly average.
All inputs live in the URL like the rest of the calculator.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Bump wrolpi-help for the Solar calculator help page
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Credit NASA POWER as it asks, and say how the data was changed
- The calculator gains a "Data attribution" link under the location with
NASA POWER's requested acknowledgment, the data reference (Climatology API
v2.10.0, downloaded 2026/10/04), the CC BY 4.0 license, and how WROLPi
modified the data (rounded to one byte per value, temperature averaged
into 1 degree cells, interpolated between cells), which CC BY 4.0 requires.
- The build script writes the same text to solar-climatology.LICENSE.txt
beside the data, with the API version taken from NASA's responses and the
date the tiles were downloaded. A test keeps the page and file in step.
- Bumps wrolpi-help for the updated attribution in the help page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Solar: fix PWM sizing, Voc sign, unit switching and loading on review
- PWM usable power now uses the string's voltage (Vmp x panels in series),
delivers nothing when the charging voltage is at or above the string's Voc,
and falls off between Vmp and Voc. The "too little voltage" warning covers
PWM as well as MPPT. Previously two panels in series counted double, and a
panel that could not charge a 48 V bank counted as fully usable.
- A Voc temperature coefficient entered without its minus sign is treated as
negative, in the math, the URL and the input, so the cold-morning check
cannot be made too generous.
- Switching Fahrenheit on or off converts the entered temperatures instead
of reinterpreting them.
- While the sunlight data downloads, the off-grid tab shows a loader rather
than asking for a location, and holds the cold-morning verdict until the
site's temperatures are known or one has been entered.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Solar: move the data attribution to the bottom of the page
The full NASA POWER acknowledgment, license and changes now sit in a small
"Data attribution" section at the end of the calculator, always visible,
instead of behind a link under the location. The source line under the
location points to it. Bumps wrolpi-help to match.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Solar: drop the data source line under the location
The attribution at the bottom of the page is enough. The line under the
location now only appears to report the data's state: loading, failed to
load, or replaced by the user's own values. Bumps wrolpi-help for the
refreshed screenshot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
cachetools 6 removed cachetools.func.mru_cache; the Zim search estimate
cache was its only user, so it becomes an lru_cache and the pin moves
to 7.2.0 with the Dependabot ignore removed.
reportWebVitals still destructured the web-vitals 2.x names (getCLS,
getFID, ...) which no longer exist, so it silently did nothing. Use
the on* names, replace FID with INP, return the import promise, and
accept any callable rather than instanceof Function.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Cypress component testing resolves the top-level webpack-dev-server
and only accepts major 5, so the bump crashed cypress.config.js on
every run ("Unexpected major version of webpack-dev-server"). The
react-scripts dev server was unaffected because it uses its own
nested 4.x, which is what the pre-merge check looked at.
Pin webpack-dev-server below 6 in dependabot.yml until Cypress
supports it.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>