Commit graph

2515 commits

Author SHA1 Message Date
Roland
1a28a5900a
repos: git inherits only an allow-listed environment
git_env copied the whole process environment, and git consults its
environment ahead of `-c`: GIT_ALLOW_PROTOCOL=file re-enabled file://
despite protocol.allow=never, and GIT_CONFIG_COUNT/KEY_*/VALUE_* could
add an `insteadOf` that rewrites an approved URL to another host.  Only
PATH, HOME, TMPDIR, TZ and the SSL certificate paths are inherited now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 17:32:59 -06:00
Roland
1c214f4033
tests: bookmark config import tests set up the app's shared context
They read `successful_import`, which lives in the shared context that
`async_client` attaches.  They only passed when an earlier test in the
same xdist worker had attached it; run first, all 13 failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 17:31:36 -06:00
Roland
8b846c57eb
help: document the Repos module
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:24 -06:00
Roland
2dd8e908c2
repos: check submodule URLs where git resolves them
A relative submodule URL is resolved by git against the repo's remote,
and can climb to another host: with the remote
https://git.example.com/owner/repo, `../../../10.0.0.5/x` is
https://10.0.0.5/x.  Relative URLs were allowed unchecked.

git now resolves the URLs first (`submodule init` + `sync` write them to
the repo's config), and those resolved URLs, the ones git fetches, are
checked.  A relative URL is never allowed by the check itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:10 -06:00
Roland
4810b8bc50
repos: close the remaining submodule SSRF gaps
* .gitmodules is read NUL-delimited, and an unreadable record refuses
  the submodules.  A submodule name with a space (`[submodule "a b"]`)
  made `--get-regexp`'s "key value" output misfile its URL as a path, so
  the URL was never checked.
* Submodule URLs must have a strict shape every parser reads the same
  way (no userinfo, backslash, query, fragment, or whitespace), so a URL
  cannot be checked as one host and fetched from another.
* Each checked host is pinned (http.curloptResolve) to the addresses
  that were checked, so DNS cannot change between the check and git's
  fetch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:10 -06:00
Roland
3b6e962dfc
repos: never fetch submodules from WROLPi's own network
A repo's .gitmodules is chosen by whoever controls the repo, so a
submodule URL could make WROLPi fetch https://localhost:8443/...,
https://10.0.0.1/..., etc. (SSRF), and show the response's error.

Every submodule URL is now checked before git fetches it: relative URLs,
the repo's own host, and hosts whose addresses are all public are
allowed; anything else defers the download with the reason.  Nested
submodules are checked out one level at a time (instead of git's
--recursive) so each level is checked, at most 5 levels deep.  Submodule
fetches do not follow HTTP redirects, so a checked host cannot redirect
git into the network.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00
Roland
03e3476631
repos: history, Download ZIP, and submodules
* GET /api/repos/<id>/log: the checked out branch's commits, newest
  first, a page at a time, with the total.  The repo page has a
  Files | History toggle (the tree is only listed for Files).
* GET /api/repos/<id>/archive.zip: `git archive HEAD` streamed to the
  browser; the file name is reduced to safe characters.  Everything is
  read from the database before the response starts, because the
  request's session closes once a streamed response begins.
* repository.submodules (default off): submodules are checked out with
  the repo's mode and the same git restrictions, and removed when the
  option is turned off.  A submodule that cannot be downloaded (e.g. an
  ssh URL) does not stop the repo from updating; the download is
  retried with the submodule's error.
* The README speed test bound is loose enough for a busy machine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00
Roland
0d1a988fa6
repos: add the Repos UI
* Repos table (CollectionTable), New Repo and Edit pages (frequency,
  mode, branch, description, tag, delete with an optional "also delete
  its files").
* A GitHub-like repo page: upstream, branch, latest commit, a status
  message when not yet downloaded or the last update failed, the file
  tree (directories through ?path=, files in the file preview), and each
  directory's README.
* Global search: a Repos section under Other with escaped README
  headlines, Repos in the suggestions, and the search text sent to the
  Other estimate.
* Settings: the Repos Directory field.
* CollectionTable columns may `render` their cell.
* Theme gallery: the repo summary, tree, and status icons.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00
Roland
5506dc0a88
repos: harden README search against untrusted READMEs
A README comes from an untrusted remote.

* Escape the README headline: snippet() marks matches with control
  characters, the text is HTML-escaped, then the markers become <b>.
  Headlines are rendered as HTML (HeadlineText).
* Never follow a symlinked README (O_NOFOLLOW, regular files only).
* Convert markdown line by line with linear-time patterns.  The old
  multiline patterns backtracked polynomially; a README of blank lines
  never finished, and runs of `<!--`, `![` or `<` were quadratic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00
Roland
5c80ad7ba4
repos: search Repos by name and README
repository_fts (FTS5, external content) indexes each Repo's search_name
(name and owner) and readme_text; a name match outranks a README
mention.  The downloader stores the README as plain text (markdown
syntax removed so snippets read well), writing it only when it changed.

* POST /api/repos/search: by search and/or Tag, with README headlines.
* /api/search_other_estimates takes an optional search_str and returns
  repo_count, so Repos count toward the global search's Other tab.
* /api/search_suggestions suggests Repos by name.

The repository FTS DDL is installed only once the repository table
exists: the baseline migration also runs install_raw_ddl, before that
table is created.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00
Roland
f49288b8ec
repos: validate branch names and repo directories
Branch names are passed to git as arguments and come from the API, the
config, a `/tree/<branch>` URL, or the remote itself (its default
branch).  A name starting with `-` would be read as an option, so every
branch must pass git's ref-name rules, including in the downloader just
before use.

A repo URL with a `.` or `..` path segment (or a `..` name) could format
a directory outside its template (e.g. `repos/<tag>/..` is the whole
repos directory).  Such URLs and names are rejected, and every repo
directory, including one from repos.yaml, must be strictly inside the
media directory with no `..`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00
Roland
b3255e715c
Add a Repos module that mirrors git repositories (backend)
A Repo is a Collection (kind='repo') plus a `repository` row, kept up to
date by a recurring Download handled by the new `git` downloader.

* Full history by default (all branches and tags, no --prune; a force
  push saves the old HEAD under refs/wrolpi/backup/); snapshot mode
  (--depth 1) and a pinned branch are optional.  `/tree/<branch>` URLs
  select a branch.
* git runs HTTPS-only, never prompts, never runs hooks, and checks out
  symlinks as plain files.  Upstream disappearing defers the download
  with git's error; the local copy is kept.
* Repos are cloned to `repos_destination` (repos/%(repo_tag)s/%(repo_name)s)
  and the `repos` directory is always ignored, so repo files are never
  indexed.
* repos.yaml is the source of truth (import, dump, backup restore).
* /api/repos: list, create, get, edit, delete (files kept unless
  delete_files), update now, and a per-directory file tree.  Tagging uses
  the collections tag endpoint.

Also:
* run_command/process_runner can merge stderr into stdout (git writes
  progress to stderr).
* move_collection deletes an old directory that only holds empty
  directories (the move left an emptied .git behind).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00
lrnselfreliance
b742a4a25f
Keep the API's logging when migrating at startup (#782)
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
WROLPi runs alembic migrations in-process at startup (db_bootstrap).
alembic/env.py called fileConfig unconditionally, which replaced the
root handlers, set the root level to WARN, and disabled every logger
that already existed, so the API's logs went quiet after any startup
migration until the next restart.  env.py now configures logging only
when alembic runs from its command line (cmd_opts is set).

This also caused an order-dependent test failure: test_jobs's capture
test failed whenever test_db_bootstrap ran first in the same worker.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:55:53 -06:00
lrnselfreliance
2b2361ed64
Save playlists config when a playlist's file is renamed or moved (#781)
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
playlists.yaml references file items by path. Renaming, moving, or
reorganizing a FileGroup updated the database but not the config, so the
next import (startup or global refresh) could not find the file and
silently dropped the item from the playlist.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 15:03:45 -06:00
lrnselfreliance
04779b8557
Preview markdown files formatted, with a raw toggle (#780)
* Preview markdown files formatted, with a raw toggle

Markdown files are typed text/markdown (libmagic reports them as text/plain, or
as a programming language when they hold code) and keep the text indexer, so
they stay deep-searchable.  Existing files switch type on their next refresh.

The file preview renders markdown with markdown-it into the page, so it follows
the theme tokens and the night media filter.  A Formatted/Raw toggle shows the
source instead, and the choice is remembered.  Relative links and images
resolve against the file's directory; URLs that climb out of /media/ are
dropped.  Raw HTML in a file is escaped rather than parsed, and the output is
also sanitized with DOMPurify.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Do not load images from other sites in the markdown preview

An image from another site (http:, https:, or protocol-relative //) loaded as
soon as the preview opened, telling that site who was reading the file and
when.  Such an image now becomes a link labeled with its alt text, so the
reader chooses whether to follow it.  Inside a link (a README badge) it becomes
the outer link's text, since links cannot nest.  Media and embedded data:
images still load.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 13:26:42 -06:00
lrnselfreliance
9739dc9e1b
tests: fix the midnight and event-loop timing flakes (#779)
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
test_import_config_preserves_completed_once_downloads created a download "5 minutes ago" and expected it to
count toward today's limit, which starts at local midnight; in the first five minutes after local midnight it
was yesterday's.  It failed in CI started at 00:00:35 UTC, and reproduces on master with the clock at 00:02.
The test now pins the clock to local noon, as the download manager reckons local time.

test_import_all_db_configs_does_not_block_event_loop required 90% of the expected 50 ms ticks, so timer
overshoot on a busy runner added up across the run (CI got 28 of the 28.8 required).  It now measures the
longest gap between ticks, including after the last tick: an import on the loop freezes it for the whole 0.2 s
sleep, while jitter delays a tick by milliseconds.  Moving the first or the last import onto the loop still
fails it.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 18:43:00 -06:00
lrnselfreliance
cc8afa5370
Add a Solar calculator (#778)
* Add a Solar calculator with offline NASA POWER data

Estimates a solar array's typical-year production from a location and the
panels; everything else has defaults under an optional "Fine-tune" section.

The model works from 12 monthly averages: Klein's average day per month, the
Erbs monthly diffuse fraction, Collares-Pereira & Rabl / Liu & Jordan hourly
split, HDKR transposition for fixed, single-axis and dual-axis arrays, then
PVWatts' ASHRAE incidence loss, SAPM cell temperature, DC power, system
losses and optional inverter.  "Best for the year" and "Best for winter" sweep
the tilt.

Sunlight, diffuse, albedo and temperature come from NASA POWER's 2001-2020
climatology on a 1 degree grid, packed into app/public/data (3.1 MB, ~1 MB
gzipped) by scripts/build_solar_climatology.py and fetched only when the
calculator opens.  Users can replace it with their own monthly values.

Every setting lives in the URL, with defaults left out, so the Share button
shares the whole configuration.

Against PVWatts v8 for a 1 kW fixed array: Phoenix +0.6%, Seattle +8.8%,
Anchorage +8.0% annually; these are recorded as tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Open the Solar calculator from the map, pins and current location

The map's right-click (or long-press) menu gains "Calculate Solar
Performance", and each saved pin a sun button; both open the calculator at
that location.  A button beside Latitude fills in the browser's current
location.  Locations are rounded to 4 decimals, and longitudes from a map
panned around the world are wrapped back to -180..180.

The map removed lat/lon/z from the URL when it closed, even after navigating
to another page, which stripped the calculator's location from its URL (and
from links shared with the Share button).  It now only cleans up while the
URL is still a map page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add off-grid sizing to the Solar calculator

A new "Off-grid sizing" tab sizes a battery system for the darkest month
from a list of devices (watts x hours per day):

- Battery bank for a number of days without sun, for LiFePO4 or lead-acid,
  at 12, 24 or 48 V.  Outdoor lead-acid is enlarged for the cold; outdoor
  lithium warns against charging below 0 C.
- Panels needed, from the worst month's DC production per kW at the chosen
  angle, with a button to apply the count.  A PWM controller's loss (battery
  voltage over the panel's Vmp) raises the count.
- Charge controller current, and the string voltage on the coldest morning
  against the controller's limit, since cold panels exceed their rated Voc.
  The coldest morning defaults to 25 C below the coldest monthly average.

All inputs live in the URL like the rest of the calculator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Bump wrolpi-help for the Solar calculator help page

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Credit NASA POWER as it asks, and say how the data was changed

- The calculator gains a "Data attribution" link under the location with
  NASA POWER's requested acknowledgment, the data reference (Climatology API
  v2.10.0, downloaded 2026/10/04), the CC BY 4.0 license, and how WROLPi
  modified the data (rounded to one byte per value, temperature averaged
  into 1 degree cells, interpolated between cells), which CC BY 4.0 requires.
- The build script writes the same text to solar-climatology.LICENSE.txt
  beside the data, with the API version taken from NASA's responses and the
  date the tiles were downloaded.  A test keeps the page and file in step.
- Bumps wrolpi-help for the updated attribution in the help page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Solar: fix PWM sizing, Voc sign, unit switching and loading on review

- PWM usable power now uses the string's voltage (Vmp x panels in series),
  delivers nothing when the charging voltage is at or above the string's Voc,
  and falls off between Vmp and Voc.  The "too little voltage" warning covers
  PWM as well as MPPT.  Previously two panels in series counted double, and a
  panel that could not charge a 48 V bank counted as fully usable.
- A Voc temperature coefficient entered without its minus sign is treated as
  negative, in the math, the URL and the input, so the cold-morning check
  cannot be made too generous.
- Switching Fahrenheit on or off converts the entered temperatures instead
  of reinterpreting them.
- While the sunlight data downloads, the off-grid tab shows a loader rather
  than asking for a location, and holds the cold-morning verdict until the
  site's temperatures are known or one has been entered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Solar: move the data attribution to the bottom of the page

The full NASA POWER acknowledgment, license and changes now sit in a small
"Data attribution" section at the end of the calculator, always visible,
instead of behind a link under the location.  The source line under the
location points to it.  Bumps wrolpi-help to match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Solar: drop the data source line under the location

The attribution at the bottom of the page is enough.  The line under the
location now only appears to report the data's state: loading, failed to
load, or replaced by the user's own values.  Bumps wrolpi-help for the
refreshed screenshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 18:36:45 -06:00
lrnselfreliance
cfbed9e4d7
db: serializers read only loaded data (#777)
* db: serializers read only loaded data

Serializing a model must not query.  A lazy load there is a hidden query per row (search serialized 24 rows
with 55 statements), and would be a `MissingGreenlet` error under an async Session.

- `wrolpi.db.no_db_access(reason)` and `@serializer` (on every model `__json__`, and around `json_response`):
  an engine `before_cursor_execute` listener raises `UnexpectedQuery` during tests, and in production logs once
  per serializer and lets the query run.  Tests record each trip as well, and the autouse
  `no_unexpected_queries` fixture fails a test even when the code caught the error (Video's serializer was
  swallowing its per-row loads of the deferred `ffprobe_json`).
- Relationships serializers read load with their parent: `selectin` for FileGroup.tag_files and
  Collection.downloads; `joined` for TagFile.tag, Collection.tag, Channel.collection, Video.channel,
  Archive.collection, ZimSubscription.download, CollectionItem.zim, and the file_group of Video, Archive, Doc,
  Zim and CollectionItem.
- Search and the Video lookups the video page uses load the deferred `ffprobe_json` (`undefer`).
- `wrolpi/test/test_serializers.py`: every serialized model, freshly loaded, serializes without SQL.

Search: 55 -> 5 statements for 24 videos (31 -> 5 for archives), and a few ms faster locally.

Not in this change: guarding the `dict()` serializers and `expire_on_commit=False`, which go together (about 50
serialize-after-commit sites, and 40 tests that read values changed elsewhere after a commit).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* db: serializer loads per query; the guard ignores transaction control

The query guard reported the engine's own BEGIN (emitted through the same event as the first statement on a
Session with no open transaction) instead of the lazy load after it, and in production that BEGIN used up the
one warning per serializer.  Transaction-control statements are now ignored.

Mapper-level eager loads reached background code: every Video/Archive load in the modelers (one per file,
inside the write transaction) joined file_group/channel/collection/tag and selectin-loaded the Collection's
Download rows with their yt-dlp info_json.  A second refresh of 100 videos took 0.40 s instead of 0.16 s.

Serializer needs are now declared per model (`json_options()`) and added only by the queries whose results
are serialized; Download.info_json is deferred there (no serializer reads it).  Mapper defaults keep only
narrow joins nearly every use needs: TagFile.tag, Collection.tag and Channel.collection.  A second refresh is
back to master's statement count and time; search still drops from 55 to 3 statements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 18:28:08 -06:00
lrnselfreliance
1f687fa825
Move entry tags to the current Zim when deleting outdated Zims (#776)
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
Deleting a Zim cascade-deletes its TagZimEntry rows, and the next tags
save writes them out of the config, so every outdated-Zim delete lost
its entry tags.  The config-import migration only runs at startup, too
late to help.  Move each outdated Zim's tags to the current Zim of the
same name before deleting, then save the tags config.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 18:04:11 -06:00
lrnselfreliance
8713a6136b
Reset file worker status when a synchronous refresh is cancelled (#775)
* Reset file worker status when a synchronous refresh is cancelled

refresh_sync runs inside request handlers.  When Sanic cancelled one on
response timeout, the phase flag cleared as its `with` block unwound but
the shared status did not, so the worker reported 'modeling' forever while
nothing ran.  Reset to idle on cancellation.

Deleting outdated Zims also ran the global modelers inside the request,
which is what timed out.  Skip post-processing there and update the
outdated_zims flag directly, the only cleanup step it relied on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* tests: observe the modeling flag in the cancelled refresh_sync test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:25:25 -06:00
lrnselfreliance
bed25b429d
db: request-path library functions use the request's connection (#774)
* db: request-path library functions use the request's connection

Library functions an API request reaches now take the request's Session (first argument) instead of
opening their own session or cursor, so a request needs one SQLite connection.  Raw SQL runs through the
new `wrolpi.db.session_curs(session)`, a cursor on the Session's connection; `RequestDB.curs()` uses it.

- Search: files, videos, archives, flasher, and docs search share the request connection for the total,
  the page of ids, and the page's models (3 connections -> 1).
- Zim search, including tag-filtered search and estimates (2 -> 1).
- Channel detail (`Channel.get_statistics` reads on the Channel's own Session) and the channel list.
- Video detail, edit, description, captions, comments and transcode.  `get_video_for_app` and
  `update_video` no longer commit; their handlers use `request.ctx.db.write()`, so editing a Video and
  reading it back is one transaction.
- Tags (list, overlaps, recent), file statistics and estimates, other estimates, bulk tag preview, the
  download list, retry, kill and the status summary, and docs/videos/archive statistics.

Not converted: file operations that refresh files inside the request (delete, rename, upload, channel
and collection tag/move) commit page by page in code shared with the file worker; the cached zim entry
lookup; config import.

The connection-count tests now assert one connection for every endpoint they cover (no xfails left), and
cover the videos and archive statistics endpoints, which had no API test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* downloads: startup retries in a write session of its own

start_single_tasks still called retry_downloads() without a Session after it began requiring one, so any
start after a completed refresh raised TypeError there: incomplete downloads were not reset and the
download_on_startup enable after it never ran.  Startup is not a request, so
DownloadManager.retry_downloads_on_startup() opens and commits its own short write transaction.

Tested through the method, not start_single_tasks: importing main registers its startup listeners on the
shared app, and every later test in that worker then runs them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:05:20 -06:00
lrnselfreliance
a2bef48c6c
db: one request-scoped database handle per API request (#773)
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
* db: one request-scoped database handle per API request

Every API request now gets `request.ctx.db` (`wrolpi.db.RequestDB`): a Session plus raw cursors on that
Session's connection, so a request needs at most one SQLite connection.  Creating it does no I/O; requests
that never touch the database never connect.  `request.ctx.session` remains the handle's Session.

A `RequestSession` binds itself to one lazily-opened Connection.  An engine-bound Session releases its
connection at every commit, and NullPool closes it, so a request that read and then wrote reconnected.

Handlers write in `request.ctx.db.write()`: BEGIN IMMEDIATE, committed when the block exits, so the
write lock is not held for the rest of the handler or by background work it starts.  This replaces the
handler-level `get_db_session(commit=True)` blocks (download create, collection item add/remove/reorder,
channel create).  The zim search handlers use the request's Session in their worker thread.

Tests count the connections a request opens with production's per-call sessions
(`count_request_connections`).  Endpoints whose library functions still open their own connection are
strict xfails, to be removed as those functions take the request's session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* tests: name the real extra connections in the request connection xfails

Channel detail: get_channel already uses the request session; the extra connection is
Channel.get_statistics opening a raw cursor (get_db_curs).  Search: count_file_groups opens a
cursor for the total, then handle_file_group_search_results opens a cursor and a session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 15:42:06 -06:00
Roland
d7deea668d
Bump version to 0.29.0-beta
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 11:53:21 -06:00
lrnselfreliance
1c8bbf1f10
deps: cachetools 7 and the current web-vitals API (#771)
cachetools 6 removed cachetools.func.mru_cache; the Zim search estimate
cache was its only user, so it becomes an lru_cache and the pin moves
to 7.2.0 with the Dependabot ignore removed.

reportWebVitals still destructured the web-vitals 2.x names (getCLS,
getFID, ...) which no longer exist, so it silently did nothing.  Use
the on* names, replace FID with INP, return the import promise, and
accept any callable rather than instanceof Function.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 11:04:16 -06:00
lrnselfreliance
c28a9e0078
Revert "build(deps-dev): bump webpack-dev-server from 5.2.6 to 6.0.0 in /app (#769)" (#772)
Cypress component testing resolves the top-level webpack-dev-server
and only accepts major 5, so the bump crashed cypress.config.js on
every run ("Unexpected major version of webpack-dev-server").  The
react-scripts dev server was unaffected because it uses its own
nested 4.x, which is what the pre-merge check looked at.

Pin webpack-dev-server below 6 in dependabot.yml until Cypress
supports it.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 11:00:27 -06:00
lrnselfreliance
f37098cb81
tests: fix the two suites failing on master (#770)
test_handle_reorganize_marks_job_failed_on_exception patched
wrolpi.files.worker.shutil.move, but #719 moved handle_reorganize and
its shutil.move call into wrolpi.files.reorganize_job.

test_file_worker_refresh_inserts_new_files loaded FileGroups into the
shared pytest session between refreshes.  The refresh upserts by raw
SQL, so the modeler then read the stale identity-map object and wrote
its old file list back over the upsert.  Production opens a fresh
session per block; expire the test session before the second refresh.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 10:50:24 -06:00
dependabot[bot]
8610f9c97e
build(deps): bump react-router in /app in the npm-dependencies group (#768)
Bumps the npm-dependencies group in /app with 1 update: [react-router](https://github.com/remix-run/react-router/tree/HEAD/packages/react-router).


Updates `react-router` from 7.18.2 to 7.18.4
- [Release notes](https://github.com/remix-run/react-router/releases)
- [Changelog](https://github.com/remix-run/react-router/blob/react-router@7.18.4/packages/react-router/CHANGELOG.md)
- [Commits](https://github.com/remix-run/react-router/commits/react-router@7.18.4/packages/react-router)

---
updated-dependencies:
- dependency-name: react-router
  dependency-version: 7.18.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: npm-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:44:22 -06:00
dependabot[bot]
6b79323d2c
build(deps-dev): bump webpack-dev-server from 5.2.6 to 6.0.0 in /app (#769)
Bumps [webpack-dev-server](https://github.com/webpack/webpack-dev-server) from 5.2.6 to 6.0.0.
- [Release notes](https://github.com/webpack/webpack-dev-server/releases)
- [Changelog](https://github.com/webpack/webpack-dev-server/blob/main/CHANGELOG.md)
- [Commits](https://github.com/webpack/webpack-dev-server/compare/v5.2.6...v6.0.0)

---
updated-dependencies:
- dependency-name: webpack-dev-server
  dependency-version: 6.0.0
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:43:52 -06:00
lrnselfreliance
51d9c07784
dependabot: also hold cypress 16, react-router 8 and testing-library/react majors (#767)
Each regenerated after the last config change and each needs its own
migration: cypress 16 needs Node 22, react-router 8 touches ~40 files,
and @testing-library/react 16 fails 15 menu/popover tests.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 10:34:52 -06:00
dependabot[bot]
9fc9929647
build(deps): bump web-vitals from 5.3.0 to 6.2.2 in /app (#764)
Bumps [web-vitals](https://github.com/GoogleChrome/web-vitals) from 5.3.0 to 6.2.2.
- [Changelog](https://github.com/GoogleChrome/web-vitals/blob/main/CHANGELOG.md)
- [Commits](https://github.com/GoogleChrome/web-vitals/compare/v5.3.0...v6.2.2)

---
updated-dependencies:
- dependency-name: web-vitals
  dependency-version: 6.2.2
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:33:23 -06:00
dependabot[bot]
39cef45c59
build(deps): bump the npm-dependencies group in /app with 3 updates (#761)
Bumps the npm-dependencies group in /app with 3 updates: [react](https://github.com/react/react/tree/HEAD/packages/react), [react-dom](https://github.com/react/react/tree/HEAD/packages/react-dom) and [react-dropzone](https://github.com/react-dropzone/react-dropzone).


Updates `react` from 18.2.0 to 18.3.1
- [Release notes](https://github.com/react/react/releases)
- [Changelog](https://github.com/react/react/blob/main/CHANGELOG.md)
- [Commits](https://github.com/react/react/commits/v18.3.1/packages/react)

Updates `react-dom` from 18.2.0 to 18.3.1
- [Release notes](https://github.com/react/react/releases)
- [Changelog](https://github.com/react/react/blob/main/CHANGELOG.md)
- [Commits](https://github.com/react/react/commits/v18.3.1/packages/react-dom)

Updates `react-dropzone` from 14.2.3 to 14.4.1
- [Release notes](https://github.com/react-dropzone/react-dropzone/releases)
- [Commits](https://github.com/react-dropzone/react-dropzone/compare/v14.2.3...v14.4.1)

---
updated-dependencies:
- dependency-name: react
  dependency-version: 18.3.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: npm-dependencies
- dependency-name: react-dom
  dependency-version: 18.3.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: npm-dependencies
- dependency-name: react-dropzone
  dependency-version: 14.4.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: npm-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:33:11 -06:00
dependabot[bot]
920b619bcc
build(deps): bump the python-dependencies group with 12 updates (#763)
Bumps the python-dependencies group with 12 updates:

| Package | From | To |
| --- | --- | --- |
| [sqlalchemy](https://github.com/sqlalchemy/sqlalchemy) | `1.3.22` | `1.3.24` |
| [beautifulsoup4](https://www.crummy.com/software/BeautifulSoup/bs4/) | `4.14.3` | `4.15.0` |
| [cachetools](https://github.com/tkem/cachetools) | `5.5.0` | `5.5.2` |
| [cryptography](https://github.com/pyca/cryptography) | `50.0.0` | `50.0.2` |
| [feedparser](https://github.com/kurtmckee/feedparser) | `6.0.12` | `6.0.14` |
| [jc](https://github.com/kellyjonbrazil/jc) | `1.25.6` | `1.26.0` |
| [libzim](https://github.com/openzim/python-libzim) | `3.10.0` | `3.13.0` |
| [pytest](https://github.com/pytest-dev/pytest) | `9.0.3` | `9.1.1` |
| [python-dotenv](https://github.com/theskumar/python-dotenv) | `1.2.2` | `1.2.3` |
| [pytz](https://github.com/stub42/pytz) | `2026.2` | `2026.4` |
| [selenium](https://github.com/SeleniumHQ/Selenium) | `4.44.0` | `4.49.0` |
| [tqdm](https://github.com/tqdm/tqdm) | `4.67.3` | `4.70.1` |


Updates `sqlalchemy` from 1.3.22 to 1.3.24
- [Release notes](https://github.com/sqlalchemy/sqlalchemy/releases)
- [Changelog](https://github.com/sqlalchemy/sqlalchemy/blob/main/CHANGES.rst)
- [Commits](https://github.com/sqlalchemy/sqlalchemy/commits)

Updates `beautifulsoup4` from 4.14.3 to 4.15.0

Updates `cachetools` from 5.5.0 to 5.5.2
- [Changelog](https://github.com/tkem/cachetools/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/tkem/cachetools/compare/v5.5.0...v5.5.2)

Updates `cryptography` from 50.0.0 to 50.0.2
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/50.0.0...50.0.2)

Updates `feedparser` from 6.0.12 to 6.0.14
- [Release notes](https://github.com/kurtmckee/feedparser/releases)
- [Changelog](https://github.com/kurtmckee/feedparser/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/kurtmckee/feedparser/compare/v6.0.12...v6.0.14)

Updates `jc` from 1.25.6 to 1.26.0
- [Release notes](https://github.com/kellyjonbrazil/jc/releases)
- [Changelog](https://github.com/kellyjonbrazil/jc/blob/master/CHANGELOG)
- [Commits](https://github.com/kellyjonbrazil/jc/compare/v1.25.6...v1.26.0)

Updates `libzim` from 3.10.0 to 3.13.0
- [Release notes](https://github.com/openzim/python-libzim/releases)
- [Changelog](https://github.com/openzim/python-libzim/blob/main/CHANGELOG.md)
- [Commits](https://github.com/openzim/python-libzim/compare/v3.10.0...v3.13.0)

Updates `pytest` from 9.0.3 to 9.1.1
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.3...9.1.1)

Updates `python-dotenv` from 1.2.2 to 1.2.3
- [Release notes](https://github.com/theskumar/python-dotenv/releases)
- [Changelog](https://github.com/theskumar/python-dotenv/blob/main/CHANGELOG.md)
- [Commits](https://github.com/theskumar/python-dotenv/compare/v1.2.2...v1.2.3)

Updates `pytz` from 2026.2 to 2026.4
- [Release notes](https://github.com/stub42/pytz/releases)
- [Commits](https://github.com/stub42/pytz/compare/release_2026.2...release_2026.4)

Updates `selenium` from 4.44.0 to 4.49.0
- [Release notes](https://github.com/SeleniumHQ/Selenium/releases)
- [Commits](https://github.com/SeleniumHQ/Selenium/compare/selenium-4.44.0...selenium-4.49.0)

Updates `tqdm` from 4.67.3 to 4.70.1
- [Release notes](https://github.com/tqdm/tqdm/releases)
- [Commits](https://github.com/tqdm/tqdm/compare/v4.67.3...v4.70.1)

---
updated-dependencies:
- dependency-name: sqlalchemy
  dependency-version: 1.3.24
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-dependencies
- dependency-name: beautifulsoup4
  dependency-version: 4.15.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-dependencies
- dependency-name: cachetools
  dependency-version: 5.5.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-dependencies
- dependency-name: cryptography
  dependency-version: 50.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-dependencies
- dependency-name: feedparser
  dependency-version: 6.0.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-dependencies
- dependency-name: jc
  dependency-version: 1.26.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-dependencies
- dependency-name: libzim
  dependency-version: 3.13.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-dependencies
- dependency-name: pytest
  dependency-version: 9.1.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-dependencies
- dependency-name: python-dotenv
  dependency-version: 1.2.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-dependencies
- dependency-name: pytz
  dependency-version: '2026.4'
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-dependencies
- dependency-name: selenium
  dependency-version: 4.49.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-dependencies
- dependency-name: tqdm
  dependency-version: 4.70.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:33:05 -06:00
dependabot[bot]
a7103002b6
build(deps): bump nanoid from 3.3.16 to 3.3.19 in /app (#759)
Bumps [nanoid](https://github.com/ai/nanoid) from 3.3.16 to 3.3.19.
- [Release notes](https://github.com/ai/nanoid/releases)
- [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ai/nanoid/compare/3.3.16...3.3.19)

---
updated-dependencies:
- dependency-name: nanoid
  dependency-version: 3.3.19
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:22:20 -06:00
dependabot[bot]
5e76914485
build(deps): bump qs and express in /app (#758)
Bumps [qs](https://github.com/ljharb/qs) and [express](https://github.com/expressjs/express). These dependencies needed to be updated together.

Updates `qs` from 6.15.3 to 6.16.0
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.15.3...v6.16.0)

Updates `express` from 4.22.2 to 4.22.3
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/v4.22.3/History.md)
- [Commits](https://github.com/expressjs/express/compare/v4.22.2...v4.22.3)

---
updated-dependencies:
- dependency-name: qs
  dependency-version: 6.16.0
  dependency-type: indirect
- dependency-name: express
  dependency-version: 4.22.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:22:15 -06:00
lrnselfreliance
0d2431c41a
dependabot: move major-version exclusions into the config (#760)
Comment-based ignores did not survive the last config change, and the
SQLAlchemy rule only blocked majors while 1.4 (the 2.0 transition
release) slipped into the python group. Hold SQLAlchemy <1.4, alembic
<1.11, cachetools majors (mru_cache removed in 6), and the npm majors
that each need a migration branch: maplibre-gl 6, react-dropzone 15+,
React 19, TypeScript 6+, Mantine 9.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 10:21:50 -06:00
dependabot[bot]
3d0eb76337
build(deps): update pymupdf requirement from >=1.27.2.3 to >=1.28.2 (#756)
Updates the requirements on [pymupdf](https://github.com/pymupdf/pymupdf) to permit the latest version.
- [Release notes](https://github.com/pymupdf/pymupdf/releases)
- [Changelog](https://github.com/pymupdf/PyMuPDF/blob/main/changes.txt)
- [Commits](https://github.com/pymupdf/pymupdf/compare/1.27.2.3...1.28.2)

---
updated-dependencies:
- dependency-name: pymupdf
  dependency-version: 1.28.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:21:14 -06:00
dependabot[bot]
3800f4d332
build(deps): update fastapi requirement in /controller (#748)
Updates the requirements on [fastapi](https://github.com/fastapi/fastapi) to permit the latest version.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](https://github.com/fastapi/fastapi/compare/0.141.1...0.142.2)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-version: 0.142.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:19:52 -06:00
dependabot[bot]
323b7e66df
build(deps): bump fast-uri from 3.1.7 to 3.1.8 in /app (#746)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.7 to 3.1.8.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.7...v3.1.8)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.8
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:19:48 -06:00
dependabot[bot]
9eb56967e4
build(deps): bump brace-expansion in /app (#745)
Bumps  and [brace-expansion](https://github.com/juliangruber/brace-expansion). These dependencies needed to be updated together.

Updates `brace-expansion` from 1.1.18 to 1.1.21
- [Release notes](https://github.com/juliangruber/brace-expansion/releases)
- [Commits](https://github.com/juliangruber/brace-expansion/compare/v1.1.18...v1.1.21)

Updates `brace-expansion` from 2.1.4 to 2.1.7
- [Release notes](https://github.com/juliangruber/brace-expansion/releases)
- [Commits](https://github.com/juliangruber/brace-expansion/compare/v1.1.18...v1.1.21)

---
updated-dependencies:
- dependency-name: brace-expansion
  dependency-version: 1.1.21
  dependency-type: indirect
- dependency-name: brace-expansion
  dependency-version: 2.1.7
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:19:43 -06:00
lrnselfreliance
ecd8ce64be
dependabot: hold alembic below 1.11 while SQLAlchemy stays on 1.3 (#747)
alembic 1.20 requires SQLAlchemy>=2.0, so every grouped python
update that included it failed to install (ResolutionImpossible).
Ignore alembic >=1.11 and SQLAlchemy major bumps until the
SQLAlchemy 2 migration is done.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 10:11:08 -06:00
dependabot[bot]
71287c9a23
build(deps): bump the npm-dependencies group in /app with 7 updates (#741)
Bumps the npm-dependencies group in /app with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [@tabler/icons-react](https://github.com/tabler/tabler-icons/tree/HEAD/packages/icons-react) | `3.46.0` | `3.48.0` |
| [@testing-library/user-event](https://github.com/testing-library/user-event) | `14.6.6` | `14.6.7` |
| [esptool-js](https://github.com/espressif/esptool-js) | `0.6.1` | `0.7.0` |
| [react-colorful](https://github.com/omgovich/react-colorful) | `5.8.0` | `5.8.1` |
| [three](https://github.com/mrdoob/three.js) | `0.185.1` | `0.186.1` |
| [@testing-library/dom](https://github.com/testing-library/dom-testing-library) | `10.4.1` | `10.4.2` |
| [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) | `26.4.0` | `26.6.3` |


Updates `@tabler/icons-react` from 3.46.0 to 3.48.0
- [Release notes](https://github.com/tabler/tabler-icons/releases)
- [Commits](https://github.com/tabler/tabler-icons/commits/v3.48.0/packages/icons-react)

Updates `@testing-library/user-event` from 14.6.6 to 14.6.7
- [Release notes](https://github.com/testing-library/user-event/releases)
- [Changelog](https://github.com/testing-library/user-event/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testing-library/user-event/compare/v14.6.6...v14.6.7)

Updates `esptool-js` from 0.6.1 to 0.7.0
- [Release notes](https://github.com/espressif/esptool-js/releases)
- [Changelog](https://github.com/espressif/esptool-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/espressif/esptool-js/compare/v0.6.1...v0.7.0)

Updates `react-colorful` from 5.8.0 to 5.8.1
- [Release notes](https://github.com/omgovich/react-colorful/releases)
- [Changelog](https://github.com/omgovich/react-colorful/blob/master/CHANGELOG.md)
- [Commits](https://github.com/omgovich/react-colorful/commits)

Updates `three` from 0.185.1 to 0.186.1
- [Release notes](https://github.com/mrdoob/three.js/releases)
- [Commits](https://github.com/mrdoob/three.js/commits)

Updates `@testing-library/dom` from 10.4.1 to 10.4.2
- [Release notes](https://github.com/testing-library/dom-testing-library/releases)
- [Changelog](https://github.com/testing-library/dom-testing-library/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testing-library/dom-testing-library/compare/v10.4.1...v10.4.2)

Updates `@types/node` from 26.4.0 to 26.6.3
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@tabler/icons-react"
  dependency-version: 3.48.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: npm-dependencies
- dependency-name: "@testing-library/user-event"
  dependency-version: 14.6.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: npm-dependencies
- dependency-name: esptool-js
  dependency-version: 0.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: npm-dependencies
- dependency-name: react-colorful
  dependency-version: 5.8.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: npm-dependencies
- dependency-name: three
  dependency-version: 0.186.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: npm-dependencies
- dependency-name: "@testing-library/dom"
  dependency-version: 10.4.2
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm-dependencies
- dependency-name: "@types/node"
  dependency-version: 26.6.3
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: npm-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:08:32 -06:00
dependabot[bot]
0837330bd7
build(deps): bump colord from 2.9.3 to 2.10.0 in /app (#703)
Bumps [colord](https://github.com/omgovich/colord) from 2.9.3 to 2.10.0.
- [Release notes](https://github.com/omgovich/colord/releases)
- [Changelog](https://github.com/omgovich/colord/blob/master/CHANGELOG.md)
- [Commits](https://github.com/omgovich/colord/commits/v2.10)

---
updated-dependencies:
- dependency-name: colord
  dependency-version: 2.10.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:07:26 -06:00
dependabot[bot]
5439fff871
build(deps): bump js-yaml in /app (#705)
Bumps  and [js-yaml](https://github.com/nodeca/js-yaml). These dependencies needed to be updated together.

Updates `js-yaml` from 3.15.0 to 3.15.2
- [Changelog](https://github.com/nodeca/js-yaml/blob/3.15.2/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/3.15.0...3.15.2)

Updates `js-yaml` from 4.3.0 to 4.3.2
- [Changelog](https://github.com/nodeca/js-yaml/blob/3.15.2/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/3.15.0...3.15.2)

---
updated-dependencies:
- dependency-name: js-yaml
  dependency-version: 3.15.2
  dependency-type: indirect
- dependency-name: js-yaml
  dependency-version: 4.3.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:07:22 -06:00
dependabot[bot]
c973a4106f
build(deps): bump sanic from 24.12.0 to 24.12.1 in /docker/archive (#714)
Bumps [sanic](https://github.com/sanic-org/sanic) from 24.12.0 to 24.12.1.
- [Release notes](https://github.com/sanic-org/sanic/releases)
- [Commits](https://github.com/sanic-org/sanic/compare/v24.12.0...v24.12.1)

---
updated-dependencies:
- dependency-name: sanic
  dependency-version: 24.12.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:07:17 -06:00
dependabot[bot]
dd7269d1ca
build(deps): update uvicorn requirement in /controller (#742)
Updates the requirements on [uvicorn](https://github.com/Kludex/uvicorn) to permit the latest version.
- [Release notes](https://github.com/Kludex/uvicorn/releases)
- [Changelog](https://github.com/Kludex/uvicorn/blob/main/docs/release-notes.md)
- [Commits](https://github.com/Kludex/uvicorn/compare/0.52.4...0.54.0)

---
updated-dependencies:
- dependency-name: uvicorn
  dependency-version: 0.54.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-03 10:07:13 -06:00
lrnselfreliance
57e647c5b4
docker: web container reaps zombies; zim and help healthchecks can pass (#744)
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
Caddy is PID 1 in the web container and reaps nothing.  The healthcheck's
busybox `wget --spider https://...` forks an `ssl_client` helper that is
orphaned to caddy and stays a zombie, one per check.  `init: true` runs
tini as PID 1 so they are reaped.

The zim healthcheck calls curl, which the debian:13 kiwix image does not
install, so every check exited 127.  The help healthcheck probed port
8086, which is Caddy's HTTPS proxy on the web container; mkdocs serves
on port 80 inside its own container.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 15:49:34 -06:00
lrnselfreliance
8d6f5b5cad
api: forked workers die on SIGTERM before Sanic installs their handlers (#743)
A Sanic worker is forked from the manager and carries the manager's
Python SIGTERM/SIGINT handlers until `_setup_system_signals` installs its
own, which happens after the startup listeners (about two seconds with
sixteen workers starting at once).  When auto-reload fires a second time
inside that window the manager terminates the half-started generation,
each child runs `WorkerManager.shutdown_signal` instead of exiting, marks
itself TERMINATED, and then finishes starting.  The manager has already
spawned replacements, so the old generation lives on: untracked, holding
the listening socket, serving stale code, and holding ~220 MB each.

Register an at-fork hook in the main process that resets both signals to
their default action in the child.  The perpetual process already did
this by hand for itself; server workers now get the same treatment.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 15:49:06 -06:00
lrnselfreliance
5c78aa332d
collections: listing answers channel and download queries from covering indexes (#740)
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
The listing still read every channel row (for the summary columns) and every download
row of the listed Collections (for the minimum frequency). Both tables store yt-dlp
info_json, so rows are tens of KB and each sits on its own scattered page; selecting
fewer columns did not help because the seek per row is the cost. From a cold disk
cache that was 5 s per table, 10+ s total, on a large library.

Two covering indexes, channel (collection_id, video_count, total_size) and
download (collection_id, frequency), answer both queries without touching a row. The
listing no longer loads Download objects at all; it computes min_download_frequency
with one grouped query and drops the unused `downloads` array from listing rows (the
detail endpoints keep it; the list views only read min_download_frequency).

Adds a models-vs-migrations test (alembic check) so an index or column added to a
model without a migration, or vice versa, fails the suite.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-27 12:53:40 -06:00
lrnselfreliance
20be5dfc27
collections: channel and domain listings read trigger-maintained summary columns (#739)
* collections: channel listing reads trigger columns and batch-loads tag/downloads

Listing channel collections summed FileGroup sizes through a video join, which reads a
random page of the file_group table per video, and lazy-loaded every collection's tag
and downloads one query at a time. On a large library with a cold disk cache the
listing took over two minutes, past the browser's 60 s timeout, so the Channels page
showed an error.

channel.video_count and channel.total_size are already maintained by triggers
(wrolpi/schema_ddl.py), so read those. Tag and downloads are selectin-loaded, so the
query count no longer grows with the number of collections.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* collections: domain listing reads trigger-maintained summary columns

Domain Collections summed file_group.size through an archive join on every listing,
one random page read per archive (about 9 s from a cold disk cache on a large
library). New triggers keep collection.item_count and collection.total_size current
for Domain Collections, as the channel triggers already do for Channels, and the
listing reads those columns. A migration installs the triggers and backfills
existing Domain Collections.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-27 11:36:08 -06:00
lrnselfreliance
4fea50943b
map: clicking a POI shows its kind, detail, elevation and coordinates (#738)
The Protomaps basemap keeps a few attributes per POI beyond the name the
map already labels: kind, kind_detail and elevation.  Clicking a POI now
opens a popup with those, plus the feature's coordinates (or the click
location when the feature has no point).  The same POI appears at several
min_zoom levels within one tile, so features are deduped by name and kind.

Map popups (this one and the existing pin popup) now follow the active
theme's tokens instead of MapLibre's fixed white, covering light, dark,
night and amber.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-27 08:23:03 -06:00