SQLAlchemy 1.3 has been end-of-life since 2021, and alembic was held at 1.10
because newer releases need SQLAlchemy 2. This is a mechanical port; the code
stays synchronous.
- SQLAlchemy[asyncio]==2.0.54, alembic==1.20.0.
- Model annotations use Mapped[...] (2.0 rejects bare annotations on mapped
attributes); declarative_base moves to sqlalchemy.orm.
- Plain SQL strings go through text() or exec_driver_sql; schema_ddl.execute_sql
accepts a SQLAlchemy Connection or a raw sqlite3 connection, so rebuild_fts
works from the baseline migration again.
- Session.get replaces Query.get; session.in_transaction() replaces
session.transaction; cache_ok on MediaPathType and FancyJSON.
- The tag-directory sync query joins through TagFile's relationships instead of
the legacy two-entity outerjoin.
- Section-hint rows use attribute access (2.0 rows are tuples).
- Migrations run on an AUTOCOMMIT connection, matching 1.3, where each
statement committed on its own. 2.0 autobegins, so the unique-stem
migration's read snapshot could not become a writer after its helpers wrote
on another connection ("database is locked"). The begin listener emits no
BEGIN for AUTOCOMMIT connections.
- Newer alembic compares column types: WrolpiKV now describes the TEXT table
its migration created.
- The test database installs its DDL inside engine.begin(); 2.0 connections
no longer autocommit, so the FTS tables were rolled back.
- Dependabot now holds SQLAlchemy at 2.0 instead of 1.3.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
GitHub Actions runs every job CircleCI did (#789), so CircleCI only
duplicated them. The README badge now shows the backend tests on master.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Run the CircleCI jobs on GitHub Actions
Backend tests (pytest, API contract, migrations + API start) and Cypress
(component, e2e) now run as GitHub Actions workflows. CircleCI stays in
place until these have run green alongside it.
- pip and npm caches via setup-python/setup-node, plus the Cypress binary;
node_modules is no longer cached, to stay inside the 10 GB cache cap.
- Superseded pull request runs are cancelled to stay under the concurrent
job limit.
- react-app-start is dropped: the e2e job waits for `npm start`, and
jest-tests.yml already runs `npm run build`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Skip the CI-unsupported tests on GitHub Actions too
skip_circleci only read CIRCLECI, so on GitHub Actions test_archive_upload
ran without the readability tooling it needs, and failed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Run the backend tests on every vCPU
-nauto counts physical cores when psutil is installed, so the four-vCPU
runner got two workers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Split the backend tests into four jobs by directory
The groups are balanced by test time, so each runs about a quarter of the
suite. `rest` ignores the other groups' paths rather than listing its own,
so a new directory is tested before it is assigned to a group.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
cachetools 6 removed cachetools.func.mru_cache; the Zim search estimate
cache was its only user, so it becomes an lru_cache and the pin moves
to 7.2.0 with the Dependabot ignore removed.
reportWebVitals still destructured the web-vitals 2.x names (getCLS,
getFID, ...) which no longer exist, so it silently did nothing. Use
the on* names, replace FID with INP, return the import promise, and
accept any callable rather than instanceof Function.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Cypress component testing resolves the top-level webpack-dev-server
and only accepts major 5, so the bump crashed cypress.config.js on
every run ("Unexpected major version of webpack-dev-server"). The
react-scripts dev server was unaffected because it uses its own
nested 4.x, which is what the pre-merge check looked at.
Pin webpack-dev-server below 6 in dependabot.yml until Cypress
supports it.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Each regenerated after the last config change and each needs its own
migration: cypress 16 needs Node 22, react-router 8 touches ~40 files,
and @testing-library/react 16 fails 15 menu/popover tests.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Comment-based ignores did not survive the last config change, and the
SQLAlchemy rule only blocked majors while 1.4 (the 2.0 transition
release) slipped into the python group. Hold SQLAlchemy <1.4, alembic
<1.11, cachetools majors (mru_cache removed in 6), and the npm majors
that each need a migration branch: maplibre-gl 6, react-dropzone 15+,
React 19, TypeScript 6+, Mantine 9.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
alembic 1.20 requires SQLAlchemy>=2.0, so every grouped python
update that included it failed to install (ResolutionImpossible).
Ignore alembic >=1.11 and SQLAlchemy major bumps until the
SQLAlchemy 2 migration is done.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The first CI run that actually enforced the timeout failed sixteen tests, and not
one of them had hung. Every failure sat at exactly 30.0s, and the successes
directly beneath them tell the story: test_get_refresh_progress passed at 29.9s,
test_create_playlist at 29.8s, test_configs_crud at 29.4s. A cluster of
refresh/worker tests straddles the limit on CircleCI's 4 vCPU box, which runs this
suite in about twice the wall-clock of a dev machine -- the ones that passed did so
by a tenth of a second, which is luck, not headroom. Thirty seconds was never a
viable limit there; it only looked fine locally because nothing local exceeds it.
So the limit is a hang detector, not a performance budget. 120s is roughly four
times the slowest legitimate test observed on CI: still short enough to name a
hung test long before CircleCI kills the job, and long enough to leave
slow-but-working tests alone. A limit that fires on healthy tests only trades one
unreliable signal for another.
Also: controller-tests.yml carried the same `pull_request: branches: [master]`
filter that jest-tests.yml did, so it never ran on a pull request into any other
branch. That is why Controller Tests is missing from the checks on the very pull
request that moved it to Python 3.13 -- the change could not be verified by the run
containing it. Now unfiltered, like jest-tests.
Not verified locally: this machine is under a load average of 45 from unrelated
work, so the suite takes more than four times its usual wall-clock here and any
timing measurement is meaningless. The change is strictly more permissive than
what already passed at 30s, so it cannot introduce a failure, and CI is the
environment in question -- it decides this one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI has been testing 3.11 while nothing ships it. docker/api/Dockerfile and
docker/help/Dockerfile are python:3.13-trixie, docker/controller is
python:3.13-slim-trixie, Trixie's system interpreter is 3.13, and Debian 12 --
the last place 3.11 was current -- is pinned to the debian12-final tag and takes
no new code. So every Python job here exercised an interpreter no user has.
All four CircleCI Python jobs and the GitHub Actions controller job move to 3.13.
The cache key goes with them: it stores the whole venv, so restoring a
deps-3.11 venv into a 3.13 container would hand it an interpreter that no longer
exists.
The disabled api-tests-3-12 job is deleted rather than repointed. It was
switched off with the note "Sanic does not yet support 3.12", which 3.13 disproves,
and carrying a second version around is what let this drift open in the first
place.
Verified on 3.13.12 / pytest 9.0.3 before changing anything: the backend suite
passes (1709 passed, 5 skipped), alembic upgrade runs and `db check` reports no
new operations, and the controller suite passes bar two docker-mode dashboard
tests that fail identically on 3.11 here -- pre-existing and environment-dependent,
not a 3.13 regression.
`sanic-api-start` is the one job I could not verify locally: wrolpi/api_utils.py
sets `Sanic.start_method = "fork"` and macOS has defaulted to spawn since 3.8, so
that job fails here on any Python version. Linux CI is unaffected -- fork is
still the default there on 3.13 -- and the test suite already runs the Sanic app
in-process, which is the substantive check. Worth knowing that 3.14 changes the
Linux default away from fork, so that line has a deadline on it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two gaps, one of them wide.
The contract tests added in c349e15c already ran inside `pytest -nauto`, but
fixture drift is silent by nature -- the frontend suite passes against a body the
API stopped sending -- so it deserves a check whose name says what broke.
`api-contract-tests` restores the same dependency cache as the other Python jobs,
skips the ffmpeg/catdoc install it does not need (verified), and runs the one
file, so it answers in under a minute instead of somewhere inside seventeen
hundred results.
The wide one: jest-tests.yml triggered on `pull_request: branches: [master]`, so
a pull request into anything else ran neither jest nor the webpack build. The
entire Semantic UI to Mantine migration went in that way -- seventy-five files
across nine pull requests into feature/custom-theme, not one of which ran either.
The build step is the one that matters most: jest and tsc both pass on a named
import that does not exist and only webpack rejects it, which is exactly how a
broken Archive.js survived a "clean" run earlier in this migration. The suite
did run on the branch after each merge, which is the wrong side of the merge to
learn about it.
`pull_request` is now unfiltered rather than listing branches, so the next
long-lived branch does not have to remember to add itself. It stays
`pull_request`, not `pull_request_target`: a fork's run gets a read-only token
and no secrets, and this workflow interpolates no event data, so widening the
trigger adds no injection or exfiltration path.
Worth knowing separately: none of this gates anything. master has no branch
protection and the only ruleset covers `release` (deletion, non-fast-forward), so
every check here is advisory. That is a repository setting, and Roland's to make.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(app): depend on @testing-library/dom directly
jest-dom 7 moved @testing-library/dom from a bundled dependency to a peer
dependency. Nothing else pulls the top-level copy in — @testing-library/react
13 carries its own nested v8 — so it was present only as a peer, and CI
installs with --legacy-peer-deps, which skips peers. Every suite then failed
to load setupTests.js with "Cannot find module '@testing-library/dom'".
Declare it as a devDependency so it installs under either resolution mode.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(app): import mount from cypress/react
Cypress 14 removed the cypress/react18 subpath; cypress/react mounts React
18 itself. The e2e support file loads commands.js, so the stale import broke
bundling for every spec, not just component tests:
Module not found: Package path ./react18 is not exported from package cypress
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: install and cache Cypress the way the bump requires
The Cypress binary lives in ~/.cache/Cypress, outside node_modules, but only
node_modules was cached. A restored cache satisfied npm, so npm skipped the
postinstall download and the 15.x binary was never fetched:
The cypress npm package is installed, but the Cypress binary is missing.
Cache both paths and install the binary explicitly.
Also align the JS toolchain with what actually ships. Debian 13 devices run
nodejs 20.19.2 and install with a plain `npm install`; the Docker app image
does the same. CI pinned older Node and passed --legacy-peer-deps, which
skips peer dependencies -- that is why jest-dom 7's peer requirement passed
review and broke CI. Match the devices instead.
Pin caddy to 2.11.4, the version Debian 13 devices run, rather than latest.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: test on Python 3.13, the version that ships
Debian 13 devices run Python 3.13.5 and the api/controller/help images are
python:3.13-trixie, but CI ran 3.11 (and one job on 3.12), so the runtime
under test was two minors behind the runtime in production.
Verified on 3.13 before switching: backend 1712 passed, 5 skipped; controller
883 passed. The only new warning is SQLAlchemy 1.3.22 reaching Python's
deprecated datetime.utcfromtimestamp(), which is noise rather than a break.
Drop api-tests-3-12: no workflow referenced it, it duplicated api-tests, and
its "Sanic does not yet support 3.12" note is stale now that devices run 3.13.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The extension is hosted in a sibling repo and
is distributed off-store. Each WROLPi release ships the prebuilt artifacts
under wrolpi/static/extensions/ and serves them at /api/extensions/<file>:
- wrolpi-chrome.zip Load-Unpacked sideload for Chrome / Brave / Edge
- wrolpi-firefox.xpi Mozilla-signed (unlisted self-distribution),
one-click install in any Firefox
- versions.json Per-browser version metadata for the install page
Backend (modules/archive/api.py, wrolpi/root_api.py):
- POST /api/archive/upload accepts an optional repeated `tagNames`
multipart form field; the upload switch handler applies tags to the
new FileGroup via FileGroup.set_tags(). Unknown tag names are auto-
created inside a per-name SAVEPOINT so concurrent inserts on the same
novel name don't abort the transaction (Tag.name has unique=True).
- GET /api/extensions returns metadata (per-file availability + size +
versions.json contents) for the install page.
- GET /api/extensions/<filename> serves the binary, whitelisted by
filename. Firefox .xpi is served as application/x-xpinstall WITHOUT a
Content-Disposition: attachment header, so Firefox triggers its
add-on install handler instead of saving the file.
Frontend (app/src/):
- New /admin/extension page (linked from AdminRoute) detects the user's
browser via navigator.userAgent and renders an install card per
Firefox / Chromium / Other. Pulls metadata from /api/extensions to
show file size and version, and gracefully reports 'Not yet
available' when binaries are missing.
- Dashboard's Getters component supports a deep link from the
extension's 'Configure Download' button:
/?downloader=<archive|video|...>&download_url=<u1>&download_url=<u2>
On mount, the matching downloader form auto-opens with URLs pre-
filled. Query params are cleared on close so a refresh doesn't
re-open the modal.
- DownloadMenu and the per-downloader forms accept an `initialUrls`
seed via the existing `download` prop pattern, joined with newlines
for the `urls` textarea (and first URL for single-URL forms).
Behavior is fully backwards compatible: pre-existing API callers can
still POST to /api/archive/upload without `tagNames`, and the
DashboardPage modal continues to open via the existing button click
when no deep-link query params are present.
