2022-08-06 05:22:07 +00:00
|
|
|
[pytest]
|
2024-04-17 11:25:26 -06:00
|
|
|
asyncio_mode = auto
|
2024-02-12 11:52:03 -07:00
|
|
|
testpaths =
|
|
|
|
|
wrolpi
|
|
|
|
|
modules
|
2026-01-09 14:01:25 -07:00
|
|
|
asyncio_default_fixture_loop_scope = function
|
db: serializers read only loaded data (#777)
* db: serializers read only loaded data
Serializing a model must not query. A lazy load there is a hidden query per row (search serialized 24 rows
with 55 statements), and would be a `MissingGreenlet` error under an async Session.
- `wrolpi.db.no_db_access(reason)` and `@serializer` (on every model `__json__`, and around `json_response`):
an engine `before_cursor_execute` listener raises `UnexpectedQuery` during tests, and in production logs once
per serializer and lets the query run. Tests record each trip as well, and the autouse
`no_unexpected_queries` fixture fails a test even when the code caught the error (Video's serializer was
swallowing its per-row loads of the deferred `ffprobe_json`).
- Relationships serializers read load with their parent: `selectin` for FileGroup.tag_files and
Collection.downloads; `joined` for TagFile.tag, Collection.tag, Channel.collection, Video.channel,
Archive.collection, ZimSubscription.download, CollectionItem.zim, and the file_group of Video, Archive, Doc,
Zim and CollectionItem.
- Search and the Video lookups the video page uses load the deferred `ffprobe_json` (`undefer`).
- `wrolpi/test/test_serializers.py`: every serialized model, freshly loaded, serializes without SQL.
Search: 55 -> 5 statements for 24 videos (31 -> 5 for archives), and a few ms faster locally.
Not in this change: guarding the `dict()` serializers and `expire_on_commit=False`, which go together (about 50
serialize-after-commit sites, and 40 tests that read values changed elsewhere after a commit).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* db: serializer loads per query; the guard ignores transaction control
The query guard reported the engine's own BEGIN (emitted through the same event as the first statement on a
Session with no open transaction) instead of the lazy load after it, and in production that BEGIN used up the
one warning per serializer. Transaction-control statements are now ignored.
Mapper-level eager loads reached background code: every Video/Archive load in the modelers (one per file,
inside the write transaction) joined file_group/channel/collection/tag and selectin-loaded the Collection's
Download rows with their yt-dlp info_json. A second refresh of 100 videos took 0.40 s instead of 0.16 s.
Serializer needs are now declared per model (`json_options()`) and added only by the queries whose results
are serialized; Download.info_json is deferred there (no serializer reads it). Mapper defaults keep only
narrow joins nearly every use needs: TagFile.tag, Collection.tag and Channel.collection. A second refresh is
back to master's statement count and time; search still drops from 55 to 3 statements.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 18:28:08 -06:00
|
|
|
markers =
|
|
|
|
|
allow_unexpected_queries: the test trips the serializer query guard on purpose (see wrolpi.db.no_db_access)
|
ci: raise the test timeout to 120s, and run controller tests on pull requests
The first CI run that actually enforced the timeout failed sixteen tests, and not
one of them had hung. Every failure sat at exactly 30.0s, and the successes
directly beneath them tell the story: test_get_refresh_progress passed at 29.9s,
test_create_playlist at 29.8s, test_configs_crud at 29.4s. A cluster of
refresh/worker tests straddles the limit on CircleCI's 4 vCPU box, which runs this
suite in about twice the wall-clock of a dev machine -- the ones that passed did so
by a tenth of a second, which is luck, not headroom. Thirty seconds was never a
viable limit there; it only looked fine locally because nothing local exceeds it.
So the limit is a hang detector, not a performance budget. 120s is roughly four
times the slowest legitimate test observed on CI: still short enough to name a
hung test long before CircleCI kills the job, and long enough to leave
slow-but-working tests alone. A limit that fires on healthy tests only trades one
unreliable signal for another.
Also: controller-tests.yml carried the same `pull_request: branches: [master]`
filter that jest-tests.yml did, so it never ran on a pull request into any other
branch. That is why Controller Tests is missing from the checks on the very pull
request that moved it to Python 3.13 -- the change could not be verified by the run
containing it. Now unfiltered, like jest-tests.
Not verified locally: this machine is under a load average of 45 from unrelated
work, so the suite takes more than four times its usual wall-clock here and any
timing measurement is meaningless. The change is strictly more permissive than
what already passed at 30s, so it cannot introduce a failure, and CI is the
environment in question -- it decides this one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 18:35:59 -06:00
|
|
|
# Do not remove timeout - tests should not run forever.
|
|
|
|
|
#
|
|
|
|
|
# This is a hang detector, not a performance budget. It was 30s, but the plugin was never
|
|
|
|
|
# declared in requirements.txt, so CI installed without it and the limit had never actually been
|
|
|
|
|
# applied there. The first run that enforced it failed sixteen tests, every one of them at
|
2026-10-08 17:19:38 -06:00
|
|
|
# 30.0s and none of them hung: a 4 vCPU CI runner runs this suite in roughly twice the
|
ci: raise the test timeout to 120s, and run controller tests on pull requests
The first CI run that actually enforced the timeout failed sixteen tests, and not
one of them had hung. Every failure sat at exactly 30.0s, and the successes
directly beneath them tell the story: test_get_refresh_progress passed at 29.9s,
test_create_playlist at 29.8s, test_configs_crud at 29.4s. A cluster of
refresh/worker tests straddles the limit on CircleCI's 4 vCPU box, which runs this
suite in about twice the wall-clock of a dev machine -- the ones that passed did so
by a tenth of a second, which is luck, not headroom. Thirty seconds was never a
viable limit there; it only looked fine locally because nothing local exceeds it.
So the limit is a hang detector, not a performance budget. 120s is roughly four
times the slowest legitimate test observed on CI: still short enough to name a
hung test long before CircleCI kills the job, and long enough to leave
slow-but-working tests alone. A limit that fires on healthy tests only trades one
unreliable signal for another.
Also: controller-tests.yml carried the same `pull_request: branches: [master]`
filter that jest-tests.yml did, so it never ran on a pull request into any other
branch. That is why Controller Tests is missing from the checks on the very pull
request that moved it to Python 3.13 -- the change could not be verified by the run
containing it. Now unfiltered, like jest-tests.
Not verified locally: this machine is under a load average of 45 from unrelated
work, so the suite takes more than four times its usual wall-clock here and any
timing measurement is meaningless. The change is strictly more permissive than
what already passed at 30s, so it cannot introduce a failure, and CI is the
environment in question -- it decides this one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 18:35:59 -06:00
|
|
|
# wall-clock of a dev machine, and a cluster of refresh/worker tests sits at 28-30s there --
|
|
|
|
|
# test_get_refresh_progress passed at 29.9s and test_create_playlist at 29.8s, which is luck
|
|
|
|
|
# rather than headroom.
|
|
|
|
|
#
|
|
|
|
|
# 120s is about four times the slowest legitimate test observed on CI, so it still catches a
|
2026-10-08 17:19:38 -06:00
|
|
|
# genuine hang long before the CI job times out, while leaving slow-but-working tests alone.
|
ci: raise the test timeout to 120s, and run controller tests on pull requests
The first CI run that actually enforced the timeout failed sixteen tests, and not
one of them had hung. Every failure sat at exactly 30.0s, and the successes
directly beneath them tell the story: test_get_refresh_progress passed at 29.9s,
test_create_playlist at 29.8s, test_configs_crud at 29.4s. A cluster of
refresh/worker tests straddles the limit on CircleCI's 4 vCPU box, which runs this
suite in about twice the wall-clock of a dev machine -- the ones that passed did so
by a tenth of a second, which is luck, not headroom. Thirty seconds was never a
viable limit there; it only looked fine locally because nothing local exceeds it.
So the limit is a hang detector, not a performance budget. 120s is roughly four
times the slowest legitimate test observed on CI: still short enough to name a
hung test long before CircleCI kills the job, and long enough to leave
slow-but-working tests alone. A limit that fires on healthy tests only trades one
unreliable signal for another.
Also: controller-tests.yml carried the same `pull_request: branches: [master]`
filter that jest-tests.yml did, so it never ran on a pull request into any other
branch. That is why Controller Tests is missing from the checks on the very pull
request that moved it to Python 3.13 -- the change could not be verified by the run
containing it. Now unfiltered, like jest-tests.
Not verified locally: this machine is under a load average of 45 from unrelated
work, so the suite takes more than four times its usual wall-clock here and any
timing measurement is meaningless. The change is strictly more permissive than
what already passed at 30s, so it cannot introduce a failure, and CI is the
environment in question -- it decides this one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 18:35:59 -06:00
|
|
|
# Raise it if CI grows slower; a named timeout failure is the point, and a limit tight enough to
|
|
|
|
|
# fire on healthy tests just trades one unreliable signal for another.
|
|
|
|
|
timeout = 120
|
2026-01-09 14:01:25 -07:00
|
|
|
timeout_method = signal
|