2023-03-24 08:58:18 -06:00
|
|
|
import asyncio
|
Postgres -> SQLite hard cutover: the database moves into the media directory.
The database now lives at <media>/config/wrolpi.db so a library (files,
configs, AND database) travels with the drive between WROLPi systems.
Core:
- wrolpi/db.py: lazy SQLite engine factory (WAL, busy_timeout=30s,
foreign_keys=ON); get_db_curs yields a plain sqlite3.Cursor; native
sqlite3 paramstyle everywhere (no compatibility shims); global sqlite3
adapters keep raw-SQL datetime formats identical to the ORM's.
- wrolpi/fts.py: FTS5 external-content search replacing tsvector/GIN.
Injection-proof websearch->FTS5 translator, bm25(10,4,2,1) reproducing
A/B/C/D weighting, fast(abc)/deep column filtering, snippet headlines.
Stemming contract pinned by dedicated tests (watch/watches/watched/
watching are interchangeable).
- wrolpi/schema_ddl.py: SQLite triggers shared verbatim by the alembic
baseline and test DBs; fixes the channel video_count DELETE/UPDATE
bugs and a latent total_size bug (it used the video id as channel id).
- wrolpi/db_bootstrap.py: startup auto-create/migrate; refuses databases
written by a newer WROLPi (db_version_mismatch); SQLite 3.40 floor;
network-filesystem refusal; never creates/migrates/trusts a database
beneath an unmounted production media directory (shadow-DB guard).
- Alembic: 85 Postgres-era migrations squashed into one baseline;
revision ids are dates (2026_07_10_1350) so filenames self-sort and
alembic_version says when a schema was created; render_as_batch=True.
- All raw Postgres SQL rewritten (ANY->json_each IN, array_agg @> ->
HAVING COUNT(DISTINCT), unnest->executemany/UPDATE..FROM,
generate_series histograms->strftime GROUP BY, DISTINCT ON->
row_number(), writable CTEs->two statements, jsonb_*->json_*, explicit
NULLS FIRST/LAST, LIMIT-before-OFFSET). Simple statements converted
to SQLAlchemy instead (flag upsert, download retry/summary/tag checks,
channel-by-tag estimate, Directory delete).
- Config imports count skipped entries; background dumps are blocked
after a lossy import so partial imports can never overwrite YAML
configs (they are the sole source of truth for rebuilds now).
- Search tolerates explicit JSON nulls for limit/offset (Postgres
treated OFFSET NULL as 0; SQLite errors).
- file_group.id/doc_section.id render as INTEGER PRIMARY KEY on SQLite
(BIGINT is not a rowid alias; FTS5 content_rowid requires it).
Ops teardown: docker-compose db service, CI postgres sidecars,
initialize_api_db.sh, controller postgresql service, psycopg2-binary all
removed. upgrade.sh stops/disables postgresql (data is NOT deleted;
`systemctl enable --now postgresql` restores it); systemd units no
longer Want postgresql.service.
Existing-user upgrade: first boot creates+migrates the DB, configs
import (tags recreate their FileGroups from disk), a refresh rebuilds
the library index. QA'd on the 10.0.0.8 RPi: rebuild matched the
Postgres baseline exactly; survived an uncontrolled hard reboot with
clean integrity checks.
Accepted behavior changes: FTS5 has no stopwords; bm25 length
normalization shifts some rank orders; fast-path d_headline is
un-highlighted; all-negative queries match nothing.
Deferred: Debian/trixie package lists (incl. sqlite3 CLI), Cypress
re-baseline, VACUUM INTO snapshots, exFAT/NTFS warning, auto-kicked
rebuild refresh with progress UX.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 13:59:20 -06:00
|
|
|
import json
|
review: ffprobe out of the write transaction, and three tests that were watching nothing
All three bugs confirmed; the two suggestions taken.
[bug] `video_modeler` awaited ffprobe -- a subprocess per video, up to 20
per batch -- inside its `commit=True` session. With IMMEDIATE as the
default that holds the write lock for the whole batch, which is the same
starvation this branch is curing in `video_cleanup`. (It was already
partly true before: the batch's first `session.flush([video])` took the
lock ahead of the remaining ffprobes.) The batch is now read, ffprobed
with no transaction open, then written. `get_or_create_ffprobe_json` in
modules/videos/common.py is the path-only entry point that makes that
possible; `Video.get_ffprobe_json` delegates to it and keeps appending
the cache file to its FileGroup. `validate()` -- and the poster
generation it can trigger -- is still inside the write transaction, and
now says so.
[bug] `test_download_dispatched_outside_immediate_transaction` probed
`getattr(_immediate_txn, 'active', False)`, which a ContextVar never has,
so it passed unconditionally. It now probes the actual write lock from a
competing connection under `production_like_sessions`. Mutation-checked:
moving the dispatch back inside the claim transaction fails it.
[bug] `asyncio.create_task` snapshots the current Context, so a flag held
for a write session's lifetime was inherited by every task spawned inside
it -- and the parent's `reset()` cannot reach a child's copy, leaving
that task issuing BEGIN IMMEDIATE for its *read* sessions.
`create_downloads` spawns `dispatch_downloads` exactly that way, so every
user-created download hit it. The flag now lives only across the
session's opening `connection()` call, which also means the write lock is
taken when the session opens rather than at the caller's first statement.
[suggestion] `renew_recurring_downloads` scanned every recurring Download
under the write lock, including the usual cycle where nothing is due. It
reads first now and takes the lock only for the rows that change,
re-checking status since the dispatcher may have claimed one in between.
[nit] Parenthesized the directory OR in the claim join.
`probe_write_lock_is_held` moved to conftest alongside
`write_lock_held_briefly`; three suites use it now.
Suite: 1712 passed, 5 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 20:04:40 -06:00
|
|
|
import pathlib
|
2026-01-31 12:54:27 -07:00
|
|
|
from typing import Callable, List, Tuple
|
2022-08-06 05:22:07 +00:00
|
|
|
|
2026-04-14 15:00:49 -06:00
|
|
|
from sqlalchemy import or_
|
2024-09-15 15:33:07 -06:00
|
|
|
from sqlalchemy.orm import Session
|
|
|
|
|
|
review: ffprobe out of the write transaction, and three tests that were watching nothing
All three bugs confirmed; the two suggestions taken.
[bug] `video_modeler` awaited ffprobe -- a subprocess per video, up to 20
per batch -- inside its `commit=True` session. With IMMEDIATE as the
default that holds the write lock for the whole batch, which is the same
starvation this branch is curing in `video_cleanup`. (It was already
partly true before: the batch's first `session.flush([video])` took the
lock ahead of the remaining ffprobes.) The batch is now read, ffprobed
with no transaction open, then written. `get_or_create_ffprobe_json` in
modules/videos/common.py is the path-only entry point that makes that
possible; `Video.get_ffprobe_json` delegates to it and keeps appending
the cache file to its FileGroup. `validate()` -- and the poster
generation it can trigger -- is still inside the write transaction, and
now says so.
[bug] `test_download_dispatched_outside_immediate_transaction` probed
`getattr(_immediate_txn, 'active', False)`, which a ContextVar never has,
so it passed unconditionally. It now probes the actual write lock from a
competing connection under `production_like_sessions`. Mutation-checked:
moving the dispatch back inside the claim transaction fails it.
[bug] `asyncio.create_task` snapshots the current Context, so a flag held
for a write session's lifetime was inherited by every task spawned inside
it -- and the parent's `reset()` cannot reach a child's copy, leaving
that task issuing BEGIN IMMEDIATE for its *read* sessions.
`create_downloads` spawns `dispatch_downloads` exactly that way, so every
user-created download hit it. The flag now lives only across the
session's opening `connection()` call, which also means the write lock is
taken when the session opens rather than at the caller's first statement.
[suggestion] `renew_recurring_downloads` scanned every recurring Download
under the write lock, including the usual cycle where nothing is due. It
reads first now and takes the lock only for the rows that change,
re-checking status since the dispatcher may have claimed one in between.
[nit] Parenthesized the directory OR in the claim join.
`probe_write_lock_is_held` moved to conftest alongside
`write_lock_held_briefly`; three suites use it now.
Suite: 1712 passed, 5 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 20:04:40 -06:00
|
|
|
from modules.videos.common import get_or_create_ffprobe_json
|
2026-08-23 15:18:15 -06:00
|
|
|
from modules.videos.models import Video, AUDIO_PLAYLIST_MIMETYPES
|
2023-03-27 20:46:31 -06:00
|
|
|
from wrolpi.common import logger, limit_concurrent, register_modeler, register_refresh_cleanup
|
2023-03-24 08:58:18 -06:00
|
|
|
from wrolpi.db import get_db_curs, get_db_session
|
|
|
|
|
from wrolpi.files.models import FileGroup
|
2022-08-06 05:22:07 +00:00
|
|
|
from wrolpi.vars import PYTEST
|
|
|
|
|
from .downloader import video_downloader # Import downloaders so they are registered.
|
|
|
|
|
|
|
|
|
|
logger = logger.getChild(__name__)
|
|
|
|
|
|
2023-03-24 08:58:18 -06:00
|
|
|
__all__ = ['video_modeler']
|
2022-08-06 05:22:07 +00:00
|
|
|
|
2023-03-24 08:58:18 -06:00
|
|
|
VIDEO_PROCESSING_LIMIT = 20
|
2022-12-14 18:52:42 -07:00
|
|
|
|
|
|
|
|
|
2026-08-23 15:18:15 -06:00
|
|
|
def _model_video(session: Session, file_group: FileGroup, video: Video | None, probed: dict) -> Video:
|
|
|
|
|
"""Create/update and validate the Video row for one FileGroup."""
|
|
|
|
|
if not video:
|
|
|
|
|
video = Video(file_group=file_group, file_group_id=file_group.id)
|
|
|
|
|
session.add(video)
|
|
|
|
|
session.flush([video])
|
|
|
|
|
if not Session.object_session(video):
|
|
|
|
|
session.add(video)
|
|
|
|
|
video.flush()
|
|
|
|
|
# Store the ffprobe data gathered outside the write transaction.
|
|
|
|
|
if (result := probed.get(file_group.id)) is not None:
|
|
|
|
|
video.ffprobe_json, ffprobe_file = result
|
|
|
|
|
if ffprobe_file:
|
|
|
|
|
# Track the .ffprobe.json cache file that was just written.
|
|
|
|
|
file_group.append_files(ffprobe_file)
|
|
|
|
|
video.flush(session)
|
|
|
|
|
# Validate and index subtitles. (Poster generation happens here when a Channel
|
|
|
|
|
# asks for it; it is the remaining slow work inside this transaction.)
|
|
|
|
|
video.validate(session)
|
|
|
|
|
return video
|
|
|
|
|
|
|
|
|
|
|
2022-08-06 05:22:07 +00:00
|
|
|
@register_modeler
|
2026-01-31 12:54:27 -07:00
|
|
|
async def video_modeler(progress_callback: Callable[[int], None] = None):
|
|
|
|
|
total_processed = 0
|
2026-08-23 15:18:15 -06:00
|
|
|
# Ids that failed to model this run. A failure leaves no Video row, so the `Video.id IS
|
|
|
|
|
# NULL` gate below would re-select it forever; failures are retried on the next refresh.
|
|
|
|
|
failed_ids: set = set()
|
2023-03-24 08:58:18 -06:00
|
|
|
while True:
|
review: ffprobe out of the write transaction, and three tests that were watching nothing
All three bugs confirmed; the two suggestions taken.
[bug] `video_modeler` awaited ffprobe -- a subprocess per video, up to 20
per batch -- inside its `commit=True` session. With IMMEDIATE as the
default that holds the write lock for the whole batch, which is the same
starvation this branch is curing in `video_cleanup`. (It was already
partly true before: the batch's first `session.flush([video])` took the
lock ahead of the remaining ffprobes.) The batch is now read, ffprobed
with no transaction open, then written. `get_or_create_ffprobe_json` in
modules/videos/common.py is the path-only entry point that makes that
possible; `Video.get_ffprobe_json` delegates to it and keeps appending
the cache file to its FileGroup. `validate()` -- and the poster
generation it can trigger -- is still inside the write transaction, and
now says so.
[bug] `test_download_dispatched_outside_immediate_transaction` probed
`getattr(_immediate_txn, 'active', False)`, which a ContextVar never has,
so it passed unconditionally. It now probes the actual write lock from a
competing connection under `production_like_sessions`. Mutation-checked:
moving the dispatch back inside the claim transaction fails it.
[bug] `asyncio.create_task` snapshots the current Context, so a flag held
for a write session's lifetime was inherited by every task spawned inside
it -- and the parent's `reset()` cannot reach a child's copy, leaving
that task issuing BEGIN IMMEDIATE for its *read* sessions.
`create_downloads` spawns `dispatch_downloads` exactly that way, so every
user-created download hit it. The flag now lives only across the
session's opening `connection()` call, which also means the write lock is
taken when the session opens rather than at the caller's first statement.
[suggestion] `renew_recurring_downloads` scanned every recurring Download
under the write lock, including the usual cycle where nothing is due. It
reads first now and takes the lock only for the rows that change,
re-checking status since the dispatcher may have claimed one in between.
[nit] Parenthesized the directory OR in the claim join.
`probe_write_lock_is_held` moved to conftest alongside
`write_lock_held_briefly`; three suites use it now.
Suite: 1712 passed, 5 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 20:04:40 -06:00
|
|
|
# Read the batch; nothing is claimed yet, so the write lock stays free while ffprobe runs.
|
|
|
|
|
with get_db_session() as session:
|
2026-08-23 15:18:15 -06:00
|
|
|
query = session.query(FileGroup.id, FileGroup.primary_path) \
|
|
|
|
|
.outerjoin(Video, Video.file_group_id == FileGroup.id) \
|
|
|
|
|
.filter(
|
2026-04-14 15:00:49 -06:00
|
|
|
or_(FileGroup.mimetype.like('video/%'), FileGroup.mimetype.like('audio/%')),
|
2026-08-23 15:18:15 -06:00
|
|
|
FileGroup.mimetype.notin_(AUDIO_PLAYLIST_MIMETYPES),
|
files: SQL FileGroup compare, stem column, drop unused discovery path (#713)
* files: SQL FileGroup compare, stem column, drop unused discovery path
Refresh walked the tree twice (find, then stat every unchanged group),
and a two-directory refresh scanned the whole media directory. Compare
now diffs in SQL on a TEMP table that already has mtime, accepts multiple
roots, and groups by a stored stem that is versioned so a suffix-rule
change cannot delete-and-reinsert tagged files.
Deletes refresh_discover_paths and /api/files/refresh_progress, which
nothing in production called. Video/doc/zim modelers use the same
(no domain row OR indexed != True) gate.
* files: keep stem current on reorganize; NUL-safe find parse
Reorganize wrote the new primary_path but not stem, so the next refresh
treated the FileGroup as deleted+new and dropped tags. GNU find now
emits NUL-delimited records so a tab or newline in a filename cannot
abort a global refresh. Stem backfill skips the table scan once the
algorithm version matches, and runs off the event loop. An empty roots
list is a global refresh, not invalid SQL.
2026-09-18 08:57:56 -06:00
|
|
|
# Exclusive-mimetype modeler: claim unmodeled rows even if apply_indexers
|
|
|
|
|
# already set indexed=True, and re-model when files changed (indexed != True).
|
2026-08-23 15:18:15 -06:00
|
|
|
or_(Video.id.is_(None), FileGroup.indexed != True),
|
|
|
|
|
)
|
|
|
|
|
if failed_ids:
|
|
|
|
|
query = query.filter(FileGroup.id.notin_(failed_ids))
|
|
|
|
|
batch: List[Tuple[int, pathlib.Path]] = list(query.limit(VIDEO_PROCESSING_LIMIT).all())
|
review: ffprobe out of the write transaction, and three tests that were watching nothing
All three bugs confirmed; the two suggestions taken.
[bug] `video_modeler` awaited ffprobe -- a subprocess per video, up to 20
per batch -- inside its `commit=True` session. With IMMEDIATE as the
default that holds the write lock for the whole batch, which is the same
starvation this branch is curing in `video_cleanup`. (It was already
partly true before: the batch's first `session.flush([video])` took the
lock ahead of the remaining ffprobes.) The batch is now read, ffprobed
with no transaction open, then written. `get_or_create_ffprobe_json` in
modules/videos/common.py is the path-only entry point that makes that
possible; `Video.get_ffprobe_json` delegates to it and keeps appending
the cache file to its FileGroup. `validate()` -- and the poster
generation it can trigger -- is still inside the write transaction, and
now says so.
[bug] `test_download_dispatched_outside_immediate_transaction` probed
`getattr(_immediate_txn, 'active', False)`, which a ContextVar never has,
so it passed unconditionally. It now probes the actual write lock from a
competing connection under `production_like_sessions`. Mutation-checked:
moving the dispatch back inside the claim transaction fails it.
[bug] `asyncio.create_task` snapshots the current Context, so a flag held
for a write session's lifetime was inherited by every task spawned inside
it -- and the parent's `reset()` cannot reach a child's copy, leaving
that task issuing BEGIN IMMEDIATE for its *read* sessions.
`create_downloads` spawns `dispatch_downloads` exactly that way, so every
user-created download hit it. The flag now lives only across the
session's opening `connection()` call, which also means the write lock is
taken when the session opens rather than at the caller's first statement.
[suggestion] `renew_recurring_downloads` scanned every recurring Download
under the write lock, including the usual cycle where nothing is due. It
reads first now and takes the lock only for the rows that change,
re-checking status since the dispatcher may have claimed one in between.
[nit] Parenthesized the directory OR in the claim join.
`probe_write_lock_is_held` moved to conftest alongside
`write_lock_held_briefly`; three suites use it now.
Suite: 1712 passed, 5 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 20:04:40 -06:00
|
|
|
|
|
|
|
|
if not batch:
|
|
|
|
|
break
|
|
|
|
|
|
|
|
|
|
# ffprobe each file with no transaction open. This is a subprocess per video and can run
|
|
|
|
|
# for seconds; inside the write transaction below it would hold the write lock for the whole
|
|
|
|
|
# batch, and every other writer on the box would wait out `busy_timeout` (30s).
|
|
|
|
|
probed = dict()
|
|
|
|
|
for file_group_id, primary_path in batch:
|
|
|
|
|
try:
|
|
|
|
|
probed[file_group_id] = await get_or_create_ffprobe_json(pathlib.Path(str(primary_path)))
|
|
|
|
|
except Exception as e:
|
|
|
|
|
if PYTEST:
|
|
|
|
|
raise
|
|
|
|
|
logger.error(f'Unable to ffprobe Video: {primary_path}', exc_info=e)
|
|
|
|
|
# Sleep to catch cancel.
|
|
|
|
|
await asyncio.sleep(0)
|
2023-03-24 08:58:18 -06:00
|
|
|
|
review: ffprobe out of the write transaction, and three tests that were watching nothing
All three bugs confirmed; the two suggestions taken.
[bug] `video_modeler` awaited ffprobe -- a subprocess per video, up to 20
per batch -- inside its `commit=True` session. With IMMEDIATE as the
default that holds the write lock for the whole batch, which is the same
starvation this branch is curing in `video_cleanup`. (It was already
partly true before: the batch's first `session.flush([video])` took the
lock ahead of the remaining ffprobes.) The batch is now read, ffprobed
with no transaction open, then written. `get_or_create_ffprobe_json` in
modules/videos/common.py is the path-only entry point that makes that
possible; `Video.get_ffprobe_json` delegates to it and keeps appending
the cache file to its FileGroup. `validate()` -- and the poster
generation it can trigger -- is still inside the write transaction, and
now says so.
[bug] `test_download_dispatched_outside_immediate_transaction` probed
`getattr(_immediate_txn, 'active', False)`, which a ContextVar never has,
so it passed unconditionally. It now probes the actual write lock from a
competing connection under `production_like_sessions`. Mutation-checked:
moving the dispatch back inside the claim transaction fails it.
[bug] `asyncio.create_task` snapshots the current Context, so a flag held
for a write session's lifetime was inherited by every task spawned inside
it -- and the parent's `reset()` cannot reach a child's copy, leaving
that task issuing BEGIN IMMEDIATE for its *read* sessions.
`create_downloads` spawns `dispatch_downloads` exactly that way, so every
user-created download hit it. The flag now lives only across the
session's opening `connection()` call, which also means the write lock is
taken when the session opens rather than at the caller's first statement.
[suggestion] `renew_recurring_downloads` scanned every recurring Download
under the write lock, including the usual cycle where nothing is due. It
reads first now and takes the lock only for the rows that change,
re-checking status since the dispatcher may have claimed one in between.
[nit] Parenthesized the directory OR in the claim join.
`probe_write_lock_is_held` moved to conftest alongside
`write_lock_held_briefly`; three suites use it now.
Suite: 1712 passed, 5 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 20:04:40 -06:00
|
|
|
with get_db_session(commit=True) as session:
|
|
|
|
|
file_groups: List[Tuple[FileGroup, Video]] = list(
|
|
|
|
|
session.query(FileGroup, Video)
|
|
|
|
|
.filter(FileGroup.id.in_([i for i, _ in batch]))
|
|
|
|
|
.outerjoin(Video, Video.file_group_id == FileGroup.id))
|
2023-03-24 08:58:18 -06:00
|
|
|
|
review: ffprobe out of the write transaction, and three tests that were watching nothing
All three bugs confirmed; the two suggestions taken.
[bug] `video_modeler` awaited ffprobe -- a subprocess per video, up to 20
per batch -- inside its `commit=True` session. With IMMEDIATE as the
default that holds the write lock for the whole batch, which is the same
starvation this branch is curing in `video_cleanup`. (It was already
partly true before: the batch's first `session.flush([video])` took the
lock ahead of the remaining ffprobes.) The batch is now read, ffprobed
with no transaction open, then written. `get_or_create_ffprobe_json` in
modules/videos/common.py is the path-only entry point that makes that
possible; `Video.get_ffprobe_json` delegates to it and keeps appending
the cache file to its FileGroup. `validate()` -- and the poster
generation it can trigger -- is still inside the write transaction, and
now says so.
[bug] `test_download_dispatched_outside_immediate_transaction` probed
`getattr(_immediate_txn, 'active', False)`, which a ContextVar never has,
so it passed unconditionally. It now probes the actual write lock from a
competing connection under `production_like_sessions`. Mutation-checked:
moving the dispatch back inside the claim transaction fails it.
[bug] `asyncio.create_task` snapshots the current Context, so a flag held
for a write session's lifetime was inherited by every task spawned inside
it -- and the parent's `reset()` cannot reach a child's copy, leaving
that task issuing BEGIN IMMEDIATE for its *read* sessions.
`create_downloads` spawns `dispatch_downloads` exactly that way, so every
user-created download hit it. The flag now lives only across the
session's opening `connection()` call, which also means the write lock is
taken when the session opens rather than at the caller's first statement.
[suggestion] `renew_recurring_downloads` scanned every recurring Download
under the write lock, including the usual cycle where nothing is due. It
reads first now and takes the lock only for the rows that change,
re-checking status since the dispatcher may have claimed one in between.
[nit] Parenthesized the directory OR in the claim join.
`probe_write_lock_is_held` moved to conftest alongside
`write_lock_held_briefly`; three suites use it now.
Suite: 1712 passed, 5 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 20:04:40 -06:00
|
|
|
for file_group, video in file_groups:
|
2023-03-24 08:58:18 -06:00
|
|
|
try:
|
2026-08-23 15:18:15 -06:00
|
|
|
_model_video(session, file_group, video, probed)
|
2023-03-24 08:58:18 -06:00
|
|
|
except Exception as e:
|
2026-08-23 15:18:15 -06:00
|
|
|
# Before the re-raise, so no failure path can re-select this id forever.
|
|
|
|
|
failed_ids.add(file_group.id)
|
2023-03-24 08:58:18 -06:00
|
|
|
if PYTEST:
|
|
|
|
|
raise
|
2026-08-23 15:18:15 -06:00
|
|
|
logger.error(f'Unable to model Video: {file_group.primary_path}', exc_info=e)
|
2022-08-06 05:22:07 +00:00
|
|
|
|
2023-03-24 08:58:18 -06:00
|
|
|
file_group.indexed = True
|
2022-08-06 05:22:07 +00:00
|
|
|
|
review: ffprobe out of the write transaction, and three tests that were watching nothing
All three bugs confirmed; the two suggestions taken.
[bug] `video_modeler` awaited ffprobe -- a subprocess per video, up to 20
per batch -- inside its `commit=True` session. With IMMEDIATE as the
default that holds the write lock for the whole batch, which is the same
starvation this branch is curing in `video_cleanup`. (It was already
partly true before: the batch's first `session.flush([video])` took the
lock ahead of the remaining ffprobes.) The batch is now read, ffprobed
with no transaction open, then written. `get_or_create_ffprobe_json` in
modules/videos/common.py is the path-only entry point that makes that
possible; `Video.get_ffprobe_json` delegates to it and keeps appending
the cache file to its FileGroup. `validate()` -- and the poster
generation it can trigger -- is still inside the write transaction, and
now says so.
[bug] `test_download_dispatched_outside_immediate_transaction` probed
`getattr(_immediate_txn, 'active', False)`, which a ContextVar never has,
so it passed unconditionally. It now probes the actual write lock from a
competing connection under `production_like_sessions`. Mutation-checked:
moving the dispatch back inside the claim transaction fails it.
[bug] `asyncio.create_task` snapshots the current Context, so a flag held
for a write session's lifetime was inherited by every task spawned inside
it -- and the parent's `reset()` cannot reach a child's copy, leaving
that task issuing BEGIN IMMEDIATE for its *read* sessions.
`create_downloads` spawns `dispatch_downloads` exactly that way, so every
user-created download hit it. The flag now lives only across the
session's opening `connection()` call, which also means the write lock is
taken when the session opens rather than at the caller's first statement.
[suggestion] `renew_recurring_downloads` scanned every recurring Download
under the write lock, including the usual cycle where nothing is due. It
reads first now and takes the lock only for the rows that change,
re-checking status since the dispatcher may have claimed one in between.
[nit] Parenthesized the directory OR in the claim join.
`probe_write_lock_is_held` moved to conftest alongside
`write_lock_held_briefly`; three suites use it now.
Suite: 1712 passed, 5 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 20:04:40 -06:00
|
|
|
# Report batch progress
|
|
|
|
|
total_processed += len(batch)
|
|
|
|
|
if progress_callback:
|
|
|
|
|
progress_callback(total_processed)
|
2026-01-31 12:54:27 -07:00
|
|
|
|
review: ffprobe out of the write transaction, and three tests that were watching nothing
All three bugs confirmed; the two suggestions taken.
[bug] `video_modeler` awaited ffprobe -- a subprocess per video, up to 20
per batch -- inside its `commit=True` session. With IMMEDIATE as the
default that holds the write lock for the whole batch, which is the same
starvation this branch is curing in `video_cleanup`. (It was already
partly true before: the batch's first `session.flush([video])` took the
lock ahead of the remaining ffprobes.) The batch is now read, ffprobed
with no transaction open, then written. `get_or_create_ffprobe_json` in
modules/videos/common.py is the path-only entry point that makes that
possible; `Video.get_ffprobe_json` delegates to it and keeps appending
the cache file to its FileGroup. `validate()` -- and the poster
generation it can trigger -- is still inside the write transaction, and
now says so.
[bug] `test_download_dispatched_outside_immediate_transaction` probed
`getattr(_immediate_txn, 'active', False)`, which a ContextVar never has,
so it passed unconditionally. It now probes the actual write lock from a
competing connection under `production_like_sessions`. Mutation-checked:
moving the dispatch back inside the claim transaction fails it.
[bug] `asyncio.create_task` snapshots the current Context, so a flag held
for a write session's lifetime was inherited by every task spawned inside
it -- and the parent's `reset()` cannot reach a child's copy, leaving
that task issuing BEGIN IMMEDIATE for its *read* sessions.
`create_downloads` spawns `dispatch_downloads` exactly that way, so every
user-created download hit it. The flag now lives only across the
session's opening `connection()` call, which also means the write lock is
taken when the session opens rather than at the caller's first statement.
[suggestion] `renew_recurring_downloads` scanned every recurring Download
under the write lock, including the usual cycle where nothing is due. It
reads first now and takes the lock only for the rows that change,
re-checking status since the dispatcher may have claimed one in between.
[nit] Parenthesized the directory OR in the claim join.
`probe_write_lock_is_held` moved to conftest alongside
`write_lock_held_briefly`; three suites use it now.
Suite: 1712 passed, 5 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 20:04:40 -06:00
|
|
|
logger.debug(f'Modeled {len(batch)} videos')
|
2022-08-06 05:22:07 +00:00
|
|
|
|
review: ffprobe out of the write transaction, and three tests that were watching nothing
All three bugs confirmed; the two suggestions taken.
[bug] `video_modeler` awaited ffprobe -- a subprocess per video, up to 20
per batch -- inside its `commit=True` session. With IMMEDIATE as the
default that holds the write lock for the whole batch, which is the same
starvation this branch is curing in `video_cleanup`. (It was already
partly true before: the batch's first `session.flush([video])` took the
lock ahead of the remaining ffprobes.) The batch is now read, ffprobed
with no transaction open, then written. `get_or_create_ffprobe_json` in
modules/videos/common.py is the path-only entry point that makes that
possible; `Video.get_ffprobe_json` delegates to it and keeps appending
the cache file to its FileGroup. `validate()` -- and the poster
generation it can trigger -- is still inside the write transaction, and
now says so.
[bug] `test_download_dispatched_outside_immediate_transaction` probed
`getattr(_immediate_txn, 'active', False)`, which a ContextVar never has,
so it passed unconditionally. It now probes the actual write lock from a
competing connection under `production_like_sessions`. Mutation-checked:
moving the dispatch back inside the claim transaction fails it.
[bug] `asyncio.create_task` snapshots the current Context, so a flag held
for a write session's lifetime was inherited by every task spawned inside
it -- and the parent's `reset()` cannot reach a child's copy, leaving
that task issuing BEGIN IMMEDIATE for its *read* sessions.
`create_downloads` spawns `dispatch_downloads` exactly that way, so every
user-created download hit it. The flag now lives only across the
session's opening `connection()` call, which also means the write lock is
taken when the session opens rather than at the caller's first statement.
[suggestion] `renew_recurring_downloads` scanned every recurring Download
under the write lock, including the usual cycle where nothing is due. It
reads first now and takes the lock only for the rows that change,
re-checking status since the dispatcher may have claimed one in between.
[nit] Parenthesized the directory OR in the claim join.
`probe_write_lock_is_held` moved to conftest alongside
`write_lock_held_briefly`; three suites use it now.
Suite: 1712 passed, 5 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 20:04:40 -06:00
|
|
|
if len(batch) < VIDEO_PROCESSING_LIMIT:
|
|
|
|
|
# Did not reach limit, do not query again.
|
|
|
|
|
break
|
2022-08-06 05:22:07 +00:00
|
|
|
|
2023-03-24 08:58:18 -06:00
|
|
|
# Sleep to catch cancel.
|
|
|
|
|
await asyncio.sleep(0)
|
2022-08-06 05:22:07 +00:00
|
|
|
|
2023-04-01 11:52:41 -06:00
|
|
|
|
db: write intent is the opt-in, not a second session helper
Three "database is locked" failures on the docker WROLPi, one cause each:
* `INSERT INTO download` from POST /api/download -- a user creating a
video download while a refresh ran (2026-08-06 14:06:46);
* six `INSERT INTO file_group` from `upsert_file`, uploads landing in
the same three-minute window;
* `UPDATE download SET status` from `renew_recurring_downloads`.
All three are the deferred read-then-write trap: a transaction that
begins as a reader and upgrades at its first write is refused
*instantly* when another connection wrote in between, because
busy_timeout is skipped for lock upgrades. `get_immediate_db_session`
existed to avoid it, but an opt-in helper only works if every author
remembers, and read-then-write is the ordinary shape of a write. So it
is gone: `get_db_session(commit=True)` -- the existing declaration of
write intent -- now begins IMMEDIATE. Rails 7.1 and Django 5.1 made the
same call, defaulting SQLite transactions to IMMEDIATE rather than
asking application code to opt in.
That default is only safe if write transactions stay short, and one did
not. `video_cleanup` ran its whole Channel-claiming join --
`UPDATE video ... FROM channel ... LEFT JOIN file_group`, O(channels x
file_groups) -- inside one write transaction: 110 to 130 seconds each of
the eight times it appears in the log, on a library with ~1500 channels.
While it ran, *every* writer on the box failed, including ones that
began IMMEDIATE, because the hold far exceeds the 30s busy_timeout. The
14:04-14:06 failures above sit inside its 130-second window. The join
is now a read -- WAL readers never block writers -- and only the
resulting UPDATEs take the lock, 500 rows per transaction.
`claim_videos_for_channels` had the same join and the same fix; channel
import calls it with every channel in channels.yaml.
Also here:
* `renew_recurring_downloads` was a read session with an inner commit;
it is a write session now. Its computed `next_download` values now
persist even when nothing was renewed -- that is the point of
computing them.
* POST /api/download no longer writes through `request.ctx.session`.
That session is committed by the response middleware, so a writer
would hold the lock across the whole request; a write session
commits before the handler returns, ahead of the background dispatch
it fires.
* `_immediate_txn` is a ContextVar, not a threading.local. The
writers that set it include coroutines, and one asyncio thread runs
many of them interleaved.
* `write_lock_held_briefly` moved from the move tests into conftest,
since two suites now need it.
Both regression tests were verified by mutation: reverting either site
alone fails its own test, and the API one fails with the production
error verbatim.
Sources: fractaledmind.com/2023/12/11/sqlite-on-rails-improving-concurrency,
tenthousandmeters.com/blog/sqlite-concurrent-writes-and-database-is-locked-errors,
berthub.eu/articles/posts/a-brief-post-on-sqlite3-database-locked-despite-timeout
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:40:41 -06:00
|
|
|
# Rows written per transaction when claiming Videos for their Channels. Small enough that the
|
|
|
|
|
# write lock is held for milliseconds at a time; large enough that a big library does not pay a
|
|
|
|
|
# transaction per row.
|
|
|
|
|
CLAIM_CHUNK_SIZE = 500
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _chunks(items: list, size: int = CLAIM_CHUNK_SIZE):
|
|
|
|
|
for start in range(0, len(items), size):
|
|
|
|
|
yield items[start:start + size]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _first_claim_per_video(rows) -> List[Tuple[int, int]]:
|
|
|
|
|
"""One `(video_id, channel_id)` per Video.
|
|
|
|
|
|
|
|
|
|
A Video nested under two Channel directories matches both; the single `UPDATE ... FROM` this
|
|
|
|
|
replaced also picked one arbitrarily, so keep the first row and drop the rest rather than
|
|
|
|
|
writing the same Video twice."""
|
|
|
|
|
claims = dict()
|
|
|
|
|
for video_id, channel_id in rows:
|
|
|
|
|
claims.setdefault(video_id, channel_id)
|
|
|
|
|
return list(claims.items())
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _claim_videos(claims: List[Tuple[int, int]]):
|
|
|
|
|
"""Assign `(video_id, channel_id)` pairs, a chunk per write transaction.
|
|
|
|
|
|
|
|
|
|
The pairs are computed by a *read* — WAL readers never block writers — and only these short
|
|
|
|
|
UPDATEs take the write lock. Doing the whole thing as one `UPDATE ... FROM channel` held the
|
|
|
|
|
lock for the entire join: 110-130 seconds per refresh on a library with ~1500 channels, which
|
|
|
|
|
is well past the 30s `busy_timeout`, so every other writer on the box failed with "database is
|
|
|
|
|
locked" for two solid minutes.
|
|
|
|
|
|
|
|
|
|
`channel_id IS NULL` is re-checked in the UPDATE because the read ran in an earlier
|
|
|
|
|
transaction: a download may have claimed the Video since, and it should win.
|
|
|
|
|
"""
|
|
|
|
|
if not claims:
|
|
|
|
|
return
|
|
|
|
|
claimed = 0
|
|
|
|
|
for chunk in _chunks(claims):
|
|
|
|
|
with get_db_curs(commit=True) as curs:
|
|
|
|
|
curs.executemany('UPDATE video SET channel_id = ? WHERE id = ? AND channel_id IS NULL',
|
|
|
|
|
[(channel_id, video_id) for video_id, channel_id in chunk])
|
|
|
|
|
claimed += len(chunk)
|
|
|
|
|
logger.debug(f'Claimed {claimed} Videos for their Channels')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# Videos in a Channel's directory (or below it) that no Channel has claimed yet.
|
|
|
|
|
_UNCLAIMED_VIDEOS_SQL = '''
|
|
|
|
|
SELECT v.id, c.id
|
|
|
|
|
FROM video v
|
|
|
|
|
INNER JOIN file_group fg ON fg.id = v.file_group_id
|
review: ffprobe out of the write transaction, and three tests that were watching nothing
All three bugs confirmed; the two suggestions taken.
[bug] `video_modeler` awaited ffprobe -- a subprocess per video, up to 20
per batch -- inside its `commit=True` session. With IMMEDIATE as the
default that holds the write lock for the whole batch, which is the same
starvation this branch is curing in `video_cleanup`. (It was already
partly true before: the batch's first `session.flush([video])` took the
lock ahead of the remaining ffprobes.) The batch is now read, ffprobed
with no transaction open, then written. `get_or_create_ffprobe_json` in
modules/videos/common.py is the path-only entry point that makes that
possible; `Video.get_ffprobe_json` delegates to it and keeps appending
the cache file to its FileGroup. `validate()` -- and the poster
generation it can trigger -- is still inside the write transaction, and
now says so.
[bug] `test_download_dispatched_outside_immediate_transaction` probed
`getattr(_immediate_txn, 'active', False)`, which a ContextVar never has,
so it passed unconditionally. It now probes the actual write lock from a
competing connection under `production_like_sessions`. Mutation-checked:
moving the dispatch back inside the claim transaction fails it.
[bug] `asyncio.create_task` snapshots the current Context, so a flag held
for a write session's lifetime was inherited by every task spawned inside
it -- and the parent's `reset()` cannot reach a child's copy, leaving
that task issuing BEGIN IMMEDIATE for its *read* sessions.
`create_downloads` spawns `dispatch_downloads` exactly that way, so every
user-created download hit it. The flag now lives only across the
session's opening `connection()` call, which also means the write lock is
taken when the session opens rather than at the caller's first statement.
[suggestion] `renew_recurring_downloads` scanned every recurring Download
under the write lock, including the usual cycle where nothing is due. It
reads first now and takes the lock only for the rows that change,
re-checking status since the dispatcher may have claimed one in between.
[nit] Parenthesized the directory OR in the claim join.
`probe_write_lock_is_held` moved to conftest alongside
`write_lock_held_briefly`; three suites use it now.
Suite: 1712 passed, 5 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 20:04:40 -06:00
|
|
|
INNER JOIN collection col
|
|
|
|
|
ON (fg.directory = col.directory
|
|
|
|
|
OR fg.directory LIKE col.directory || '/%')
|
db: write intent is the opt-in, not a second session helper
Three "database is locked" failures on the docker WROLPi, one cause each:
* `INSERT INTO download` from POST /api/download -- a user creating a
video download while a refresh ran (2026-08-06 14:06:46);
* six `INSERT INTO file_group` from `upsert_file`, uploads landing in
the same three-minute window;
* `UPDATE download SET status` from `renew_recurring_downloads`.
All three are the deferred read-then-write trap: a transaction that
begins as a reader and upgrades at its first write is refused
*instantly* when another connection wrote in between, because
busy_timeout is skipped for lock upgrades. `get_immediate_db_session`
existed to avoid it, but an opt-in helper only works if every author
remembers, and read-then-write is the ordinary shape of a write. So it
is gone: `get_db_session(commit=True)` -- the existing declaration of
write intent -- now begins IMMEDIATE. Rails 7.1 and Django 5.1 made the
same call, defaulting SQLite transactions to IMMEDIATE rather than
asking application code to opt in.
That default is only safe if write transactions stay short, and one did
not. `video_cleanup` ran its whole Channel-claiming join --
`UPDATE video ... FROM channel ... LEFT JOIN file_group`, O(channels x
file_groups) -- inside one write transaction: 110 to 130 seconds each of
the eight times it appears in the log, on a library with ~1500 channels.
While it ran, *every* writer on the box failed, including ones that
began IMMEDIATE, because the hold far exceeds the 30s busy_timeout. The
14:04-14:06 failures above sit inside its 130-second window. The join
is now a read -- WAL readers never block writers -- and only the
resulting UPDATEs take the lock, 500 rows per transaction.
`claim_videos_for_channels` had the same join and the same fix; channel
import calls it with every channel in channels.yaml.
Also here:
* `renew_recurring_downloads` was a read session with an inner commit;
it is a write session now. Its computed `next_download` values now
persist even when nothing was renewed -- that is the point of
computing them.
* POST /api/download no longer writes through `request.ctx.session`.
That session is committed by the response middleware, so a writer
would hold the lock across the whole request; a write session
commits before the handler returns, ahead of the background dispatch
it fires.
* `_immediate_txn` is a ContextVar, not a threading.local. The
writers that set it include coroutines, and one asyncio thread runs
many of them interleaved.
* `write_lock_held_briefly` moved from the move tests into conftest,
since two suites now need it.
Both regression tests were verified by mutation: reverting either site
alone fails its own test, and the API one fails with the production
error verbatim.
Sources: fractaledmind.com/2023/12/11/sqlite-on-rails-improving-concurrency,
tenthousandmeters.com/blog/sqlite-concurrent-writes-and-database-is-locked-errors,
berthub.eu/articles/posts/a-brief-post-on-sqlite3-database-locked-despite-timeout
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:40:41 -06:00
|
|
|
INNER JOIN channel c ON c.collection_id = col.id
|
|
|
|
|
WHERE v.channel_id IS NULL
|
|
|
|
|
'''
|
|
|
|
|
|
|
|
|
|
|
2023-03-27 20:46:31 -06:00
|
|
|
@register_refresh_cleanup
|
2022-08-06 05:22:07 +00:00
|
|
|
@limit_concurrent(1)
|
|
|
|
|
def video_cleanup():
|
|
|
|
|
logger.info('Claiming Videos for their Channels')
|
2026-08-23 15:18:15 -06:00
|
|
|
# Read the FileGroups that are no longer video/audio (audio playlists never were),
|
|
|
|
|
# then unmodel them in short transactions.
|
db: write intent is the opt-in, not a second session helper
Three "database is locked" failures on the docker WROLPi, one cause each:
* `INSERT INTO download` from POST /api/download -- a user creating a
video download while a refresh ran (2026-08-06 14:06:46);
* six `INSERT INTO file_group` from `upsert_file`, uploads landing in
the same three-minute window;
* `UPDATE download SET status` from `renew_recurring_downloads`.
All three are the deferred read-then-write trap: a transaction that
begins as a reader and upgrades at its first write is refused
*instantly* when another connection wrote in between, because
busy_timeout is skipped for lock upgrades. `get_immediate_db_session`
existed to avoid it, but an opt-in helper only works if every author
remembers, and read-then-write is the ordinary shape of a write. So it
is gone: `get_db_session(commit=True)` -- the existing declaration of
write intent -- now begins IMMEDIATE. Rails 7.1 and Django 5.1 made the
same call, defaulting SQLite transactions to IMMEDIATE rather than
asking application code to opt in.
That default is only safe if write transactions stay short, and one did
not. `video_cleanup` ran its whole Channel-claiming join --
`UPDATE video ... FROM channel ... LEFT JOIN file_group`, O(channels x
file_groups) -- inside one write transaction: 110 to 130 seconds each of
the eight times it appears in the log, on a library with ~1500 channels.
While it ran, *every* writer on the box failed, including ones that
began IMMEDIATE, because the hold far exceeds the 30s busy_timeout. The
14:04-14:06 failures above sit inside its 130-second window. The join
is now a read -- WAL readers never block writers -- and only the
resulting UPDATEs take the lock, 500 rows per transaction.
`claim_videos_for_channels` had the same join and the same fix; channel
import calls it with every channel in channels.yaml.
Also here:
* `renew_recurring_downloads` was a read session with an inner commit;
it is a write session now. Its computed `next_download` values now
persist even when nothing was renewed -- that is the point of
computing them.
* POST /api/download no longer writes through `request.ctx.session`.
That session is committed by the response middleware, so a writer
would hold the lock across the whole request; a write session
commits before the handler returns, ahead of the background dispatch
it fires.
* `_immediate_txn` is a ContextVar, not a threading.local. The
writers that set it include coroutines, and one asyncio thread runs
many of them interleaved.
* `write_lock_held_briefly` moved from the move tests into conftest,
since two suites now need it.
Both regression tests were verified by mutation: reverting either site
alone fails its own test, and the API one fails with the production
error verbatim.
Sources: fractaledmind.com/2023/12/11/sqlite-on-rails-improving-concurrency,
tenthousandmeters.com/blog/sqlite-concurrent-writes-and-database-is-locked-errors,
berthub.eu/articles/posts/a-brief-post-on-sqlite3-database-locked-despite-timeout
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:40:41 -06:00
|
|
|
with get_db_curs() as curs:
|
2023-03-24 08:58:18 -06:00
|
|
|
curs.execute('''
|
db: write intent is the opt-in, not a second session helper
Three "database is locked" failures on the docker WROLPi, one cause each:
* `INSERT INTO download` from POST /api/download -- a user creating a
video download while a refresh ran (2026-08-06 14:06:46);
* six `INSERT INTO file_group` from `upsert_file`, uploads landing in
the same three-minute window;
* `UPDATE download SET status` from `renew_recurring_downloads`.
All three are the deferred read-then-write trap: a transaction that
begins as a reader and upgrades at its first write is refused
*instantly* when another connection wrote in between, because
busy_timeout is skipped for lock upgrades. `get_immediate_db_session`
existed to avoid it, but an opt-in helper only works if every author
remembers, and read-then-write is the ordinary shape of a write. So it
is gone: `get_db_session(commit=True)` -- the existing declaration of
write intent -- now begins IMMEDIATE. Rails 7.1 and Django 5.1 made the
same call, defaulting SQLite transactions to IMMEDIATE rather than
asking application code to opt in.
That default is only safe if write transactions stay short, and one did
not. `video_cleanup` ran its whole Channel-claiming join --
`UPDATE video ... FROM channel ... LEFT JOIN file_group`, O(channels x
file_groups) -- inside one write transaction: 110 to 130 seconds each of
the eight times it appears in the log, on a library with ~1500 channels.
While it ran, *every* writer on the box failed, including ones that
began IMMEDIATE, because the hold far exceeds the 30s busy_timeout. The
14:04-14:06 failures above sit inside its 130-second window. The join
is now a read -- WAL readers never block writers -- and only the
resulting UPDATEs take the lock, 500 rows per transaction.
`claim_videos_for_channels` had the same join and the same fix; channel
import calls it with every channel in channels.yaml.
Also here:
* `renew_recurring_downloads` was a read session with an inner commit;
it is a write session now. Its computed `next_download` values now
persist even when nothing was renewed -- that is the point of
computing them.
* POST /api/download no longer writes through `request.ctx.session`.
That session is committed by the response middleware, so a writer
would hold the lock across the whole request; a write session
commits before the handler returns, ahead of the background dispatch
it fires.
* `_immediate_txn` is a ContextVar, not a threading.local. The
writers that set it include coroutines, and one asyncio thread runs
many of them interleaved.
* `write_lock_held_briefly` moved from the move tests into conftest,
since two suites now need it.
Both regression tests were verified by mutation: reverting either site
alone fails its own test, and the API one fails with the production
error verbatim.
Sources: fractaledmind.com/2023/12/11/sqlite-on-rails-improving-concurrency,
tenthousandmeters.com/blog/sqlite-concurrent-writes-and-database-is-locked-errors,
berthub.eu/articles/posts/a-brief-post-on-sqlite3-database-locked-despite-timeout
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:40:41 -06:00
|
|
|
SELECT id
|
|
|
|
|
FROM file_group
|
Postgres -> SQLite hard cutover: the database moves into the media directory.
The database now lives at <media>/config/wrolpi.db so a library (files,
configs, AND database) travels with the drive between WROLPi systems.
Core:
- wrolpi/db.py: lazy SQLite engine factory (WAL, busy_timeout=30s,
foreign_keys=ON); get_db_curs yields a plain sqlite3.Cursor; native
sqlite3 paramstyle everywhere (no compatibility shims); global sqlite3
adapters keep raw-SQL datetime formats identical to the ORM's.
- wrolpi/fts.py: FTS5 external-content search replacing tsvector/GIN.
Injection-proof websearch->FTS5 translator, bm25(10,4,2,1) reproducing
A/B/C/D weighting, fast(abc)/deep column filtering, snippet headlines.
Stemming contract pinned by dedicated tests (watch/watches/watched/
watching are interchangeable).
- wrolpi/schema_ddl.py: SQLite triggers shared verbatim by the alembic
baseline and test DBs; fixes the channel video_count DELETE/UPDATE
bugs and a latent total_size bug (it used the video id as channel id).
- wrolpi/db_bootstrap.py: startup auto-create/migrate; refuses databases
written by a newer WROLPi (db_version_mismatch); SQLite 3.40 floor;
network-filesystem refusal; never creates/migrates/trusts a database
beneath an unmounted production media directory (shadow-DB guard).
- Alembic: 85 Postgres-era migrations squashed into one baseline;
revision ids are dates (2026_07_10_1350) so filenames self-sort and
alembic_version says when a schema was created; render_as_batch=True.
- All raw Postgres SQL rewritten (ANY->json_each IN, array_agg @> ->
HAVING COUNT(DISTINCT), unnest->executemany/UPDATE..FROM,
generate_series histograms->strftime GROUP BY, DISTINCT ON->
row_number(), writable CTEs->two statements, jsonb_*->json_*, explicit
NULLS FIRST/LAST, LIMIT-before-OFFSET). Simple statements converted
to SQLAlchemy instead (flag upsert, download retry/summary/tag checks,
channel-by-tag estimate, Directory delete).
- Config imports count skipped entries; background dumps are blocked
after a lossy import so partial imports can never overwrite YAML
configs (they are the sole source of truth for rebuilds now).
- Search tolerates explicit JSON nulls for limit/offset (Postgres
treated OFFSET NULL as 0; SQLite errors).
- file_group.id/doc_section.id render as INTEGER PRIMARY KEY on SQLite
(BIGINT is not a rowid alias; FTS5 content_rowid requires it).
Ops teardown: docker-compose db service, CI postgres sidecars,
initialize_api_db.sh, controller postgresql service, psycopg2-binary all
removed. upgrade.sh stops/disables postgresql (data is NOT deleted;
`systemctl enable --now postgresql` restores it); systemd units no
longer Want postgresql.service.
Existing-user upgrade: first boot creates+migrates the DB, configs
import (tags recreate their FileGroups from disk), a refresh rebuilds
the library index. QA'd on the 10.0.0.8 RPi: rebuild matched the
Postgres baseline exactly; survived an uncontrolled hard reboot with
clean integrity checks.
Accepted behavior changes: FTS5 has no stopwords; bm25 length
normalization shifts some rank orders; fast-path d_headline is
un-highlighted; all-negative queries match nothing.
Deferred: Debian/trixie package lists (incl. sqlite3 CLI), Cypress
re-baseline, VACUUM INTO snapshots, exFAT/NTFS warning, auto-kicked
rebuild refresh with progress UX.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 13:59:20 -06:00
|
|
|
WHERE model = 'video'
|
2026-08-23 15:18:15 -06:00
|
|
|
AND ((mimetype NOT LIKE 'video/%' AND mimetype NOT LIKE 'audio/%')
|
|
|
|
|
OR mimetype IN (SELECT value FROM json_each(:playlists)))
|
|
|
|
|
''', dict(playlists=json.dumps(AUDIO_PLAYLIST_MIMETYPES)))
|
db: write intent is the opt-in, not a second session helper
Three "database is locked" failures on the docker WROLPi, one cause each:
* `INSERT INTO download` from POST /api/download -- a user creating a
video download while a refresh ran (2026-08-06 14:06:46);
* six `INSERT INTO file_group` from `upsert_file`, uploads landing in
the same three-minute window;
* `UPDATE download SET status` from `renew_recurring_downloads`.
All three are the deferred read-then-write trap: a transaction that
begins as a reader and upgrades at its first write is refused
*instantly* when another connection wrote in between, because
busy_timeout is skipped for lock upgrades. `get_immediate_db_session`
existed to avoid it, but an opt-in helper only works if every author
remembers, and read-then-write is the ordinary shape of a write. So it
is gone: `get_db_session(commit=True)` -- the existing declaration of
write intent -- now begins IMMEDIATE. Rails 7.1 and Django 5.1 made the
same call, defaulting SQLite transactions to IMMEDIATE rather than
asking application code to opt in.
That default is only safe if write transactions stay short, and one did
not. `video_cleanup` ran its whole Channel-claiming join --
`UPDATE video ... FROM channel ... LEFT JOIN file_group`, O(channels x
file_groups) -- inside one write transaction: 110 to 130 seconds each of
the eight times it appears in the log, on a library with ~1500 channels.
While it ran, *every* writer on the box failed, including ones that
began IMMEDIATE, because the hold far exceeds the 30s busy_timeout. The
14:04-14:06 failures above sit inside its 130-second window. The join
is now a read -- WAL readers never block writers -- and only the
resulting UPDATEs take the lock, 500 rows per transaction.
`claim_videos_for_channels` had the same join and the same fix; channel
import calls it with every channel in channels.yaml.
Also here:
* `renew_recurring_downloads` was a read session with an inner commit;
it is a write session now. Its computed `next_download` values now
persist even when nothing was renewed -- that is the point of
computing them.
* POST /api/download no longer writes through `request.ctx.session`.
That session is committed by the response middleware, so a writer
would hold the lock across the whole request; a write session
commits before the handler returns, ahead of the background dispatch
it fires.
* `_immediate_txn` is a ContextVar, not a threading.local. The
writers that set it include coroutines, and one asyncio thread runs
many of them interleaved.
* `write_lock_held_briefly` moved from the move tests into conftest,
since two suites now need it.
Both regression tests were verified by mutation: reverting either site
alone fails its own test, and the API one fails with the production
error verbatim.
Sources: fractaledmind.com/2023/12/11/sqlite-on-rails-improving-concurrency,
tenthousandmeters.com/blog/sqlite-concurrent-writes-and-database-is-locked-errors,
berthub.eu/articles/posts/a-brief-post-on-sqlite3-database-locked-despite-timeout
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:40:41 -06:00
|
|
|
stale_ids = [i['id'] for i in curs.fetchall()]
|
|
|
|
|
for chunk in _chunks(stale_ids):
|
|
|
|
|
ids = json.dumps(chunk)
|
|
|
|
|
with get_db_curs(commit=True) as curs:
|
|
|
|
|
curs.execute('''
|
|
|
|
|
UPDATE file_group
|
|
|
|
|
SET model = NULL
|
|
|
|
|
WHERE id IN (SELECT value FROM json_each(:ids))
|
|
|
|
|
''', dict(ids=ids))
|
Postgres -> SQLite hard cutover: the database moves into the media directory.
The database now lives at <media>/config/wrolpi.db so a library (files,
configs, AND database) travels with the drive between WROLPi systems.
Core:
- wrolpi/db.py: lazy SQLite engine factory (WAL, busy_timeout=30s,
foreign_keys=ON); get_db_curs yields a plain sqlite3.Cursor; native
sqlite3 paramstyle everywhere (no compatibility shims); global sqlite3
adapters keep raw-SQL datetime formats identical to the ORM's.
- wrolpi/fts.py: FTS5 external-content search replacing tsvector/GIN.
Injection-proof websearch->FTS5 translator, bm25(10,4,2,1) reproducing
A/B/C/D weighting, fast(abc)/deep column filtering, snippet headlines.
Stemming contract pinned by dedicated tests (watch/watches/watched/
watching are interchangeable).
- wrolpi/schema_ddl.py: SQLite triggers shared verbatim by the alembic
baseline and test DBs; fixes the channel video_count DELETE/UPDATE
bugs and a latent total_size bug (it used the video id as channel id).
- wrolpi/db_bootstrap.py: startup auto-create/migrate; refuses databases
written by a newer WROLPi (db_version_mismatch); SQLite 3.40 floor;
network-filesystem refusal; never creates/migrates/trusts a database
beneath an unmounted production media directory (shadow-DB guard).
- Alembic: 85 Postgres-era migrations squashed into one baseline;
revision ids are dates (2026_07_10_1350) so filenames self-sort and
alembic_version says when a schema was created; render_as_batch=True.
- All raw Postgres SQL rewritten (ANY->json_each IN, array_agg @> ->
HAVING COUNT(DISTINCT), unnest->executemany/UPDATE..FROM,
generate_series histograms->strftime GROUP BY, DISTINCT ON->
row_number(), writable CTEs->two statements, jsonb_*->json_*, explicit
NULLS FIRST/LAST, LIMIT-before-OFFSET). Simple statements converted
to SQLAlchemy instead (flag upsert, download retry/summary/tag checks,
channel-by-tag estimate, Directory delete).
- Config imports count skipped entries; background dumps are blocked
after a lossy import so partial imports can never overwrite YAML
configs (they are the sole source of truth for rebuilds now).
- Search tolerates explicit JSON nulls for limit/offset (Postgres
treated OFFSET NULL as 0; SQLite errors).
- file_group.id/doc_section.id render as INTEGER PRIMARY KEY on SQLite
(BIGINT is not a rowid alias; FTS5 content_rowid requires it).
Ops teardown: docker-compose db service, CI postgres sidecars,
initialize_api_db.sh, controller postgresql service, psycopg2-binary all
removed. upgrade.sh stops/disables postgresql (data is NOT deleted;
`systemctl enable --now postgresql` restores it); systemd units no
longer Want postgresql.service.
Existing-user upgrade: first boot creates+migrates the DB, configs
import (tags recreate their FileGroups from disk), a refresh rebuilds
the library index. QA'd on the 10.0.0.8 RPi: rebuild matched the
Postgres baseline exactly; survived an uncontrolled hard reboot with
clean integrity checks.
Accepted behavior changes: FTS5 has no stopwords; bm25 length
normalization shifts some rank orders; fast-path d_headline is
un-highlighted; all-negative queries match nothing.
Deferred: Debian/trixie package lists (incl. sqlite3 CLI), Cypress
re-baseline, VACUUM INTO snapshots, exFAT/NTFS warning, auto-kicked
rebuild refresh with progress UX.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 13:59:20 -06:00
|
|
|
curs.execute('''
|
|
|
|
|
DELETE
|
|
|
|
|
FROM video
|
|
|
|
|
WHERE file_group_id IN (SELECT value FROM json_each(:ids))
|
db: write intent is the opt-in, not a second session helper
Three "database is locked" failures on the docker WROLPi, one cause each:
* `INSERT INTO download` from POST /api/download -- a user creating a
video download while a refresh ran (2026-08-06 14:06:46);
* six `INSERT INTO file_group` from `upsert_file`, uploads landing in
the same three-minute window;
* `UPDATE download SET status` from `renew_recurring_downloads`.
All three are the deferred read-then-write trap: a transaction that
begins as a reader and upgrades at its first write is refused
*instantly* when another connection wrote in between, because
busy_timeout is skipped for lock upgrades. `get_immediate_db_session`
existed to avoid it, but an opt-in helper only works if every author
remembers, and read-then-write is the ordinary shape of a write. So it
is gone: `get_db_session(commit=True)` -- the existing declaration of
write intent -- now begins IMMEDIATE. Rails 7.1 and Django 5.1 made the
same call, defaulting SQLite transactions to IMMEDIATE rather than
asking application code to opt in.
That default is only safe if write transactions stay short, and one did
not. `video_cleanup` ran its whole Channel-claiming join --
`UPDATE video ... FROM channel ... LEFT JOIN file_group`, O(channels x
file_groups) -- inside one write transaction: 110 to 130 seconds each of
the eight times it appears in the log, on a library with ~1500 channels.
While it ran, *every* writer on the box failed, including ones that
began IMMEDIATE, because the hold far exceeds the 30s busy_timeout. The
14:04-14:06 failures above sit inside its 130-second window. The join
is now a read -- WAL readers never block writers -- and only the
resulting UPDATEs take the lock, 500 rows per transaction.
`claim_videos_for_channels` had the same join and the same fix; channel
import calls it with every channel in channels.yaml.
Also here:
* `renew_recurring_downloads` was a read session with an inner commit;
it is a write session now. Its computed `next_download` values now
persist even when nothing was renewed -- that is the point of
computing them.
* POST /api/download no longer writes through `request.ctx.session`.
That session is committed by the response middleware, so a writer
would hold the lock across the whole request; a write session
commits before the handler returns, ahead of the background dispatch
it fires.
* `_immediate_txn` is a ContextVar, not a threading.local. The
writers that set it include coroutines, and one asyncio thread runs
many of them interleaved.
* `write_lock_held_briefly` moved from the move tests into conftest,
since two suites now need it.
Both regression tests were verified by mutation: reverting either site
alone fails its own test, and the API one fails with the production
error verbatim.
Sources: fractaledmind.com/2023/12/11/sqlite-on-rails-improving-concurrency,
tenthousandmeters.com/blog/sqlite-concurrent-writes-and-database-is-locked-errors,
berthub.eu/articles/posts/a-brief-post-on-sqlite3-database-locked-despite-timeout
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:40:41 -06:00
|
|
|
''', dict(ids=ids))
|
|
|
|
|
|
|
|
|
|
# Claim all Videos in a Channel's directory for that Channel. But, only if they have not yet
|
|
|
|
|
# been claimed. The join is the slow part, so it runs as a read; only the writes take the lock.
|
|
|
|
|
with get_db_curs() as curs:
|
|
|
|
|
curs.execute(_UNCLAIMED_VIDEOS_SQL)
|
|
|
|
|
claims = _first_claim_per_video(curs.fetchall())
|
|
|
|
|
_claim_videos(claims)
|
2025-12-05 19:49:39 -07:00
|
|
|
|
|
|
|
|
|
2026-01-09 14:01:25 -07:00
|
|
|
|
2025-12-05 19:49:39 -07:00
|
|
|
def claim_videos_for_channels(channel_ids: List[int]):
|
|
|
|
|
"""
|
|
|
|
|
Assign Videos to specific Channels based on directory paths.
|
|
|
|
|
|
|
|
|
|
This is a targeted version of video_cleanup() that only processes specific channel IDs.
|
|
|
|
|
More efficient than video_cleanup() for channel import because it limits the scope
|
|
|
|
|
of the SQL update to only the channels that were just imported/updated.
|
|
|
|
|
|
|
|
|
|
Args:
|
|
|
|
|
channel_ids: List of channel IDs to process
|
|
|
|
|
"""
|
|
|
|
|
if not channel_ids:
|
|
|
|
|
return
|
|
|
|
|
|
|
|
|
|
logger.info(f'Claiming Videos for {len(channel_ids)} channel(s)')
|
db: write intent is the opt-in, not a second session helper
Three "database is locked" failures on the docker WROLPi, one cause each:
* `INSERT INTO download` from POST /api/download -- a user creating a
video download while a refresh ran (2026-08-06 14:06:46);
* six `INSERT INTO file_group` from `upsert_file`, uploads landing in
the same three-minute window;
* `UPDATE download SET status` from `renew_recurring_downloads`.
All three are the deferred read-then-write trap: a transaction that
begins as a reader and upgrades at its first write is refused
*instantly* when another connection wrote in between, because
busy_timeout is skipped for lock upgrades. `get_immediate_db_session`
existed to avoid it, but an opt-in helper only works if every author
remembers, and read-then-write is the ordinary shape of a write. So it
is gone: `get_db_session(commit=True)` -- the existing declaration of
write intent -- now begins IMMEDIATE. Rails 7.1 and Django 5.1 made the
same call, defaulting SQLite transactions to IMMEDIATE rather than
asking application code to opt in.
That default is only safe if write transactions stay short, and one did
not. `video_cleanup` ran its whole Channel-claiming join --
`UPDATE video ... FROM channel ... LEFT JOIN file_group`, O(channels x
file_groups) -- inside one write transaction: 110 to 130 seconds each of
the eight times it appears in the log, on a library with ~1500 channels.
While it ran, *every* writer on the box failed, including ones that
began IMMEDIATE, because the hold far exceeds the 30s busy_timeout. The
14:04-14:06 failures above sit inside its 130-second window. The join
is now a read -- WAL readers never block writers -- and only the
resulting UPDATEs take the lock, 500 rows per transaction.
`claim_videos_for_channels` had the same join and the same fix; channel
import calls it with every channel in channels.yaml.
Also here:
* `renew_recurring_downloads` was a read session with an inner commit;
it is a write session now. Its computed `next_download` values now
persist even when nothing was renewed -- that is the point of
computing them.
* POST /api/download no longer writes through `request.ctx.session`.
That session is committed by the response middleware, so a writer
would hold the lock across the whole request; a write session
commits before the handler returns, ahead of the background dispatch
it fires.
* `_immediate_txn` is a ContextVar, not a threading.local. The
writers that set it include coroutines, and one asyncio thread runs
many of them interleaved.
* `write_lock_held_briefly` moved from the move tests into conftest,
since two suites now need it.
Both regression tests were verified by mutation: reverting either site
alone fails its own test, and the API one fails with the production
error verbatim.
Sources: fractaledmind.com/2023/12/11/sqlite-on-rails-improving-concurrency,
tenthousandmeters.com/blog/sqlite-concurrent-writes-and-database-is-locked-errors,
berthub.eu/articles/posts/a-brief-post-on-sqlite3-database-locked-despite-timeout
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:40:41 -06:00
|
|
|
# Read the claims, then write them in chunks -- the join must not hold the write lock (see
|
|
|
|
|
# `_claim_videos`). Channel import calls this with every channel in channels.yaml, so it is
|
|
|
|
|
# just as slow as the full `video_cleanup` sweep.
|
|
|
|
|
with get_db_curs() as curs:
|
|
|
|
|
curs.execute(f'''
|
|
|
|
|
{_UNCLAIMED_VIDEOS_SQL}
|
Postgres -> SQLite hard cutover: the database moves into the media directory.
The database now lives at <media>/config/wrolpi.db so a library (files,
configs, AND database) travels with the drive between WROLPi systems.
Core:
- wrolpi/db.py: lazy SQLite engine factory (WAL, busy_timeout=30s,
foreign_keys=ON); get_db_curs yields a plain sqlite3.Cursor; native
sqlite3 paramstyle everywhere (no compatibility shims); global sqlite3
adapters keep raw-SQL datetime formats identical to the ORM's.
- wrolpi/fts.py: FTS5 external-content search replacing tsvector/GIN.
Injection-proof websearch->FTS5 translator, bm25(10,4,2,1) reproducing
A/B/C/D weighting, fast(abc)/deep column filtering, snippet headlines.
Stemming contract pinned by dedicated tests (watch/watches/watched/
watching are interchangeable).
- wrolpi/schema_ddl.py: SQLite triggers shared verbatim by the alembic
baseline and test DBs; fixes the channel video_count DELETE/UPDATE
bugs and a latent total_size bug (it used the video id as channel id).
- wrolpi/db_bootstrap.py: startup auto-create/migrate; refuses databases
written by a newer WROLPi (db_version_mismatch); SQLite 3.40 floor;
network-filesystem refusal; never creates/migrates/trusts a database
beneath an unmounted production media directory (shadow-DB guard).
- Alembic: 85 Postgres-era migrations squashed into one baseline;
revision ids are dates (2026_07_10_1350) so filenames self-sort and
alembic_version says when a schema was created; render_as_batch=True.
- All raw Postgres SQL rewritten (ANY->json_each IN, array_agg @> ->
HAVING COUNT(DISTINCT), unnest->executemany/UPDATE..FROM,
generate_series histograms->strftime GROUP BY, DISTINCT ON->
row_number(), writable CTEs->two statements, jsonb_*->json_*, explicit
NULLS FIRST/LAST, LIMIT-before-OFFSET). Simple statements converted
to SQLAlchemy instead (flag upsert, download retry/summary/tag checks,
channel-by-tag estimate, Directory delete).
- Config imports count skipped entries; background dumps are blocked
after a lossy import so partial imports can never overwrite YAML
configs (they are the sole source of truth for rebuilds now).
- Search tolerates explicit JSON nulls for limit/offset (Postgres
treated OFFSET NULL as 0; SQLite errors).
- file_group.id/doc_section.id render as INTEGER PRIMARY KEY on SQLite
(BIGINT is not a rowid alias; FTS5 content_rowid requires it).
Ops teardown: docker-compose db service, CI postgres sidecars,
initialize_api_db.sh, controller postgresql service, psycopg2-binary all
removed. upgrade.sh stops/disables postgresql (data is NOT deleted;
`systemctl enable --now postgresql` restores it); systemd units no
longer Want postgresql.service.
Existing-user upgrade: first boot creates+migrates the DB, configs
import (tags recreate their FileGroups from disk), a refresh rebuilds
the library index. QA'd on the 10.0.0.8 RPi: rebuild matched the
Postgres baseline exactly; survived an uncontrolled hard reboot with
clean integrity checks.
Accepted behavior changes: FTS5 has no stopwords; bm25 length
normalization shifts some rank orders; fast-path d_headline is
un-highlighted; all-negative queries match nothing.
Deferred: Debian/trixie package lists (incl. sqlite3 CLI), Cypress
re-baseline, VACUUM INTO snapshots, exFAT/NTFS warning, auto-kicked
rebuild refresh with progress UX.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 13:59:20 -06:00
|
|
|
AND c.id IN (SELECT value FROM json_each(:channel_ids))
|
|
|
|
|
''', dict(channel_ids=json.dumps(list(channel_ids))))
|
db: write intent is the opt-in, not a second session helper
Three "database is locked" failures on the docker WROLPi, one cause each:
* `INSERT INTO download` from POST /api/download -- a user creating a
video download while a refresh ran (2026-08-06 14:06:46);
* six `INSERT INTO file_group` from `upsert_file`, uploads landing in
the same three-minute window;
* `UPDATE download SET status` from `renew_recurring_downloads`.
All three are the deferred read-then-write trap: a transaction that
begins as a reader and upgrades at its first write is refused
*instantly* when another connection wrote in between, because
busy_timeout is skipped for lock upgrades. `get_immediate_db_session`
existed to avoid it, but an opt-in helper only works if every author
remembers, and read-then-write is the ordinary shape of a write. So it
is gone: `get_db_session(commit=True)` -- the existing declaration of
write intent -- now begins IMMEDIATE. Rails 7.1 and Django 5.1 made the
same call, defaulting SQLite transactions to IMMEDIATE rather than
asking application code to opt in.
That default is only safe if write transactions stay short, and one did
not. `video_cleanup` ran its whole Channel-claiming join --
`UPDATE video ... FROM channel ... LEFT JOIN file_group`, O(channels x
file_groups) -- inside one write transaction: 110 to 130 seconds each of
the eight times it appears in the log, on a library with ~1500 channels.
While it ran, *every* writer on the box failed, including ones that
began IMMEDIATE, because the hold far exceeds the 30s busy_timeout. The
14:04-14:06 failures above sit inside its 130-second window. The join
is now a read -- WAL readers never block writers -- and only the
resulting UPDATEs take the lock, 500 rows per transaction.
`claim_videos_for_channels` had the same join and the same fix; channel
import calls it with every channel in channels.yaml.
Also here:
* `renew_recurring_downloads` was a read session with an inner commit;
it is a write session now. Its computed `next_download` values now
persist even when nothing was renewed -- that is the point of
computing them.
* POST /api/download no longer writes through `request.ctx.session`.
That session is committed by the response middleware, so a writer
would hold the lock across the whole request; a write session
commits before the handler returns, ahead of the background dispatch
it fires.
* `_immediate_txn` is a ContextVar, not a threading.local. The
writers that set it include coroutines, and one asyncio thread runs
many of them interleaved.
* `write_lock_held_briefly` moved from the move tests into conftest,
since two suites now need it.
Both regression tests were verified by mutation: reverting either site
alone fails its own test, and the API one fails with the production
error verbatim.
Sources: fractaledmind.com/2023/12/11/sqlite-on-rails-improving-concurrency,
tenthousandmeters.com/blog/sqlite-concurrent-writes-and-database-is-locked-errors,
berthub.eu/articles/posts/a-brief-post-on-sqlite3-database-locked-despite-timeout
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:40:41 -06:00
|
|
|
claims = _first_claim_per_video(curs.fetchall())
|
|
|
|
|
_claim_videos(claims)
|