api: run the perpetual loops in one Sanic-managed process (#718)
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled

* api: run the perpetual loops in one Sanic-managed process

The singleton background loops (file worker, downloads, switches, jobs,
bulk tag, flag checks, update check) now run in a dedicated process that
Sanic's Worker Manager owns: `run_perpetual_process` is registered with
`app.manager.manage()` from `main_process_ready`.  The manager guarantees
a single instance and SIGTERMs it on shutdown.  No server worker runs the
loops, so the owner-pid / heartbeat / watchdog claim scheme is deleted
rather than hardened.

`perpetual_signal` keeps its name and is now a plain registrar.  A new
`per_worker_task` registrar covers loops every server worker must run for
itself: syncing its log level (previously only the owner did, so the other
workers never picked up level changes) and a health check that asks the
manager to restart the perpetual process when it is FAILED (Sanic marks a
dead managed process but does not restart it).

Downloads are started as background tasks instead of Sanic signals; the
perpetual process has no server, so no finalized signal router.  The
update check and the upgrade-check endpoint run git in a thread via
`refresh_update_status`.  /api/status lists the perpetual process next
to the servers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* perpetual: survive a predecessor's death, exit cleanly, coordinate restarts

Review fixes for the managed perpetual process.

Shutdown: cancel_background_tasks / cancel_refresh_tasks gather with
return_exceptions, so a cancelled child's CancelledError no longer aborts
the shutdown that called them (the process exited 1 with a download in
flight).

Restart after a crash: reconcile_after_predecessor heals shared locks a
dead process left held, fails RUNNING Jobs owned by a dead pid, clears
processing_domains, renews pending Downloads, and resets the file-worker
status before the loops start.  N server workers issue one restart
request per cooldown through a shared timestamp in shared_ctx.

The log-level sync now also runs in the perpetual process.  Manage() is
registered through register_perpetual_process so it is testable; the
entry point refuses to run with no loops registered (spawn).

Tests exercise the real entry point in a forked process with a live
background task under SIGINT and SIGTERM, Sanic's real WorkerManager
marking a crashed managed process FAILED and reviving it on restart,
Sanic's process naming and ProcessState enum, the listener registration,
and the reconcile step.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* perpetual: import the jobs helper inside its reconcile step

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
lrnselfreliance 2026-09-22 21:52:55 -06:00 • committed by Roland
parent ca80f9428f
commit 69167391c7
No known key found for this signature in database
GPG key ID: 1BA620CE6C9A4680
16 changed files with 982 additions and 522 deletions

View file

@ -211,8 +211,6 @@ async def test_channel_download_requires_refresh(
assert channel.refreshed == expected
assert bool(channel.info_json) == expected
await async_client.sanic_app.dispatch('wrolpi.download.download')
assert_refreshed(False)
test_session.commit()