rt/wrolpi

mirror of https://github.com/lrnselfreliance/wrolpi.git synced 2026-10-26 10:51:36 -04:00

History

lrnselfreliance 811793d767 perpetual: clear in-progress flags after a crash, downloads only in the perpetual process, KillMode=mixed (#726 ) * perpetual: clear in-progress flags after a crash; downloads only in the perpetual process Found by a kill -9 stress test on a device: A SIGKILL inside `with flags.file_worker_counting` (or any of the file worker's progress flags) left the shared flag set, so every later refresh raised "flag is already set" until the whole API was restarted. reconcile_after_predecessor now clears the perpetual process's in-progress flags (file_worker_, global_refresh_active) and reports it. create_downloads started a dispatch in whatever process called it, so an API worker executed downloads (and could take transcode_lock) despite the single-process design. It now dispatches only inside the perpetual process (or under pytest); the download loop picks new rows up within seconds otherwise. The shared restart cooldown drops from 60 s to 15 s: it only has to outlast the manager acting on a request, and a replacement that dies right away was left dead for the rest of the minute. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> systemd: KillMode=mixed for the API so main shuts its own children down With control-group kill, SIGTERM reaches the multiprocessing Manager at the same moment as Sanic's main process; main's cleanup then writes worker state to a dead Manager (broken pipe) and exits 1, so a plain `systemctl stop` is recorded as a failure whenever a child takes a moment to stop (a transcode being killed, for instance). Signal only main; it stops the workers and the perpetual process itself. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * perpetual: clear an in-progress flag only when its holder is dead `with flag:` now records the holder pid in shared_ctx.flag_holders and drops it on exit. The reconcile step clears the flags only the perpetual process ever enters (file_worker_counting, file_worker_discovery, global_refresh_active) whenever they are set, and the flags API workers also enter (file_worker_modeling / indexing / cleanup via refresh_sync, map_search_building via the map rebuild endpoints) only when the recorded holder is dead. file_worker_busy is no longer touched: a collection move in an API worker holds it as a mutex, and process_queue drops a stale one itself. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-26 11:22:58 -06:00
..
raspberrypios	perpetual: clear in-progress flags after a crash, downloads only in the perpetual process, KillMode=mixed (#726 )	2026-09-26 11:22:58 -06:00
README.md	Combining install scripts into a new repair script. Removing create_db script in favor of repair script.	2023-10-03 12:15:29 -06:00

lrnselfreliance 811793d767

perpetual: clear in-progress flags after a crash, downloads only in the perpetual process, KillMode=mixed (#726 )

* perpetual: clear in-progress flags after a crash; downloads only in the perpetual process

Found by a kill -9 stress test on a device:

A SIGKILL inside `with flags.file_worker_counting` (or any of the file
worker's progress flags) left the shared flag set, so every later refresh
raised "flag is already set" until the whole API was restarted.
reconcile_after_predecessor now clears the perpetual process's in-progress
flags (file_worker_*, global_refresh_active) and reports it.

create_downloads started a dispatch in whatever process called it, so an
API worker executed downloads (and could take transcode_lock) despite the
single-process design.  It now dispatches only inside the perpetual
process (or under pytest); the download loop picks new rows up within
seconds otherwise.

The shared restart cooldown drops from 60 s to 15 s: it only has to
outlast the manager acting on a request, and a replacement that dies
right away was left dead for the rest of the minute.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* systemd: KillMode=mixed for the API so main shuts its own children down

With control-group kill, SIGTERM reaches the multiprocessing Manager at
the same moment as Sanic's main process; main's cleanup then writes
worker state to a dead Manager (broken pipe) and exits 1, so a plain
`systemctl stop` is recorded as a failure whenever a child takes a
moment to stop (a transcode being killed, for instance).  Signal only
main; it stops the workers and the perpetual process itself.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* perpetual: clear an in-progress flag only when its holder is dead

`with flag:` now records the holder pid in shared_ctx.flag_holders and
drops it on exit.  The reconcile step clears the flags only the perpetual
process ever enters (file_worker_counting, file_worker_discovery,
global_refresh_active) whenever they are set, and the flags API workers
also enter (file_worker_modeling / indexing / cleanup via refresh_sync,
map_search_building via the map rebuild endpoints) only when the
recorded holder is dead.  file_worker_busy is no longer touched: a
collection move in an API worker holds it as a mutex, and
process_queue drops a stale one itself.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

2026-09-26 11:22:58 -06:00

raspberrypios

perpetual: clear in-progress flags after a crash, downloads only in the perpetual process, KillMode=mixed (#726 )

2026-09-26 11:22:58 -06:00

README.md

Combining install scripts into a new repair script. Removing create_db script in favor of repair script.

2023-10-03 12:15:29 -06:00

README.md

Configuration Files

The files in etc are files necessary to build an on-the-metal WROLPi. Raspberry Pi has more config files because it is assumed to be the main OS. Debian uses the Raspberry Pi configs when possible.