Commit graph

3 commits

Author SHA1 Message Date
lrnselfreliance
8edace08bd
Python packages install from hash-locked requirements; yt-dlp still floats (#913)
* Python packages install from hash-locked requirements; yt-dlp still floats

Every pip install trusted PyPI over TLS: requirements.txt pins only top-level versions
(some with >=), the controller's are almost all >=, and dependencies were never pinned, so
a forged certificate (as the eMudhra key compromise allowed) could change code in the API,
the controller, or the archive container.  scripts/lock_requirements.sh generates a
universal, hash-locked .lock beside each requirements.txt, and
install_python_requirements.sh installs it with --require-hashes at every site.  yt-dlp is
left out of the lock and upgraded afterwards so devices still get YouTube fixes; its
dependencies are locked.  The unpinned `pip install --upgrade pip` steps are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* The unhashed yt-dlp install cannot replace locked packages, and nothing builds from source

Review of #913 found two holes.  The yt-dlp step ran without --no-deps, so pip's
only-if-needed strategy would still fetch an unhashed dependency whenever yt-dlp's metadata
asked for a version the lock did not install.  And srt is published only as a source
archive, whose build installed setuptools with hash checking off on every install.

The yt-dlp step now uses --no-deps (its dependencies come only from the lock), both steps
are retried as the image builds did before, and installs and locks are wheels-only.  srt
3.5.3 is vendored unmodified as wrolpi/vendor/srt.py (MIT), verified against the sdist
hash the lock pinned; webvtt-py's SRT reader rejects WROLPi's own test file, so it is not
a replacement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 12:00:39 -07:00
Roland
950a53017f
upgrade: rebuild venvs after a Python major-version change
A Debian major upgrade (e.g. 12 -> 13) replaces /usr/bin/python3
(3.11 -> 3.13).  Existing venvs still resolve `python3` to the system
interpreter, but their packages live in lib/python3.11/site-packages,
which the new interpreter never searches -- every import fails and
`pip install --upgrade` cannot repair them, taking down the API,
Controller, yt-dlp, and Help.

upgrade.sh now compares each venv's recorded Python version against the
current one and does a full rebuild via reset_virtual_environments.sh
when they differ.  Detecting a mismatch (rather than literally "3.11")
also self-heals future bumps and is a no-op once versions match or on a
fresh install with no pyvenv.cfg yet.

reset_virtual_environments.sh gains a -y/--yes flag to skip its
confirmation prompt; the interactive yes_or_no read loops forever on EOF
under the tty-less wrolpi-upgrade.service.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 09:08:30 -06:00
Roland
1deaeb3144
Clearing cached python files in repair.sh. Creating reset_virtual_environments.sh script which deletes the virtual environments and rebuilds them (requires internet)
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-02-28 16:11:12 -07:00