wrolpi/scripts/lock_requirements.sh

23 lines
1.1 KiB
Bash
Raw Normal View History

Python packages install from hash-locked requirements; yt-dlp still floats (#913) * Python packages install from hash-locked requirements; yt-dlp still floats Every pip install trusted PyPI over TLS: requirements.txt pins only top-level versions (some with >=), the controller's are almost all >=, and dependencies were never pinned, so a forged certificate (as the eMudhra key compromise allowed) could change code in the API, the controller, or the archive container. scripts/lock_requirements.sh generates a universal, hash-locked .lock beside each requirements.txt, and install_python_requirements.sh installs it with --require-hashes at every site. yt-dlp is left out of the lock and upgraded afterwards so devices still get YouTube fixes; its dependencies are locked. The unpinned `pip install --upgrade pip` steps are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * The unhashed yt-dlp install cannot replace locked packages, and nothing builds from source Review of #913 found two holes. The yt-dlp step ran without --no-deps, so pip's only-if-needed strategy would still fetch an unhashed dependency whenever yt-dlp's metadata asked for a version the lock did not install. And srt is published only as a source archive, whose build installed setuptools with hash checking off on every install. The yt-dlp step now uses --no-deps (its dependencies come only from the lock), both steps are retried as the image builds did before, and installs and locks are wheels-only. srt 3.5.3 is vendored unmodified as wrolpi/vendor/srt.py (MIT), verified against the sdist hash the lock pinned; webvtt-py's SRT reader rejects WROLPi's own test file, so it is not a replacement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 12:00:29 -07:00
#!/usr/bin/env bash
# Regenerates the hash-locked .lock beside each requirements.txt. Run it after editing a requirements.txt (or merging
# a dependabot bump); wrolpi/test/test_python_requirements.py fails until the lock satisfies its requirements.txt.
#
# The locks are universal: they carry the hashes for every platform's wheels (Raspberry Pi and amd64), for the Python
# each environment runs. Wheels only: building a source archive installs its build tools (setuptools) without hashes,
# so a package with no wheel fails here. yt-dlp is left out; scripts/install_python_requirements.sh upgrades it.
set -e
cd "$(dirname "$(dirname "$(realpath "$0")")")"
lock() {
local requirements=$1 python=$2
shift 2
uv pip compile "${requirements}" --universal --python-version "${python}" --generate-hashes --only-binary :all: \
--no-header --quiet "$@" -o "${requirements%.txt}.lock"
}
lock requirements.txt 3.13 --no-emit-package yt-dlp --no-emit-package yt-dlp-ejs
lock controller/requirements.txt 3.13
# browserless/chrome is Ubuntu 22.04.
lock docker/archive/requirements.txt 3.10