mirror of
https://github.com/lrnselfreliance/wrolpi.git
synced 2026-11-19 14:36:22 -05:00
* Python packages install from hash-locked requirements; yt-dlp still floats Every pip install trusted PyPI over TLS: requirements.txt pins only top-level versions (some with >=), the controller's are almost all >=, and dependencies were never pinned, so a forged certificate (as the eMudhra key compromise allowed) could change code in the API, the controller, or the archive container. scripts/lock_requirements.sh generates a universal, hash-locked .lock beside each requirements.txt, and install_python_requirements.sh installs it with --require-hashes at every site. yt-dlp is left out of the lock and upgraded afterwards so devices still get YouTube fixes; its dependencies are locked. The unpinned `pip install --upgrade pip` steps are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * The unhashed yt-dlp install cannot replace locked packages, and nothing builds from source Review of #913 found two holes. The yt-dlp step ran without --no-deps, so pip's only-if-needed strategy would still fetch an unhashed dependency whenever yt-dlp's metadata asked for a version the lock did not install. And srt is published only as a source archive, whose build installed setuptools with hash checking off on every install. The yt-dlp step now uses --no-deps (its dependencies come only from the lock), both steps are retried as the image builds did before, and installs and locks are wheels-only. srt 3.5.3 is vendored unmodified as wrolpi/vendor/srt.py (MIT), verified against the sdist hash the lock pinned; webvtt-py's SRT reader rejects WROLPi's own test file, so it is not a replacement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
34 lines
1.3 KiB
Bash
Executable file
34 lines
1.3 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Usage: install_python_requirements.sh PIP REQUIREMENTS_TXT
|
|
#
|
|
# Installs REQUIREMENTS_TXT's generated lock (the .lock beside it, see scripts/lock_requirements.sh) with
|
|
# --require-hashes, so every package is checked against a hash committed with WROLPi instead of trusting PyPI over TLS.
|
|
# Wheels only: building a source archive would install its build tools (setuptools) without hashes.
|
|
#
|
|
# Then upgrades the yt-dlp lines of REQUIREMENTS_TXT without hashes: yt-dlp is left out of the lock so devices get
|
|
# YouTube fixes as soon as they are released. --no-deps keeps that unhashed download to yt-dlp's own distributions; its
|
|
# dependencies come only from the lock, so a yt-dlp that needs newer ones waits for the lock to be regenerated.
|
|
set -e
|
|
|
|
PIP=$1
|
|
REQUIREMENTS=$2
|
|
LOCK="${REQUIREMENTS%.txt}.lock"
|
|
|
|
# Retried: image builds and fresh installs can hit a flaky network.
|
|
retry() {
|
|
local attempt
|
|
for attempt in 1 2 3; do
|
|
"$@" && return 0
|
|
done
|
|
return 1
|
|
}
|
|
|
|
retry "${PIP}" install --require-hashes --only-binary :all: -r "${LOCK}"
|
|
|
|
yt_dlp=()
|
|
while IFS= read -r line; do
|
|
yt_dlp+=("${line}")
|
|
done < <(grep -E '^yt-dlp' "${REQUIREMENTS}" | sed 's/[[:space:]]*#.*//')
|
|
if [ ${#yt_dlp[@]} -gt 0 ]; then
|
|
retry "${PIP}" install --upgrade --no-deps --only-binary :all: "${yt_dlp[@]}"
|
|
fi
|