wrolpi/scripts
lrnselfreliance 8edace08bd
Python packages install from hash-locked requirements; yt-dlp still floats (#913)
* Python packages install from hash-locked requirements; yt-dlp still floats

Every pip install trusted PyPI over TLS: requirements.txt pins only top-level versions
(some with >=), the controller's are almost all >=, and dependencies were never pinned, so
a forged certificate (as the eMudhra key compromise allowed) could change code in the API,
the controller, or the archive container.  scripts/lock_requirements.sh generates a
universal, hash-locked .lock beside each requirements.txt, and
install_python_requirements.sh installs it with --require-hashes at every site.  yt-dlp is
left out of the lock and upgraded afterwards so devices still get YouTube fixes; its
dependencies are locked.  The unpinned `pip install --upgrade pip` steps are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* The unhashed yt-dlp install cannot replace locked packages, and nothing builds from source

Review of #913 found two holes.  The yt-dlp step ran without --no-deps, so pip's
only-if-needed strategy would still fetch an unhashed dependency whenever yt-dlp's metadata
asked for a version the lock did not install.  And srt is published only as a source
archive, whose build installed setuptools with hash checking off on every install.

The yt-dlp step now uses --no-deps (its dependencies come only from the lock), both steps
are retried as the image builds did before, and installs and locks are wheels-only.  srt
3.5.3 is vendored unmodified as wrolpi/vendor/srt.py (MIT), verified against the sdist
hash the lock pinned; webvtt-py's SRT reader rejects WROLPi's own test file, so it is not
a replacement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 12:00:39 -07:00
..
add_caddy_apt_repo.sh Caddy's apt repository is trusted through the key committed with WROLPi (#911) 2026-10-31 10:22:19 -06:00
api_perf.py files: SQL FileGroup compare, stem column, drop unused discovery path (#713) 2026-09-18 08:57:56 -06:00
api_perf_compare.py Fix API stalls: non-blocking switch worker, threaded searches, indexed Videos page (#678) 2026-08-29 16:17:53 -06:00
api_perf_hosts.example.json Fix API stalls: non-blocking switch worker, threaded searches, indexed Videos page (#678) 2026-08-29 16:17:53 -06:00
build_app.sh Bake the app build into images; rebuild on-device only when stale (#536) 2026-07-15 15:19:25 -06:00
build_solar_climatology.py Add a Solar calculator (#778) 2026-10-04 18:36:45 -06:00
check_database_integrity.sh check_database_integrity.sh: address greptile review 2026-07-22 19:47:48 -06:00
check_dead_code.py Python packages install from hash-locked requirements; yt-dlp still floats (#913) 2026-11-01 12:00:39 -07:00
cleanup_orphaned_tags.py Creating script to clean up orphaned tag files. 2026-02-28 17:05:02 -07:00
delete_downloads.py Many fixes split out from the feature/unified-query branch. Installing diagnostic tools in the built images. Displaying video comment statistics. Echoing error when install.sh fails. 2024-04-14 13:13:30 -06:00
delete_similarly_named_videos.py scripts: set exec bit on delete_similarly_named_videos.py 2026-07-22 19:47:35 -06:00
dev_build.sh Postgres -> SQLite hard cutover: the database moves into the media directory. 2026-07-12 14:09:37 -06:00
find_and_convert_srt_to_vtt.py Adding option to ignore errors in find_and_convert_srt_to_vtt.py. 2023-08-04 08:36:30 -06:00
find_and_rename_orphan_video_files.py Bumping version for new feature. 2022-12-07 18:15:28 -07:00
find_corrupt_videos.py Postgres -> SQLite hard cutover: the database moves into the media directory. 2026-07-12 14:09:37 -06:00
fix_media_permissions.sh File permissions fix script checks for exfat and other file systems before trying to repair, displays a warning and gives advice on how to fix the issue. 2026-03-05 18:18:53 -07:00
generate_certificates.sh Certificates: only include SANs permitted by the Root CA name constraints. 2026-07-05 12:38:08 -06:00
generate_subtitles.py Creating script which finds all SRT files and converts them to VTT. 2023-08-03 22:36:11 -06:00
install_debian.sh Python packages install from hash-locked requirements; yt-dlp still floats (#913) 2026-11-01 12:00:39 -07:00
install_help_service.sh Replacing Tag names with escaped tag names to avoid errors in Tags Directory. 2024-10-16 18:56:10 -06:00
install_npm_tools.sh serve, single-file and readability-extractor install from a committed lockfile (#912) 2026-10-31 10:39:08 -06:00
install_protomaps.sh Remove stale Postgres references after SQLite cutover. 2026-07-18 13:28:54 -06:00
install_python_requirements.sh Python packages install from hash-locked requirements; yt-dlp still floats (#913) 2026-11-01 12:00:39 -07:00
install_raspberrypios.sh Python packages install from hash-locked requirements; yt-dlp still floats (#913) 2026-11-01 12:00:39 -07:00
lock_requirements.sh Python packages install from hash-locked requirements; yt-dlp still floats (#913) 2026-11-01 12:00:39 -07:00
renew_certificates.sh Certificates: help.sh validates the CA and leaf certs; fix renew exec bit. 2026-07-09 16:00:43 -06:00
reset_api_db.sh Postgres -> SQLite hard cutover: the database moves into the media directory. 2026-07-12 14:09:37 -06:00
reset_tags.sh Postgres -> SQLite hard cutover: the database moves into the media directory. 2026-07-12 14:09:37 -06:00
reset_virtual_environments.sh Python packages install from hash-locked requirements; yt-dlp still floats (#913) 2026-11-01 12:00:39 -07:00
search_order_perf.py search: don't COUNT(*) OVER() the page query (#702) 2026-09-12 14:44:30 -06:00
singlefile_browser_wrapper.sh Single file deno (#478) 2026-06-12 17:07:28 -06:00
singlefile_to_archive.py A Channel can be tagged with a single Tag. The Channel can be moved into a Tag Directory according to the config. 2024-10-16 18:56:19 -06:00
start_kiwix_serve.sh zim: address PR review — config race, YAML null, path traversal, pinned build 2026-07-20 19:10:09 -06:00
upgrade.sh Python packages install from hash-locked requirements; yt-dlp still floats (#913) 2026-11-01 12:00:39 -07:00
upgrade_yt-dlp.sh Creating script which updates yt-dlp in the wrolpi venv to the latest available. 2025-01-23 17:46:37 -07:00
vulture_baseline.txt Fail CI when vulture finds new dead code (#889) 2026-10-27 16:10:52 -06:00
wrolpi-usb.sh Docs: recommend Raspberry Pi Imager instead of Etcher. 2026-07-03 18:58:54 -06:00