wrolpi/docker/api/Dockerfile

53 lines
2.3 KiB
Text
Raw Normal View History

Debian 13 (trixie) rebase: docker, debian-live-config, and pi-gen builds. All three build targets move from Debian 12/bookworm to Debian 13/trixie for fresh installs; existing Debian 12 systems remain supported in place. Docker: api/help on python:3.13-trixie, controller on 3.13-slim-trixie, app on node:20-trixie (matches trixie's nodejs 20), samba/zim on debian:13. Full backend suite passes on Python 3.13 in the api container. debian-live-config: --distribution trixie; postgres teardown for the SQLite cutover (pg_dropcluster hook block removed, wrolpi-bootstrap.sh no longer creates a cluster or calls the deleted initialize_api_db.sh — the API creates wrolpi.db itself on first start). README documents the SQLite layout and drops Etcher recommendations. pi-gen: upstream has no trixie branch — the arm64 branch now defaults to RELEASE=trixie; KEYBOARD_KEYMAP=en_US is rejected by trixie's console-setup (ckbcomp validates), use 'us'; stop stamping a Postgres .pgpass into /etc/skel. Package lists (both targets): postgresql-15/postgresql-doc/libpq-dev deleted; libtiff5-dev renamed to libtiff-dev; software-properties-common and cpufrequtils removed from trixie (Controller CPU throttle falls back to sysfs, dead cpufreq-set sudoers rule removed); locales and gnupg added (no longer arrive transitively under --apt-recommends false); sqlite3 CLI added for reset_tags.sh. help.sh: replace the Postgres service/port/socket/cluster/psql checks (all false FAILEDs after the SQLite cutover) with SQLite checks: wrolpi.db exists, PRAGMA quick_check, file_group table + row count, run as the wrolpi user so root never creates the -shm sidecar. Both build scripts take a WROLPI_BUILD_DIR override so development builds can run beside the release pipeline's build trees; the sudo re-exec passes it through explicitly (a plain sudo would drop it). test_common.py: the truncate tests hard-coded sys.getsizeof results from the 3.11 object layout (str/list shrank in 3.12); assert the contract (fits budget, prefix, monotonic in budget) so they pass on 3.11 and 3.13. Verified: trixie ISO (3.7G) and pi-gen desktop image (4.3G) both built from scratch and passed their sanity checks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 15:16:00 -06:00
FROM python:3.13-trixie
ENV DOCKER=true
# Deno trusts the system store, which apt keeps current, instead of its bundled roots.
ENV DENO_TLS_CA_STORE=system
WORKDIR /opt/wrolpi
# Update dependencies for the services install.
A WROLPi serves its library over rsync, and makes a database snapshot on request (#988) * A WROLPi serves its library over rsync, and makes a database snapshot on request The first part of replacing the sync: a secondary will copy its primary's whole media directory with rsync and replace its database with the primary's, instead of downloading the files of FileGroups over HTTP (which never copies an ignored directory, such as the map's). - An rsync daemon serves the media directory read-only on port 8873, on every WROLPi. It never serves the TLS keys, the cookies, or the database in use. On a Raspberry Pi and Debian it is the wrolpi-rsyncd unit; in Docker the Controller runs it in a helper container made from the api container (its image, user and volumes), so the media directory is still configured in one place. All three read controller/rsyncd.conf. - POST /api/sync/snapshot asks for a snapshot of the database, which the daemon serves from config/.sync. An unchanged database is not written again. - /api/sync/info also says the database's migration and whether the media directory is mounted. Nothing uses these yet; the existing sync is unchanged. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * The rsync daemon follows no links, and no snapshot is made in WROL Mode - A client that asked the daemon to follow symbolic links could read what a link in the library points at. A relative link to config/ssl was read this way although that directory is left out of the module. The daemon now refuses the three options that follow links; a link is sent as a link. - A database snapshot is a new file on the primary's drive, so WROL Mode refuses it, as it does a backup. A snapshot made earlier is still served. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * No rsync helper from an image without rsync, and the helper follows the api container Review found: - A helper made from an api image built before rsync was added exits at once, and Docker restarted it forever while the Controller reported it created. The image is now looked at first; no helper is made from one without rsync, an older helper is left serving, and the reason is logged once. - The helper was only checked when the Controller started, so a recreated or rebuilt api container left it stale. It is now checked every minute. - A snapshot cut off by its process being killed stayed marked as running and held back every later request. - The snapshot's half-written temporary file was served by the daemon. - The comment on the daemon's refused options claimed more than they give: a link that is part of the requested path is still followed. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-18 16:47:02 -07:00
RUN apt-get update && apt-get install -y ffmpeg catdoc sysstat aria2 iputils-ping unrar-free unzip rsync \
build-essential libsqlite3-dev zlib1g-dev git
# Install tippecanoe (for map search index building via tippecanoe-decode).
RUN git clone --depth 1 --branch 2.79.0 https://github.com/felt/tippecanoe.git /tmp/tippecanoe \
&& cd /tmp/tippecanoe && make -j$(nproc) && make install \
&& rm -rf /tmp/tippecanoe \
&& tippecanoe-decode --help 2>&1 | head -1
Squashed commit of the following: commit 26bde0fb48d59a86c2fb9f53a7218d49f22d4860 Author: Roland <roland@learningselfreliance.com> Date: Tue Mar 8 13:25:56 2022 -0700 Using osm2pgsql append to preserve other imported maps. commit c6c52623eaf02e691b1e75dcf39c3a8334ead55a Author: Roland <roland@learningselfreliance.com> Date: Tue Mar 8 08:20:38 2022 -0700 Using placeholder instead of loader when table is loading. commit 44fec6490e26603c88a98f6bc70fc18686e17f09 Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 20:58:34 2022 -0700 Simplifying tableRow function. commit 11b8b57f1fa59556397620f3cad9a33a0237501d Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 20:53:51 2022 -0700 Removing scientific humanFileSize. commit 0a09d0a6963afebed7f388aad71bbdd3151d2e9e Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 20:46:22 2022 -0700 Fixing active loader. commit 07f4deeac861351d7483ef57278eb2e0941e31fb Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 20:40:44 2022 -0700 Increasing recommended map file size. commit af3ac492c81f4c8f84563a448bc272ae9334d32d Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 20:35:13 2022 -0700 Reworking logic order for simplicity. Fetching downloads on interval. commit 3d9d31fcec87ed02ced7f67c34907faa5a3b54e5 Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 20:24:39 2022 -0700 Handling when pbf import status are loading. commit 3798961a48e7c9eb165ca4e4051f239dbb088d0c Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 19:53:50 2022 -0700 Fetching import status every 30 seconds. commit f7a8193b0955d14ead8090aa74ce3c5a681b57ec Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 19:33:17 2022 -0700 Stopping renderd during installation. commit 5bf050b59e0a6e391184596f5e6f0737dea1221d Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 19:26:59 2022 -0700 Removing duplciate serve. commit 892e3315eeeb5d2516996b901a32a73a66dae078 Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 19:22:14 2022 -0700 Setting bash flags in sub-install script. commit d5d2acd2cd85e089c78aa85a27c5d966222a1347 Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 19:20:19 2022 -0700 Moving bulk of installation script into separate script so the latest code will be used during initial installation. commit ab030d8af834a4a8772edd06df360c51e72fbb02 Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 18:40:34 2022 -0700 Removing mod_tile/ajt directory instead of entire contents of mod_tile. Showing import as relative to media directory. commit c05199db4ecf56f4d096051b58689457882af12a Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 18:39:59 2022 -0700 Remvoing marker from map. commit ba5df4cf62d61b55281277a6254836e973c766ae Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 11:57:41 2022 -0700 Creating import_map.sh script. commit 1a16e46130e775efe375adb22917ba12e44855cd Author: Roland <roland@learningselfreliance.com> Date: Mon Mar 7 11:11:01 2022 -0700 Increasing ram usage during import to avoid hanging import. commit 0f5ce1a9f806f1c0878dd804b04587b6340ffd16 Author: Roland <roland@learningselfreliance.com> Date: Sat Mar 5 19:19:47 2022 -0700 Increasing map_file.size size. commit eb6e01512af59fb4a18ab15643c7bc61f3d48e53 Author: Roland <roland@learningselfreliance.com> Date: Sat Mar 5 18:55:55 2022 -0700 Copying leaflet js/css files into docker container and to Apache directory so they will be available offline. commit 5b687324e119f021673f42e8d079b83034b70d53 Author: Roland <roland@learningselfreliance.com> Date: Sat Mar 5 18:36:06 2022 -0700 Matching tab height on Map page. commit 8031786d672e18366e9a2eb2d7bdfcd8611d27f3 Author: Roland <roland@learningselfreliance.com> Date: Sat Mar 5 18:34:20 2022 -0700 Shrinking HelpPopup Icon. commit 5ea94488e43bbc5c414e61c9173a3cfa908c7104 Author: Roland <roland@learningselfreliance.com> Date: Sat Mar 5 18:31:30 2022 -0700 Testing if no PBFs are requested to be imported. commit c4e86be1faed78bd3af1f3487471c01a1958a433 Author: Roland <roland@learningselfreliance.com> Date: Sat Mar 5 18:18:48 2022 -0700 Using Icon instead of Button for HelpPopup. commit 51d94f6d85885803813544fea16a56372babaa82 Author: Roland <roland@learningselfreliance.com> Date: Sat Mar 5 18:07:03 2022 -0700 PBF files are sorted by the api. commit 40f3a7ff04a83db0153f64a88a37f1d56ce043aa Author: Roland <roland@learningselfreliance.com> Date: Fri Mar 4 18:35:51 2022 -0700 Adding help popup. commit 1da8d9fa403ec2a3b3c16c11a3937de37c895a35 Author: Roland <roland@learningselfreliance.com> Date: Fri Mar 4 09:41:01 2022 -0700 Moving message up. Clearing cached tiles after import. commit 3192adf95192554c194c9e0a8aeb149822462682 Author: Roland <roland@learningselfreliance.com> Date: Thu Mar 3 21:37:13 2022 -0700 Removing osm2pgsql from api docker because it seems to be unsupported. commit 0e10522f3cdc7edd374c14f230bb2dbd68ab1a94 Author: Roland <roland@learningselfreliance.com> Date: Thu Mar 3 21:36:39 2022 -0700 Disabling import button for docker. Adding download instructions. commit e1d8f55d159fb4d4ef2ece859ef7cc1059885791 Author: Roland <roland@learningselfreliance.com> Date: Thu Mar 3 21:14:55 2022 -0700 Map files can be imported on RPi. commit 3128530e09968430a71b27cd2e75a567f85a708b Author: Roland <roland@learningselfreliance.com> Date: Thu Mar 3 15:31:54 2022 -0700 Adding shm_size to increase size to render large map tiles. commit f3cf80fae0ce46f44ea9ac5033f68a32d0197ae3 Author: Roland <roland@learningselfreliance.com> Date: Wed Mar 2 10:39:18 2022 -0700 Can get status of map imports. commit 3b7bbce004cd55a5d89980c2c09124b72c450bc6 Author: Roland <roland@learningselfreliance.com> Date: Tue Mar 1 20:51:13 2022 -0700 Adding map link to 50x.html. Swapping links using hostname to make it easier on the user. commit a8b6b60fe182233f40c2f1ecd825636c3d37fd23 Author: Roland <roland@learningselfreliance.com> Date: Tue Mar 1 12:01:50 2022 -0700 Moving Inventory link to end. commit c5346b8dc2ff83262d6fd153130912809dfd7574 Author: Roland <roland@learningselfreliance.com> Date: Tue Mar 1 12:00:47 2022 -0700 Squashed commit of the following: commit 39f0b8f3fc049fffd140d551cd91d7d9dba6d416 Author: Roland <roland@learningselfreliance.com> Date: Tue Mar 1 12:00:13 2022 -0700 Starting map refresh function. commit 2fac9ddf07e378389ddd85ca972f33b3323d30ac Author: Roland <roland@learningselfreliance.com> Date: Mon Feb 28 16:43:47 2022 -0700 Updating readme with map requirements. commit 8325e1ea89537128329e313ab4b5588f8809d1c9 Author: Roland <roland@learningselfreliance.com> Date: Mon Feb 28 16:05:35 2022 -0700 Fixing order of map db init scripts. Creating index.html to display map. Using :8084 index file to display the map. commit 8f81539dd9d25fc26607f472c156d41ddcfc11f5 Author: Roland <roland@learningselfreliance.com> Date: Sun Feb 27 19:17:14 2022 -0700 Installing apache2. commit c7aaef07f8291070aeaf5e7df553e34aca0719ec Author: Roland <roland@learningselfreliance.com> Date: Sun Feb 27 18:41:07 2022 -0700 Always configure postgres extensions. Initialize gis with dc. commit aa725ed4b8ec46d17accebd96dd41508ea2d9caf Author: Roland <roland@learningselfreliance.com> Date: Sun Feb 27 15:48:20 2022 -0700 Map is displayed as iframe. commit de9bb92dcaf44996e7727adce7774f01895ddf52 Author: Roland <roland@learningselfreliance.com> Date: Sun Feb 27 12:16:56 2022 -0700 Running script in openstreetmap directory. commit f277388095d2221aa6407643ab655f26dc349eb7 Author: Roland <roland@learningselfreliance.com> Date: Sun Feb 27 12:06:41 2022 -0700 Bumping version for map. commit afd4aaeffb2aced2dc5c1a8f94038885c8287265 Author: Roland <roland@learningselfreliance.com> Date: Sun Feb 27 11:59:31 2022 -0700 Checking that the installation script is on an rpi. commit a799382ceac3c201fc01d41fd63e37607282831a Author: Roland <roland@learningselfreliance.com> Date: Sun Feb 27 11:49:21 2022 -0700 Only running mod_tile install once. commit 0b98bb19cd4d86c9630a863cab4eb46410d1c9d1 Author: Roland <roland@learningselfreliance.com> Date: Fri Feb 25 11:31:21 2022 -0700 Building out map install script. commit 7c799775c11df573fa6cd65451adea28c0832cb0 Author: Roland <roland@learningselfreliance.com> Date: Thu Feb 24 12:16:53 2022 -0700 Creating OSM installation script. commit d3b93a3ee97d1ae4a00b6a0423ed59ea8b897ccd Author: Roland <roland@learningselfreliance.com> Date: Tue Feb 22 15:39:04 2022 -0700 Removing unused map module.
2022-03-08 13:26:03 -07:00
RUN ffmpeg -version
# Install Deno runtime for yt-dlp. yt-dlp's EJS challenge solver requires Deno >=2.3.0; pin
# v2.8.3 to match scripts/upgrade.sh and docker/archive/Dockerfile.
RUN DENO_VERSION="v2.8.3" \
&& ARCH=$(uname -m) \
&& if [ "$ARCH" = "aarch64" ]; then \
DENO_ARCH="aarch64-unknown-linux-gnu"; \
DENO_SHA256="d4589cc1ffcbf1995c92a0127d932aaf832ac70cfdcc6d5b7bf38043cf303575"; \
elif [ "$ARCH" = "x86_64" ]; then \
DENO_ARCH="x86_64-unknown-linux-gnu"; \
DENO_SHA256="30455b845ffa6082209c3590269c910ad3b7efdf28c9879afd4006c47ae54197"; \
else echo "Unsupported architecture: $ARCH" && exit 1; fi \
&& curl -fsSL "https://github.com/denoland/deno/releases/download/${DENO_VERSION}/deno-${DENO_ARCH}.zip" -o /tmp/deno.zip \
# Assert the hash to avoid malicious or invalid files.
&& echo "${DENO_SHA256} /tmp/deno.zip" | sha256sum -c - \
&& unzip -o /tmp/deno.zip -d /usr/local/bin/ \
&& chmod +x /usr/local/bin/deno \
&& rm /tmp/deno.zip \
&& deno --version
# yt-dlp needs cache directory
RUN mkdir /.cache && chown -R 1000:1000 /.cache
Python packages install from hash-locked requirements; yt-dlp still floats (#913) * Python packages install from hash-locked requirements; yt-dlp still floats Every pip install trusted PyPI over TLS: requirements.txt pins only top-level versions (some with >=), the controller's are almost all >=, and dependencies were never pinned, so a forged certificate (as the eMudhra key compromise allowed) could change code in the API, the controller, or the archive container. scripts/lock_requirements.sh generates a universal, hash-locked .lock beside each requirements.txt, and install_python_requirements.sh installs it with --require-hashes at every site. yt-dlp is left out of the lock and upgraded afterwards so devices still get YouTube fixes; its dependencies are locked. The unpinned `pip install --upgrade pip` steps are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * The unhashed yt-dlp install cannot replace locked packages, and nothing builds from source Review of #913 found two holes. The yt-dlp step ran without --no-deps, so pip's only-if-needed strategy would still fetch an unhashed dependency whenever yt-dlp's metadata asked for a version the lock did not install. And srt is published only as a source archive, whose build installed setuptools with hash checking off on every install. The yt-dlp step now uses --no-deps (its dependencies come only from the lock), both steps are retried as the image builds did before, and installs and locks are wheels-only. srt 3.5.3 is vendored unmodified as wrolpi/vendor/srt.py (MIT), verified against the sdist hash the lock pinned; webvtt-py's SRT reader rejects WROLPi's own test file, so it is not a replacement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 12:00:29 -07:00
# Install dependencies from the hash-locked requirements.
COPY requirements.txt requirements.lock /opt/wrolpi/
COPY scripts/install_python_requirements.sh /opt/wrolpi/scripts/install_python_requirements.sh
RUN /opt/wrolpi/scripts/install_python_requirements.sh pip3 /opt/wrolpi/requirements.txt
# Install WROLPi.
COPY main.py /opt/wrolpi/
COPY wrolpi /opt/wrolpi/wrolpi
COPY modules /opt/wrolpi/modules
ENTRYPOINT [ "python3", "/opt/wrolpi/main.py"]
2025-12-16 19:20:04 -07:00
# Log level controlled by LOG_LEVEL env var or wrolpi.yaml config (default: info)
CMD ["api", "--host", "0.0.0.0"]