Installed Debian systems keep Caddy (#831)

* Installed Debian systems keep Caddy

The Debian installer purges every package listed in the ISO's
live/filesystem.packages-remove.  live-build lists there every package
installed after the package-lists install pass, which includes anything a
chroot hook installs with apt -- and the hook installed Caddy that way.
An installed system therefore had no Caddy: wrolpi-bootstrap failed copying
the Caddyfile, the API/app/Kiwix were never enabled, and HTTPS never came up.

Caddy now comes from package-lists/, with its repository and signing key in
config/archives/ (installed into both the build chroot and the image).
check-iso.sh fails the build when the installer would remove any package,
and can be run by hand against any ISO.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Debian ISOs install Caddy from Caddy's repository

Moving caddy into package-lists/ made apt read config/archives/ before
ca-certificates was installed, so the HTTPS fetch of Caddy's repository
failed certificate verification.  apt ignores a failed repository, and the
image silently got Debian's much older caddy 2.6.2 instead of 2.11.

ca-certificates is now part of the bootstrap, and Debian's caddy is pinned
to -1, so an unreachable Caddy repository fails the build and installed
systems never switch to Debian's caddy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

debian-live-config/README.md

@@ -17,8 +17,10 @@ sudo ./build.sh                # build from 'release' branch
 sudo ./build.sh -b master      # build from a specific branch
 ```
 
-Requires Debian / Ubuntu with `live-build` installed.  The resulting ISO is
-written to this directory as `WROLPi-v${VERSION}-amd64.iso`.
+Requires Debian / Ubuntu with `live-build` and `libarchive-tools` (bsdtar)
+installed.  The resulting ISO is written to this directory as
+`WROLPi-v${VERSION}-amd64.iso`, and `check-iso.sh` then checks it for
+problems that would only appear after the Debian installer runs.
 
 Flash with Raspberry Pi Imager, Rufus, or:
 

debian-live-config/build.sh

@@ -73,6 +73,8 @@ set -e
 # usual graphical/text installer flow.  GRUB gets "Install" + "Graphical
 # install" + "Live" entries.  noswap prevents the live system from
 # activating swap partitions found on the host machine's internal disks.
+# ca-certificates must be in the bootstrap so apt can reach the HTTPS
+# repositories in config/archives/ before the package lists are installed.
 lb config \
  --binary-images iso-hybrid \
  --mode debian \
@@ -85,6 +87,7 @@ lb config \
  --updates true \
  --security true \
  --apt-recommends false \
+ --debootstrap-options "--include=ca-certificates" \
  --firmware-binary true \
  --firmware-chroot true \
  --win32-loader false \
@@ -183,4 +186,8 @@ if command -v blkid >/dev/null 2>&1; then
 fi
 echo "ISO sanity checks passed ($(numfmt --to=iec "${ISO_BYTES}" 2>/dev/null || echo "${ISO_BYTES} bytes"), iso9660)."
 
+# Problems that only appear once the Debian installer has run (e.g. packages it
+# would purge from the installed system).
+"${SCRIPT_DIR}/check-iso.sh" "${DEST}" || exit 1
+
 echo "Build has completed. ISO output ${DEST}"

debian-live-config/check-iso.sh

@@ -0,0 +1,54 @@
+#!/bin/bash
+# Check a built WROLPi ISO for problems that only show up after the Debian
+# installer has run.  build.sh runs this on every build; it can also be run by
+# hand against any ISO:
+#
+#   ./check-iso.sh WROLPi-v0.30.0-beta-amd64.iso
+#
+# Requires bsdtar (Debian package libarchive-tools; built into macOS).
+
+ISO=$1
+if [ -z "${ISO}" ] || [ ! -f "${ISO}" ]; then
+  echo "Usage: check-iso.sh ISO" >&2
+  exit 1
+fi
+
+if ! command -v bsdtar >/dev/null 2>&1; then
+  echo "ERROR: bsdtar is required (apt install libarchive-tools)." >&2
+  exit 1
+fi
+
+TMP=$(mktemp -d)
+trap 'rm -rf "${TMP}"' EXIT
+
+# extract PATH: copy one file out of the ISO into ${TMP}.
+extract() {
+  bsdtar -xOf "${ISO}" "$1" > "${TMP}/$(basename "$1")" 2>/dev/null
+}
+
+FAILED=0
+
+# The package manifest is always present; failing to read it means the ISO (or
+# bsdtar) is broken, and the checks below would pass vacuously.
+if ! extract live/filesystem.packages; then
+  echo "ERROR: could not read /live/filesystem.packages from ${ISO}." >&2
+  exit 1
+fi
+
+# The Debian installer (finish-install.d/14remove-live-packages) purges every
+# package listed in filesystem.packages-remove.  live-build lists every package
+# installed after the package-lists install pass -- including anything a chroot
+# hook installs with apt.  WROLPi has no live-only packages, so the list must not
+# exist.  Install packages through config/package-lists/ (and config/archives/
+# for third-party repos) instead of apt-get in a hook.
+if extract live/filesystem.packages-remove && [ -s "${TMP}/filesystem.packages-remove" ]; then
+  echo "ERROR: the Debian installer would remove these packages from an installed WROLPi:" >&2
+  sed 's/^/  /' "${TMP}/filesystem.packages-remove" >&2
+  echo "Install them via config/package-lists/ instead of a chroot hook." >&2
+  FAILED=1
+fi
+
+if [ "${FAILED}" -ne 0 ]; then
+  exit 1
+fi
+echo "ISO install checks passed."

debian-live-config/config/archives/caddy-stable.key

@@ -0,0 +1,109 @@
+-----BEGIN PGP PUBLIC KEY BLOCK-----
+
+mQINBFb+quEBEACl3/YkFekflvauEASL+neZjCctYWyt57Dv5AdRmUPO4zkxylLG
+d/9JawlUfHuYYU4emz7940S2wR8kbBimiLgxMqyGP5+RQnggNZhjYIXoqkkh0G8v
+purq+58d+VNYf0LWnWlwuJC0dtpi4bPqZTc5ST4QOItFK0s7F2xZJyOkuAPDI782
+pGMR8UzpburHt9JwIUv1oOHFfFA/4HFQ++A6RF9bjYQFNMreaXsvMKIA5VQKcnDd
+SbKEfKnr0bwGr59MsnsQBgr1Ats2W722jIs89YevBanS6n0FWeiSxUqUrNypTLkL
+QHVPlK7Agq1XGWUhu55clFC6loQXboph9BhnSxSn9Kou4toXDQj6AMDuLGcV+VQ+
+fVfSZFXsp/evzqkjbc0jsUTVOZgZhhRP8DD+vjkzJFfCq/tAWu4qgqnOwE9kEEQL
+MXsnsZNSYS3MvWnQFPBmg0B483iKxaA/Oe89WckTnjt+jlpAKhOoS5ZURdOtwv1i
+yrKlYiXYMQCMhOd3BCw5RELb7Qtpz+gBaOoxQMMyRRYwKiturpQdV53FVvu/re/x
+xXVuxRyRI2Yo94ba3a5bEGjR3CNjvx7LuGuWplYyzDWn+OXa/HiTqWM153ho+oUl
+s3ntiHQ16jtgyhcNSuMffCcMLYanfmB+2m4HZmkl97vs7XvclClEXNV6VwARAQAB
+tCpDYWRkeSBXZWIgU2VydmVyIDxjb250YWN0QGNhZGR5c2VydmVyLmNvbT6JAjQE
+EwEKAB4FAlb+quECGwMDCwkHAxUKCAIeAQIXgAMWAgECGQEACgkQFVttecpW6jRx
+cQ/9GHdVoYf15rcU0ip3Vw1MF06ndRxLmilgBvdweZ5NcRttbu8ESh+MP59Z0gOp
+0uX/CqBnqZb9E2vbYyly1plq5GwP4tcCHwwkyOT1doGcyP1XylPkJkieP9YUWsIA
+3oG/wCsqxxwVYzwvm0opBdrNf6pAYg2tGNCqxh8bmYPDaReu3t2LZ6qeJ4obhYTx
+IwAh36oF5dVG5OW2dnMNFVpjoEgCavvTNTcJCgonLct6Zl+Q7xptJyBv3LS8L674
+V2nxcoLvtTjXG86D3yPJvD1I5WYPEZMpHznj1PEztgOrvLo+Fyu+T5vCHqfTY6mG
+89BXz8L4o5aBr2uY+ZV5oQa6GuV8GIiiWIZNyDwXTnUiW/GsUFNwg0AP05rva8fF
+2a3ybwsq/Sv2nraKQMpYRltBQZkg+l5nZD7znHpYBfJiH6eW3/7ft3w8OptiIcu6
+87UzhI28yoFSNE+85V3sz7JphZ/XFaU2ApESO1ahjDzP96w4u0HeSds6tbkR3OlC
+ECcFOmX79MhWfjDaVNnknBqGzjy1JdQ0ZKNWMZRVyxZ9fKiZxFw+q40Sta7ynxfH
+p4v0bM8vDLM3cxxOj38U5jsP/ChctyZO3P0nCEzIAR9kvumc5PSqpjiqWlbaHsxa
+fXohi3LAIi/clgIOV7bIVRmTz6b61Ngf+C8VYzlUph0ygS25AQ0EVv6q4QEIALhF
+nG37+5lJVqrxeI27XTsMQlXMPHcFq10nynZAPxJD4BZXlDxLeQt7oOKAcsx+sGUY
+BLRTUM9QnXx7acXBV8TRxVRtRCjwSbjXyn2KUZDlCoh2fff5Mw1AyBmKBxBGvtsF
+F0sbqshAcbX+OEwyi81pWNYu822S3a8932TkG0gO5parM+NMwJCaxjQf4uKb2Jmc
+6ah5jFGmgd8umW5EBAnk0eLle0ttcbiWC1mKHc6Jog+VGuJwKWtv6AERwleIxy/g
+BaeCrUxl1Ymxb8NW60jbjMEx6lhUUhcaj3wP/aLUfEWKVR3xaGwvxEACOf5t9AzS
+OOJ31YPJy3K+xjayQu8AEQEAAYkDRAQYAQoADwUCVv6q4QUJDwmcAAIbDAEpCRAV
+W215ylbqNMBdIAQZAQoABgUCVv6q4QAKCRBYhKGLd5yF0V65B/9VML4Aacsms2oo
+mQPA00vItQ164JU3nDpFLPHc5lhj89ZN2dxnDu2VKpfvM2/UkPzZvX+ifMGm8pqV
+JV8sLGUttXvGuJjin/yhKkikj+eHeEeVlLxSFQkqHJRpnJYRQffbQgt0UDfvGIoF
+PKuSJifrXIa2UJ9VLMuasOEkPQfQOLvZOa7BQ0XlrXWrWluhkrgEW9Yi7o5nM+Re
+Ys0rsduauJnpvUiAJLsAXwGmseKg3cIzkJhj2ea4FVl9A7JqsJO5zYAam/seOlEQ
+f5b7a+oTRaoqdd4onIoD+SZ94MRSM0MCJjKDduf7ZzMBjrQlBwO37d5diVh1g6wb
+4VbH8v+UyyoP/im0j/XqIfJdyRIP9/zgsmz+5rYTUzMJaUVI/Y6IQs0pnRRW14hm
+z7DcJRNIEQO1njl9VGCXYZqcCVSbhc73+tsjl7tFxJlly6YzM66nKnXu0N4CSVnl
+/oPyh70zLmXzsShLFAfUZBD/nTMk2oIGg65ifMg7OCmKIbYWSqJxg/8LVEKwxpVo
+CA6cpyI//V7nQS53ZJ7QguNq5Uk/1dTKYbHx9EcCuHfc/8cypMhYjYK47uqTzrcW
+57ELqf6+/4wZudeBrc7TBF5wmvEzpYg6GZNt9ZmS7dEgu4kpqYkqOXjENUNhloPl
+GxyIB3Zu/pDyT4/bYBdJfPOdxGm3dINgt8sLtCk5+doqdG77w5uZ7dVaRnoSU8vO
+FvgUUiD2n4nOztXygtwBaj9FJtmOJ+R4P1bD2OTbxxqJ7FG2065JnvHnbbCCx/Sm
+jBEhkrvCkkordHLI2UQ+YuAeBwjOMHvzxd9FQFGDxF+vmBg1OoGcpzL5zsir0oIJ
+H+OJtra9g9Ax66sW5DSZhek+c4GC2NlEjxO+HBEq3XgaTcuRazqsf5QXyxcL8DVT
+UmhLlSgSSHHCzDeZvS1nSOrToUZbINXLpD4OxCA1aMq5pzaHQyUzece7esibc99X
+VNt4OIr10cI8c3xX/318kOO+DFPRcHDPa1kF0F+HIRvHZt5LJ+yrz9UvuQENBFb+
+quEBCACr8U+xJP7bZHjWmHZxoHM3pGrhyvLiYAyNBRWgymB70ocpTxmnN2DpYOf4
+eeZ/rZs3CTpcZIUJwUJDCD15AjjCnQigTptQmBlsaQMDYU23QOfxKcPpNTNtn0WV
+xX00yGNxgfqYWL+6Tt4/qQKNGXnwIdH8ctL/8p0IZJJ6Nns+Y1szhUNK3TMX9Xzy
+87XGBEamJqqt426kJdtbyEZ6sv+r5XhjfH7GSJdTFqT7aR9iRt1b2ewHimW893X7
+W6jqmUcHhkI8sVklr6e67PsyCBn4Z1gM6ihyPyZylBT8YgQs6pUXcZyHu33GuPSV
+2fzJvvin5xlBQQCfZDc11ikqB0SBABEBAAGJA0QEGAEKAA8FAlb+quEFCQ8JnAAC
+GyIBKQkQFVttecpW6jTAXSAEGQEKAAYFAlb+quEACgkQUxprIPoFinDU3Qf/Yug5
+GvNF+lDmFD65PDajGZKNs+1KFomH5lfs/roy2UNyKbo5XOrCzBeme8dxO61Fxa9v
+U/aJPHPhUw4VPzA23c2XZ9UONdj7MeLvXnk1ZWOR4Q2L0dCSJ6VyL5FlELkqKzub
+TH51HZecEiSTViR0SYXIHqFEDr4NOedXU3Jyo0xRdwqzdxFXBShfAiVzzqQEto8K
+XWzafxnA3Ry7FAWiLWLyQcHdrtBEprtflZNN1r0JgWvPxuWwCk+XmJXbJGMhyaz1
+9ohPs0Azutnc1I4m9Ywuu1o4e9IX9hh+xR2TCWA3fJaNx2Ih8n0lu4gs6FjP/YgZ
+VXN2dF1AVvQyW4oTOiltD/47IdGKmumB3RDDVOR37n28xwyJY97stSqTPxGv3YNE
+Fk6M3Q9TqVqnDGUQdhnfIU0yKWfCT4JSc3drgs6KUGAIrgud0c4JTIamovUFwFRp
+yOVHEXKgr/DpPhSZesoOJGH9ZQb0Si9POAGVb9hEVCbywXbk9bH0417iVoru36JV
+OxUgXIxWz4/JqQs4zgZdciHdy8pRvkRPJ9T4CUCJDtOVpkToGAyDN4sdu37+Riiv
+IODBhyVl4R7DdQafaOFs+i86Il2Yh/9Qzx4MC5aGe4OqcNHDV2FnMVfRNFITu69o
+6MtKxy/99BSqTf3DoCjjFm2eC5pKk/L+Qhuv97PGLGHEwG35jUtDg1X+4B6qLGQQ
+stGHeW9Db/OA1Gaq4nDhm+vJKzz19REohjQDWMr5HHT6ATVHcBWYhJSeupc3H5K+
+dXcesQqdohKr75MSupiPUA/F8/9NwmWF7eFMenNS2eaG+QNCYIkde4OjzlxIlvnl
+D5WElVj11QcIVZOJx9+cQ1hsBw/IYH2hzYMiiSGHCqPel4Ym8ecc19RXrxF7/HaW
+bzAlA+7IEQ8CCalGWcDe2mFRJS0YvLzMxN8sbDQ7IVOvr46buFqguOop9MaUeZLX
+FVIW0hhVAY43EojnMBTsGtOdqlWAlc6q7jQPFpNPv7GY0IPp/2PmUn5sOe4LHT0h
+FbkCDQRf65yQARAArKoTSdtlrz/omzlMa+VEaL+lvn1zpwL7KoHd3urM64gn/rVb
+3IJeWJBFlIHfZ0ZeyoNrfvk+0aUTmnuJ3g83B6DHYLh8U3o9Z3E7VvCE6etxSi/p
+geCCRyNoUmppahr8S8UNSAQp3R3VYbM0eWaQzQ/qm4nyVs9bdJ5a2/5XzcG9L0eR
+v//jcco2TYndxX/5E1syx8RHkJEAVhgez+XJPaalhTl2KyXOThUNltHuLLNcitQQ
+BB2ld7yK0ViMV/zHMB6oOxR59quvjzlRKdNPFC1wEpfTqu4eJBRII6zsdHpwMmLf
+fZxNdPuRJurei6r2cHyhrGGXYjbSRUFap1HLNUDC0a+ORYhi5S14imC7MCql+wG/
+ogn9bkbmPD4cRQlIJGvquULXP1qAYIqXnc1NXiAepFjYalg+vYo03e5NWiOKCcIL
+0EQ/R5GqPzvpf3RjuJ1C4N1r/5BocfqG1R79bnWPlEtVaywItNnAc9mZ4e3HPpTo
+g4S2VK6yjsg3gdCx1qOWY4/6R83LWApU5b1+8XYelS9jZw3CNIA73oLcrjDjP/KQ
+dyj/oeFlwBv8zJ0wo/JwjyNQMpr74OTzzG+r8CxEf76uLlYhjef8ciQPLEZfkoLL
+EB9TXG4aSiye7uIHGjPZhUhXkZTCN/vKw6aCRvsUBxvveFScJ0bFxtaJCh8AEQEA
+AYkEbAQYAQoAIAIbAhYhBGV2DFHt6iAXzqLKFRVbbXnKVuo0BQJpQDmmAkDBdCAE
+GQEKAB0WIQQvXDvpiGrNKRMpnvurofm4h1pmYQUCX+uckAAKCRCrofm4h1pmYWSC
+D/9hcQs+NHr/7EetKMz0YP3jKgMfDPLZsAAoz5POLQma8k0UPXUNpysyFiEL7ay5
+Y5dDlNwgi4B0R5eZRI04Hg0n76wi4LLOPHygHpwZRyxk82jKnSOm7NX4OyHoeXWU
+HfQq3+sSwu7QUoxM2qmj7HrsfbRHSxGx5/KCZUAIMpluMAZKPspq4maY7ONx8Yq/
+Ih5/x/qEiXuLTsGhpi82zGOSzH371FlSp6GnIyMfdl4lZVFcqZIdtokmtLqbM77H
+rvoYsrcATMsVPfXbEcDkMtgWXdz/7QsKZwyPIUmQswU8rqOlzYMCx2c71LjSnKl2
+SuJMT0U1Rsrq/Uta2G2ap5j3+Fsu8BbcEVL21b3ugVXXO7/c1rE5BLUxCeAboRh0
+sAU3RisvAiSYETfCtrphgj8Z1Db6AjrKlusSCU6dm2jNZ0VmX9VLzY4FYcMibQJa
+zJNcXZgw6lVDvNGOaeyZ9htVz5UDlR3I8oxbTOLCFj6p60c08weMUwGdabrIdxDs
+rNvuCBi4Euz+s9KvuAscLZ5t9mbnkA1skAOYDjmQKHW/BwpLurCCIhfsxrazTheO
+cbCgKCVT7eVbx+Mziivk22MhwDf1I6X9Pc1607Zq5x8IwfDw/lKtjm3q/2X+UODB
+p80AMv0JMPLLnIiMYCwvZAjcCSBUVCxzE1LABWgKvi/fiAkQFVttecpW6jTfPA//
+e23DyHzgHij5rShaP4q+x+rNfPoGDzU2s7QDlii0NcAqb3FnE0NMSL1d4jJVcQtg
+/BrC4Lgqz5+FmUwkciycBIGLiiqw2l7Uav6WJpUDAcQ197tSOwFlSRq4eIZwzM2/
+cHsjZymNC3krKLmoD/wDkffAYwvQfHVl1gqCY5HFJ7jge95XhL0vTsvEk2Q3+ZrV
+ovGP7tyEo/5RP7MNHdOOWk6tK80FwwN8MbkDURxslRoEYcXZ1G9LieL/57D63UWe
+xCjDWlRCa90dpdTvwVB01CM5OVvtJlTKY5wF42LOup6cvV5bdRPPZ2anxT/bzWP9
+Z6x1383zLKF0lYJSXhm+SHQt6D6c8SUUJV6l6E1VBAZ8+yXSCulJ7bbetKOnlmqq
+SU7U3MwFbGkPK4zDx2h/9Ai0fTAfj/OlXvQx2f4VuC6V2sAzaIoW5hmdtUCO8CJw
+5aoiz++oi6zcR3PSYx1ccAF0w+eajtO+G6w2BJ0V/r2Rh8oe49F4sm/m5fADZ9d6
+/BYCH68oAPjkdLPyywkamWUwSPDkzW85YvLkcpqZRrbBYFlEj8uHBHWmoa7YU9Wa
+TsFZHeZzy7BlX6wIEYt0ftpXkuG4MC08yCrluSQfDT/JFvH798UfHJCZu6iblp6A
+jYHjnr+6GNhoAm6i9Xa+Nm6BqqaSd//WFqYkJI4It3o=
+=wl92
+-----END PGP PUBLIC KEY BLOCK-----

debian-live-config/config/archives/caddy-stable.list

@@ -0,0 +1,6 @@
+# Caddy's official apt repository.  live-build installs this (and
+# caddy-stable.key, fingerprint 65760C51EDEA2017CEA2CA15155B6D79CA56EA34)
+# into the build chroot and into the final image, so caddy can be listed in
+# package-lists/ -- packages installed by a chroot hook instead are purged by
+# the Debian installer (see check-iso.sh).
+deb https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version main

debian-live-config/config/archives/caddy-stable.pref

@@ -0,0 +1,7 @@
+# Only ever install Caddy from Caddy's repository (caddy-stable.list).  Debian
+# ships a much older caddy; if Caddy's repository is unreachable, the build
+# fails instead of silently falling back to it, and installed systems never
+# switch to it.
+Package: caddy
+Pin: release o=Debian
+Pin-Priority: -1

debian-live-config/config/hooks/normal/9999-wrolpi.hook.chroot

@@ -106,13 +106,9 @@ python3 -m venv /opt/wrolpi/controller/venv
   /opt/wrolpi/controller/venv/bin/pip install -r /opt/wrolpi/controller/requirements.txt
 echo "Controller pre-installed"
 
-# Install Caddy from official apt repo.
-curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
-curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | tee /etc/apt/sources.list.d/caddy-stable.list
-apt-get update
-apt-get install -y caddy
-
-# Configure Caddy.
+# Configure Caddy.  Caddy itself comes from package-lists/ (repository in
+# config/archives/), never apt-get here: the Debian installer purges packages
+# that a hook installs.
 mkdir -p /etc/caddy
 cp /opt/wrolpi/etc/raspberrypios/Caddyfile /etc/caddy/Caddyfile
 mkdir -p /var/www

debian-live-config/config/package-lists/wrolpi.list.chroot

@@ -6,6 +6,7 @@ bind9-dnsutils
 build-essential
 bzip2
 ca-certificates
+caddy
 calibre
 cargo
 catdoc