Commit graph

16 commits

Author SHA1 Message Date
Roland
02eb8d70e4
Caddy install: run gpg --dearmor with --batch --yes.
If the caddy keyring file already exists (left by a prior partial
install), gpg prompts to overwrite via /dev/tty, which fails under the
tty-less wrolpi-upgrade.service ("cannot open '/dev/tty'") and aborts
the upgrade. --batch --yes overwrites silently, making the caddy
install idempotent in all four scripts that share this pattern.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 13:03:11 -06:00
lrnselfreliance
1acb148c90
Single file deno (#478)
* Singlefile runs under Deno 2.8.3; browser leftovers are reaped.

single-file-cli is designed for Deno.  Under node 18 (Debian 12/RPi OS) it
cannot reach the browser CDP on IPv6-enabled hosts ("localhost" resolves to
::1 first with no IPv4 fallback) and it never exits after a successful save,
blocking the download worker forever.

The native path and the Archive docker container now run single-file with
"deno run".  Deno is pinned at v2.8.3 everywhere because yt-dlp 2026.06.09
requires at least 2.3.0 (our node is Debian's v18, below yt-dlp's v22
minimum).  Deno >= 2.3 kills the browser single-file spawns (piped stderr
that single-file never reads), so the browser is run through
singlefile_browser_wrapper.sh which discards its stderr.

Commands are run in their own process group which is killed when they
finish, reaping the browser single-file leaves behind.  The Archive
downloader now has a ten minute timeout.  Install scripts install Deno
(previously only upgrade.sh and the image builds did).

* Singlefile writes to a file instead of dumping to stdout.

Deno >= 2.3 truncates large --dump-content output at exit; compressed
(SingleFileZ) singlefiles of large pages lost their ZIP tail, failed the
compression check in write_archive_files, and were destroyed by the HTML
prettifier (blank white archives).  single-file now writes its result to a
temporary file which WROLPi reads back.

As insurance, a singlefile marked compressed (<html data-sfz) whose ZIP is
unreadable is now stored verbatim instead of being prettified.

* Compressed Archive screenshots and readability images are no longer broken.

The index.html inside a compressed (SingleFileZ) singlefile references its
resources by relative ZIP paths.  Screenshots are now taken from the fully
extracted ZIP so those references resolve.  Readability images are inlined
as data URIs read from the ZIP (readability-extractor absolutizes relative
srcs against the page URL, so both forms are matched).

* Compressed Archives are created by converting the uncompressed capture.

The page is always captured uncompressed; readability, screenshot, and
title come from that canonical page (its images are inline data URIs, so
no ZIP extraction or image inlining is needed).  When compress_singlefile
is set, a second single-file run converts the captured file into a
compressed (SingleFileZ) self-extracting file.  The conversion is a local
file:// operation - the page is not fetched again, and single-file
preserves the original URL header when re-saving its own pages.  If the
conversion fails, the uncompressed singlefile is stored instead.

* Consolidating duplicate compressed-singlefile tests.

The write_archive_files tests (verbatim write, title fallback, broken-ZIP
guard, readability inlining) are combined into one test; the helper probes
and the setting-plumbing tests are each merged into one.  The do_singlefile
compression-failure scenario and the timeout constant are folded into the
main do_singlefile test, which shared their setup.

* Addressing review comments: removeprefix, PermissionError when killing.

* Bumping version to 0.24.1-beta.

* Install scripts assert the deno.zip hash to avoid malicious or invalid files.

The hashes are the official sha256sum assets of the Deno v2.8.3 release,
per architecture.

* Archive Dockerfile asserts the deno.zip hash to avoid malicious or invalid files.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 17:07:28 -06:00
Roland
b8a532c8d9
Key changes:
- Add MapLibre GL JS viewer with light/dark themes, terrain layer,
  and embedded static map page for coordinate sharing
- Add map pin support with persistent map_pins.yaml config
- Add map region subscriptions and downloader with progress tracking,
  sourcing pre-extracted PMTiles from a cold CDN
- Add map export to PNG and GPS coordinate copy via context menu
- Add MapViewer React component alongside reworked Map page
- Add map catalog and region-building script (build_map_regions.py)
- Remove OSM import/render scripts, leaflet files, Apache configs,
  renderd service, and osm-map-dumper Docker container
- Update Caddyfile to serve PMTiles and map static assets
- Update build/install/repair scripts for Protomaps dependencies
- Drop map_file table, add/drop map_subscription table (alembic)
- Adding button to skip primary drive mounting during onboarding.
2026-04-04 13:48:19 -06:00
Roland
fbb71e012d Replace nginx with Caddy for TLS termination and reverse proxying.
- Replace nginx with Caddy as the web server for both Docker and native
  (RPi/Debian) deployments. Caddy handles HTTPS termination, reverse
  proxying to all services (API, React app, Controller, Kiwix, Help, Map),
  and serves the media/download file browsers.
- Generate a Root CA and signed leaf certificates for HTTPS. The Root CA
  uses Name Constraints to restrict signing to local addresses only
  (.local, localhost, and RFC 1918 private IP ranges), mitigating MITM
  risk if the CA key is compromised.
- Add an HTTP landing page that guides users through one-time CA
  certificate installation on iOS, Android, macOS, Windows, Linux, and
  Firefox, with auto-redirect when the cert is already trusted.
- Add trailing-slash redirects for /controller, /docs, /media, and
  /download so bare paths don't fall through to the React catch-all.
- Add a Root CA download link on the Settings page under Configs.
- Update help.sh diagnostics for IPv6 listener detection and HTTP/2
  status codes.
- Remove all nginx config files and the https_proxy Docker container.
- Bump wrolpi-help submodule with certificate documentation.
2026-03-07 18:48:44 -07:00
Roland
531ec3d65e
Creating new /api/archive/upload which accepts a file from the SingleFile browser extension and converts it to an Archive. 2025-01-30 18:41:32 -07:00
Roland
af5d4dfd51
Disabling JIT for map database as recommended by mod_tile because of excess memory usgae. 2025-01-11 11:06:01 -07:00
Roland
8e4d173791
Converting map blob to use "custom" postgres dump format so restore can use multiple threads. Renaming map db blob from gis-map.dump.gz to map-db-gis.dump. 2025-01-02 12:04:03 -07:00
Roland
218fb3e760
Replacing Tag names with escaped tag names to avoid errors in Tags Directory. 2024-10-16 18:56:10 -06:00
Roland
86355b664d
Using older version of single-file-cli to avoid error. 2024-05-14 20:29:57 -06:00
Roland
2924867f55
Bumping single-file-cli version for all systems. 2024-05-02 09:09:21 -06:00
Roland
6c87c25e07
Adding openstreetmap-carto to git safe directory list. 2023-11-25 13:02:55 -07:00
Roland
a7805745ab
Combining help service install into a script. 2023-11-23 15:49:01 -07:00
Roland
844bc9468c
Installing help during image build. Using wrolpi.local hostname for all links. 2023-11-22 15:08:29 -07:00
Roland
57262442bd
Installing WROLPi help environment for RPi and Debian. 2023-11-22 11:21:49 -07:00
Roland
7fd8a8527a
Reworking Debian install with real system. 2023-10-03 19:21:12 -06:00
Roland
6e0f9a999c
Combining install scripts into a new repair script. Removing create_db script in favor of repair script. 2023-10-03 12:15:29 -06:00