Commit graph

4 commits

Author SHA1 Message Date
Roland
00d7816cfd
repos: pin the repo's host once, and only exempt its own host and port
A submodule on the repo's host was allowed unpinned, on any port.  The
owner of the repo's domain could serve the clone from a public address,
then answer the submodule's lookup with an internal address (DNS
rebinding), or point it at another port on that name.

The repo's host is now resolved once per download, and every fetch
(ls-remote, clone, fetch, submodule update) is pinned to that answer.  A
submodule on the repo's host *and port* is fetched from those same
addresses, still allowing a LAN forge the user chose; another port on
that name is checked like any other host.  Tests no longer use real DNS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 17:38:45 -06:00
Roland
3b6e962dfc
repos: never fetch submodules from WROLPi's own network
A repo's .gitmodules is chosen by whoever controls the repo, so a
submodule URL could make WROLPi fetch https://localhost:8443/...,
https://10.0.0.1/..., etc. (SSRF), and show the response's error.

Every submodule URL is now checked before git fetches it: relative URLs,
the repo's own host, and hosts whose addresses are all public are
allowed; anything else defers the download with the reason.  Nested
submodules are checked out one level at a time (instead of git's
--recursive) so each level is checked, at most 5 levels deep.  Submodule
fetches do not follow HTTP redirects, so a checked host cannot redirect
git into the network.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00
Roland
03e3476631
repos: history, Download ZIP, and submodules
* GET /api/repos/<id>/log: the checked out branch's commits, newest
  first, a page at a time, with the total.  The repo page has a
  Files | History toggle (the tree is only listed for Files).
* GET /api/repos/<id>/archive.zip: `git archive HEAD` streamed to the
  browser; the file name is reduced to safe characters.  Everything is
  read from the database before the response starts, because the
  request's session closes once a streamed response begins.
* repository.submodules (default off): submodules are checked out with
  the repo's mode and the same git restrictions, and removed when the
  option is turned off.  A submodule that cannot be downloaded (e.g. an
  ssh URL) does not stop the repo from updating; the download is
  retried with the submodule's error.
* The README speed test bound is loose enough for a busy machine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00
Roland
b3255e715c
Add a Repos module that mirrors git repositories (backend)
A Repo is a Collection (kind='repo') plus a `repository` row, kept up to
date by a recurring Download handled by the new `git` downloader.

* Full history by default (all branches and tags, no --prune; a force
  push saves the old HEAD under refs/wrolpi/backup/); snapshot mode
  (--depth 1) and a pinned branch are optional.  `/tree/<branch>` URLs
  select a branch.
* git runs HTTPS-only, never prompts, never runs hooks, and checks out
  symlinks as plain files.  Upstream disappearing defers the download
  with git's error; the local copy is kept.
* Repos are cloned to `repos_destination` (repos/%(repo_tag)s/%(repo_name)s)
  and the `repos` directory is always ignored, so repo files are never
  indexed.
* repos.yaml is the source of truth (import, dump, backup restore).
* /api/repos: list, create, get, edit, delete (files kept unless
  delete_files), update now, and a per-directory file tree.  Tagging uses
  the collections tag endpoint.

Also:
* run_command/process_runner can merge stderr into stdout (git writes
  progress to stderr).
* move_collection deletes an old directory that only holds empty
  directories (the move left an emptied .git behind).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00