A submodule on the repo's host was allowed unpinned, on any port. The
owner of the repo's domain could serve the clone from a public address,
then answer the submodule's lookup with an internal address (DNS
rebinding), or point it at another port on that name.
The repo's host is now resolved once per download, and every fetch
(ls-remote, clone, fetch, submodule update) is pinned to that answer. A
submodule on the repo's host *and port* is fetched from those same
addresses, still allowing a LAN forge the user chose; another port on
that name is checked like any other host. Tests no longer use real DNS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A repo's .gitmodules is chosen by whoever controls the repo, so a
submodule URL could make WROLPi fetch https://localhost:8443/...,
https://10.0.0.1/..., etc. (SSRF), and show the response's error.
Every submodule URL is now checked before git fetches it: relative URLs,
the repo's own host, and hosts whose addresses are all public are
allowed; anything else defers the download with the reason. Nested
submodules are checked out one level at a time (instead of git's
--recursive) so each level is checked, at most 5 levels deep. Submodule
fetches do not follow HTTP redirects, so a checked host cannot redirect
git into the network.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* GET /api/repos/<id>/log: the checked out branch's commits, newest
first, a page at a time, with the total. The repo page has a
Files | History toggle (the tree is only listed for Files).
* GET /api/repos/<id>/archive.zip: `git archive HEAD` streamed to the
browser; the file name is reduced to safe characters. Everything is
read from the database before the response starts, because the
request's session closes once a streamed response begins.
* repository.submodules (default off): submodules are checked out with
the repo's mode and the same git restrictions, and removed when the
option is turned off. A submodule that cannot be downloaded (e.g. an
ssh URL) does not stop the repo from updating; the download is
retried with the submodule's error.
* The README speed test bound is loose enough for a busy machine.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Repo is a Collection (kind='repo') plus a `repository` row, kept up to
date by a recurring Download handled by the new `git` downloader.
* Full history by default (all branches and tags, no --prune; a force
push saves the old HEAD under refs/wrolpi/backup/); snapshot mode
(--depth 1) and a pinned branch are optional. `/tree/<branch>` URLs
select a branch.
* git runs HTTPS-only, never prompts, never runs hooks, and checks out
symlinks as plain files. Upstream disappearing defers the download
with git's error; the local copy is kept.
* Repos are cloned to `repos_destination` (repos/%(repo_tag)s/%(repo_name)s)
and the `repos` directory is always ignored, so repo files are never
indexed.
* repos.yaml is the source of truth (import, dump, backup restore).
* /api/repos: list, create, get, edit, delete (files kept unless
delete_files), update now, and a per-directory file tree. Tagging uses
the collections tag endpoint.
Also:
* run_command/process_runner can merge stderr into stdout (git writes
progress to stderr).
* move_collection deletes an old directory that only holds empty
directories (the move left an emptied .git behind).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>