* A submodule's working-tree `.git` names the git directory git uses for
it. Before any submodule command (update, or deinit when submodules
are turned off), and before an import, every `.git` below the top of
the working tree must be a file pointing inside the repo's own
.git/modules; a link, a directory, or a file pointing elsewhere could
make git use a config that was never checked.
* Import validates and sanitizes the clone where it is, then moves it,
and moves it back if marking it fails, so a refused or failed import
never leaves a clone moved without a Repo.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ownership marker held the Repo's URL key, which anyone can write, so
a copied clone, an extracted archive or a hand-made `.git` passed the
check and was reset and cleaned. Each Repo now has a random clone_token
(repository column and repos.yaml, so a Repo restored from the config
still owns its clone); the marker holds it, and only a clone holding the
Repo's token is updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A submodule git directory which borrows another (commondir) has no
objects of its own, and was skipped instead of refused, because
"has objects/" decided what a git directory was. The walk now follows
git's layout: under .git/modules every directory holding HEAD is a git
directory (whatever the submodule's name, even `refs`), which must not
borrow, must have its own objects and a regular config, and is
sanitized; nested submodules are found in its own `modules`, and its
refs and logs are never mistaken for git directories.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The checks covered .git but not the submodules' git directories in
.git/modules: a linked submodule config was skipped (and still read by
git), and a submodule's commondir or alternates were not refused. Every
git directory now needs a regular config and must not borrow another
directory, and nothing under .git/modules may be a link; otherwise the
update fails. A git directory is one with HEAD and objects (a ref named
HEAD is not one).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A clone with .git/commondir (git then reads another directory's config,
refs and objects, so the sanitized config is not the one read) or
alternates is neither imported nor updated.
* branch.*.remote/merge are no longer kept: WROLPi checks out with -B and
needs neither, and branch.<name>.remote can be a URL git uses to resolve
relative submodule URLs.
* An import directory is resolved and confined to the media directory
before anything about it is reported, so a link cannot reveal whether a
path outside exists.
* git_command also turns off automatic maintenance (gc.auto,
maintenance.auto) and core.alternateRefsCommand.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A clone being imported is controlled by whoever put it in the media
directory. A link at .git/wrolpi-repo or .git/config.wrolpi made the
marker and the sanitized config write wherever it pointed. The marker
is written with O_NOFOLLOW (after removing a link), the sanitized config
through mkstemp, and a .git/config which is not a regular file is
refused (by Import, and before an update).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST /api/repos/import/inspect reports what importing a clone in the
media directory would do, without changing it: its origin (an SSH or
git:// origin is converted to https://), branch, latest commit, commits
not on the remote, whether it is shallow, and where it will be moved.
It refuses special directories, Collections' directories, `.git`
files/links, bare repos, and partial clones. It runs no git command
which could run a program from the clone's config (the latest commit is
read with cat-file, not log).
POST /api/repos/import (confirmed, with a URL that is the clone's
origin) moves the clone into the Repos directory (the old location is
refreshed away), sanitizes its config, points origin at the URL, marks
it as WROLPi's, and creates the Repo and its Download. The Repo shows
the clone's commit and README at once; its first update makes it a
mirror (local work is discarded, and the page warns of local commits).
The Repos page has an Import button; the Import page inspects the
chosen directory, then imports.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* The sanitized config is written as INI by WROLPi (values quoted), so
no kept value is ever passed to `git config` as an argument.
* origin's url and fetch refspec are set with --replace-all (a planted
duplicate made `git config key value` fail), in both modes.
* format_repo_destination formats a Repo's directory from its name, Tag,
owner and host, so a directory can be known before the Repo exists
(Import shows it); Collection.format_destination uses it for repos.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An update resets and cleans the clone, and fetched from whatever the
clone's `origin` said, trusting its .git/config. A clone WROLPi did not
make (in the Repo's directory, or swapped in) could lose local work, be
mirrored from another repo, or redirect git with planted settings
(include.path, url.*.insteadOf, http.proxy, core.sshCommand, ...).
* WROLPi marks its clones (.git/wrolpi-repo holds the Repo's URL key);
an unmarked clone, or `.git` that is not a directory, is not updated.
* Every update fetches the Repo's URL (origin is reset to it).
* Before an update, the clone's config (and its submodules') keeps only
the settings WROLPi needs; a submodule's work tree must be inside the
repo.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The constant `repos` was always ignored, but the Repos Directory
setting accepted any relative template (`library/%(repo_name)s`,
`%(repo_name)s`), whose checkouts (and .git) would be indexed.
The fixed directory at the start of the destination is now the ignored
one (on config import and when the setting is saved). A destination
must start with a fixed directory, contain %(repo_name)s, use only the
repo variables, and not overlap another special directory; a
hand-edited config that breaks these rules falls back to the default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A submodule on the repo's host was allowed unpinned, on any port. The
owner of the repo's domain could serve the clone from a public address,
then answer the submodule's lookup with an internal address (DNS
rebinding), or point it at another port on that name.
The repo's host is now resolved once per download, and every fetch
(ls-remote, clone, fetch, submodule update) is pinned to that answer. A
submodule on the repo's host *and port* is fetched from those same
addresses, still allowing a LAN forge the user chose; another port on
that name is checked like any other host. Tests no longer use real DNS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`ipaddress.is_global` accepts multicast (224.0.0.1), deprecated
site-local (fec0::/10), and NAT64 addresses which carry a private IPv4
(64:ff9b::169.254.169.254, the metadata address). is_public_address
refuses multicast, reserved, link-local, loopback, private, unspecified,
site-local, local-use NAT64 and Teredo addresses, and checks the IPv4
inside NAT64, 6to4 and IPv4-mapped addresses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
git_env copied the whole process environment, and git consults its
environment ahead of `-c`: GIT_ALLOW_PROTOCOL=file re-enabled file://
despite protocol.allow=never, and GIT_CONFIG_COUNT/KEY_*/VALUE_* could
add an `insteadOf` that rewrites an approved URL to another host. Only
PATH, HOME, TMPDIR, TZ and the SSL certificate paths are inherited now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
They read `successful_import`, which lives in the shared context that
`async_client` attaches. They only passed when an earlier test in the
same xdist worker had attached it; run first, all 13 failed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A relative submodule URL is resolved by git against the repo's remote,
and can climb to another host: with the remote
https://git.example.com/owner/repo, `../../../10.0.0.5/x` is
https://10.0.0.5/x. Relative URLs were allowed unchecked.
git now resolves the URLs first (`submodule init` + `sync` write them to
the repo's config), and those resolved URLs, the ones git fetches, are
checked. A relative URL is never allowed by the check itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* .gitmodules is read NUL-delimited, and an unreadable record refuses
the submodules. A submodule name with a space (`[submodule "a b"]`)
made `--get-regexp`'s "key value" output misfile its URL as a path, so
the URL was never checked.
* Submodule URLs must have a strict shape every parser reads the same
way (no userinfo, backslash, query, fragment, or whitespace), so a URL
cannot be checked as one host and fetched from another.
* Each checked host is pinned (http.curloptResolve) to the addresses
that were checked, so DNS cannot change between the check and git's
fetch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A repo's .gitmodules is chosen by whoever controls the repo, so a
submodule URL could make WROLPi fetch https://localhost:8443/...,
https://10.0.0.1/..., etc. (SSRF), and show the response's error.
Every submodule URL is now checked before git fetches it: relative URLs,
the repo's own host, and hosts whose addresses are all public are
allowed; anything else defers the download with the reason. Nested
submodules are checked out one level at a time (instead of git's
--recursive) so each level is checked, at most 5 levels deep. Submodule
fetches do not follow HTTP redirects, so a checked host cannot redirect
git into the network.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* GET /api/repos/<id>/log: the checked out branch's commits, newest
first, a page at a time, with the total. The repo page has a
Files | History toggle (the tree is only listed for Files).
* GET /api/repos/<id>/archive.zip: `git archive HEAD` streamed to the
browser; the file name is reduced to safe characters. Everything is
read from the database before the response starts, because the
request's session closes once a streamed response begins.
* repository.submodules (default off): submodules are checked out with
the repo's mode and the same git restrictions, and removed when the
option is turned off. A submodule that cannot be downloaded (e.g. an
ssh URL) does not stop the repo from updating; the download is
retried with the submodule's error.
* The README speed test bound is loose enough for a busy machine.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Repos table (CollectionTable), New Repo and Edit pages (frequency,
mode, branch, description, tag, delete with an optional "also delete
its files").
* A GitHub-like repo page: upstream, branch, latest commit, a status
message when not yet downloaded or the last update failed, the file
tree (directories through ?path=, files in the file preview), and each
directory's README.
* Global search: a Repos section under Other with escaped README
headlines, Repos in the suggestions, and the search text sent to the
Other estimate.
* Settings: the Repos Directory field.
* CollectionTable columns may `render` their cell.
* Theme gallery: the repo summary, tree, and status icons.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A README comes from an untrusted remote.
* Escape the README headline: snippet() marks matches with control
characters, the text is HTML-escaped, then the markers become <b>.
Headlines are rendered as HTML (HeadlineText).
* Never follow a symlinked README (O_NOFOLLOW, regular files only).
* Convert markdown line by line with linear-time patterns. The old
multiline patterns backtracked polynomially; a README of blank lines
never finished, and runs of `<!--`, `