git_env copied the whole process environment, and git consults its
environment ahead of `-c`: GIT_ALLOW_PROTOCOL=file re-enabled file://
despite protocol.allow=never, and GIT_CONFIG_COUNT/KEY_*/VALUE_* could
add an `insteadOf` that rewrites an approved URL to another host. Only
PATH, HOME, TMPDIR, TZ and the SSL certificate paths are inherited now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A relative submodule URL is resolved by git against the repo's remote,
and can climb to another host: with the remote
https://git.example.com/owner/repo, `../../../10.0.0.5/x` is
https://10.0.0.5/x. Relative URLs were allowed unchecked.
git now resolves the URLs first (`submodule init` + `sync` write them to
the repo's config), and those resolved URLs, the ones git fetches, are
checked. A relative URL is never allowed by the check itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* .gitmodules is read NUL-delimited, and an unreadable record refuses
the submodules. A submodule name with a space (`[submodule "a b"]`)
made `--get-regexp`'s "key value" output misfile its URL as a path, so
the URL was never checked.
* Submodule URLs must have a strict shape every parser reads the same
way (no userinfo, backslash, query, fragment, or whitespace), so a URL
cannot be checked as one host and fetched from another.
* Each checked host is pinned (http.curloptResolve) to the addresses
that were checked, so DNS cannot change between the check and git's
fetch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A repo's .gitmodules is chosen by whoever controls the repo, so a
submodule URL could make WROLPi fetch https://localhost:8443/...,
https://10.0.0.1/..., etc. (SSRF), and show the response's error.
Every submodule URL is now checked before git fetches it: relative URLs,
the repo's own host, and hosts whose addresses are all public are
allowed; anything else defers the download with the reason. Nested
submodules are checked out one level at a time (instead of git's
--recursive) so each level is checked, at most 5 levels deep. Submodule
fetches do not follow HTTP redirects, so a checked host cannot redirect
git into the network.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* GET /api/repos/<id>/log: the checked out branch's commits, newest
first, a page at a time, with the total. The repo page has a
Files | History toggle (the tree is only listed for Files).
* GET /api/repos/<id>/archive.zip: `git archive HEAD` streamed to the
browser; the file name is reduced to safe characters. Everything is
read from the database before the response starts, because the
request's session closes once a streamed response begins.
* repository.submodules (default off): submodules are checked out with
the repo's mode and the same git restrictions, and removed when the
option is turned off. A submodule that cannot be downloaded (e.g. an
ssh URL) does not stop the repo from updating; the download is
retried with the submodule's error.
* The README speed test bound is loose enough for a busy machine.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A README comes from an untrusted remote.
* Escape the README headline: snippet() marks matches with control
characters, the text is HTML-escaped, then the markers become <b>.
Headlines are rendered as HTML (HeadlineText).
* Never follow a symlinked README (O_NOFOLLOW, regular files only).
* Convert markdown line by line with linear-time patterns. The old
multiline patterns backtracked polynomially; a README of blank lines
never finished, and runs of `<!--`, `![` or `<` were quadratic.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
repository_fts (FTS5, external content) indexes each Repo's search_name
(name and owner) and readme_text; a name match outranks a README
mention. The downloader stores the README as plain text (markdown
syntax removed so snippets read well), writing it only when it changed.
* POST /api/repos/search: by search and/or Tag, with README headlines.
* /api/search_other_estimates takes an optional search_str and returns
repo_count, so Repos count toward the global search's Other tab.
* /api/search_suggestions suggests Repos by name.
The repository FTS DDL is installed only once the repository table
exists: the baseline migration also runs install_raw_ddl, before that
table is created.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Branch names are passed to git as arguments and come from the API, the
config, a `/tree/<branch>` URL, or the remote itself (its default
branch). A name starting with `-` would be read as an option, so every
branch must pass git's ref-name rules, including in the downloader just
before use.
A repo URL with a `.` or `..` path segment (or a `..` name) could format
a directory outside its template (e.g. `repos/<tag>/..` is the whole
repos directory). Such URLs and names are rejected, and every repo
directory, including one from repos.yaml, must be strictly inside the
media directory with no `..`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Repo is a Collection (kind='repo') plus a `repository` row, kept up to
date by a recurring Download handled by the new `git` downloader.
* Full history by default (all branches and tags, no --prune; a force
push saves the old HEAD under refs/wrolpi/backup/); snapshot mode
(--depth 1) and a pinned branch are optional. `/tree/<branch>` URLs
select a branch.
* git runs HTTPS-only, never prompts, never runs hooks, and checks out
symlinks as plain files. Upstream disappearing defers the download
with git's error; the local copy is kept.
* Repos are cloned to `repos_destination` (repos/%(repo_tag)s/%(repo_name)s)
and the `repos` directory is always ignored, so repo files are never
indexed.
* repos.yaml is the source of truth (import, dump, backup restore).
* /api/repos: list, create, get, edit, delete (files kept unless
delete_files), update now, and a per-directory file tree. Tagging uses
the collections tag endpoint.
Also:
* run_command/process_runner can merge stderr into stdout (git writes
progress to stderr).
* move_collection deletes an old directory that only holds empty
directories (the move left an emptied .git behind).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>