Commit graph

4 commits

Author SHA1 Message Date
Roland
a94261276c
repos: import an existing git clone as a Repo
POST /api/repos/import/inspect reports what importing a clone in the
media directory would do, without changing it: its origin (an SSH or
git:// origin is converted to https://), branch, latest commit, commits
not on the remote, whether it is shallow, and where it will be moved.
It refuses special directories, Collections' directories, `.git`
files/links, bare repos, and partial clones.  It runs no git command
which could run a program from the clone's config (the latest commit is
read with cat-file, not log).

POST /api/repos/import (confirmed, with a URL that is the clone's
origin) moves the clone into the Repos directory (the old location is
refreshed away), sanitizes its config, points origin at the URL, marks
it as WROLPi's, and creates the Repo and its Download.  The Repo shows
the clone's commit and README at once; its first update makes it a
mirror (local work is discarded, and the page warns of local commits).

The Repos page has an Import button; the Import page inspects the
chosen directory, then imports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 18:09:52 -06:00
Roland
03e3476631
repos: history, Download ZIP, and submodules
* GET /api/repos/<id>/log: the checked out branch's commits, newest
  first, a page at a time, with the total.  The repo page has a
  Files | History toggle (the tree is only listed for Files).
* GET /api/repos/<id>/archive.zip: `git archive HEAD` streamed to the
  browser; the file name is reduced to safe characters.  Everything is
  read from the database before the response starts, because the
  request's session closes once a streamed response begins.
* repository.submodules (default off): submodules are checked out with
  the repo's mode and the same git restrictions, and removed when the
  option is turned off.  A submodule that cannot be downloaded (e.g. an
  ssh URL) does not stop the repo from updating; the download is
  retried with the submodule's error.
* The README speed test bound is loose enough for a busy machine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00
Roland
5c80ad7ba4
repos: search Repos by name and README
repository_fts (FTS5, external content) indexes each Repo's search_name
(name and owner) and readme_text; a name match outranks a README
mention.  The downloader stores the README as plain text (markdown
syntax removed so snippets read well), writing it only when it changed.

* POST /api/repos/search: by search and/or Tag, with README headlines.
* /api/search_other_estimates takes an optional search_str and returns
  repo_count, so Repos count toward the global search's Other tab.
* /api/search_suggestions suggests Repos by name.

The repository FTS DDL is installed only once the repository table
exists: the baseline migration also runs install_raw_ddl, before that
table is created.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00
Roland
b3255e715c
Add a Repos module that mirrors git repositories (backend)
A Repo is a Collection (kind='repo') plus a `repository` row, kept up to
date by a recurring Download handled by the new `git` downloader.

* Full history by default (all branches and tags, no --prune; a force
  push saves the old HEAD under refs/wrolpi/backup/); snapshot mode
  (--depth 1) and a pinned branch are optional.  `/tree/<branch>` URLs
  select a branch.
* git runs HTTPS-only, never prompts, never runs hooks, and checks out
  symlinks as plain files.  Upstream disappearing defers the download
  with git's error; the local copy is kept.
* Repos are cloned to `repos_destination` (repos/%(repo_tag)s/%(repo_name)s)
  and the `repos` directory is always ignored, so repo files are never
  indexed.
* repos.yaml is the source of truth (import, dump, backup restore).
* /api/repos: list, create, get, edit, delete (files kept unless
  delete_files), update now, and a per-directory file tree.  Tagging uses
  the collections tag endpoint.

Also:
* run_command/process_runner can merge stderr into stdout (git writes
  progress to stderr).
* move_collection deletes an old directory that only holds empty
  directories (the move left an emptied .git behind).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:56:09 -06:00