Commit graph

2 commits

Author SHA1 Message Date
lrnselfreliance
cbf69e3013
Repo changes wait for a concurrent writer (#853)
Some checks failed
Backend Tests / api-tests (core) (push) Has been cancelled
Backend Tests / api-tests (files) (push) Has been cancelled
Backend Tests / api-tests (media) (push) Has been cancelled
Backend Tests / api-tests (rest) (push) Has been cancelled
Backend Tests / api-contract-tests (push) Has been cancelled
Backend Tests / api-migrations-and-start (push) Has been cancelled
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Cypress Tests / cypress-component-tests (push) Has been cancelled
Cypress Tests / cypress-e2e-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
Adding, importing, changing, deleting and updating a Repo read, then
wrote, in the request's deferred transaction, so they failed with a 500
("database is locked") when another connection wrote in between.  They
now write inside request.ctx.db.write().

Importing reads the clone with git and walks it for its size; those run
before the write transaction, so it is not held for them.  Everything
that can refuse the import still runs before the clone is changed.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 16:50:09 -06:00
lrnselfreliance
93d878952c
Add a Repos module that mirrors git repositories (#783)
Repos keeps local mirrors of git repositories, cloned over https and kept up to date on a schedule.

- A Repo is a Collection (kind='repo') plus a `repository` row.  The new `git` downloader clones it, then updates it
  (weekly by default, or Never: the clone is kept as it is and only updated on request).
- Repos are saved in `repos/<tag>/<name>` (the Repos Directory, ignored by default) or a directory the user chooses.
  An existing clone can be imported where it is.  repos.yaml is the source of truth.
- A GitHub-like page shows each repo's README, files, and history, with a ZIP download.  READMEs are searchable
  under the global search's Other tab.
- git is hardened against hostile repos and configs: https only, pinned addresses with no private addresses (for the
  repo and its submodules), sanitized configs, no hooks or links, and only clones WROLPi made or imported are
  updated.
- Tagging a Repo moves its clone as one directory.  Repo changes wait for repos.yaml to be imported at startup.
- The file mover groups unindexed files with no primary file as one FileGroup, and never replaces a file at the
  destination.  A failed tag move restores the Collection's tag.  A Collection's directory cannot leave the media
  directory.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 15:16:58 -06:00