diff --git a/docs/index.md b/docs/index.md index 5f23e2e..4ef5145 100644 --- a/docs/index.md +++ b/docs/index.md @@ -41,6 +41,7 @@ Information about the parts of the WROLPi interface. * [Docs](modules/docs/index.md) * [Map](modules/map/index.md) * [Zim](modules/zim/index.md) +* [Repos](modules/repos/index.md) * [Bookmarks](modules/bookmarks/index.md) * [Inventory](modules/inventory/index.md) * [Summary](modules/inventory/summary.md) diff --git a/docs/modules/repos/index.md b/docs/modules/repos/index.md new file mode 100644 index 0000000..4ac9cbe --- /dev/null +++ b/docs/modules/repos/index.md @@ -0,0 +1,104 @@ +# Repos + +The Repos module keeps a local copy of git repositories (source code, documentation, datasets) and keeps each copy up +to date while you have Internet access. You can browse a repo's files and read its README offline. + +![Repos Page](repos-page.png) + +## Adding a Repo + +> To add a repo, click **Repos** in the navigation bar, then click **New Repo**. + +![New Repo Form](new-repo-form.png) + +| Field | Purpose | +|--------------------|-----------------------------------------------------------------------------------------------| +| URL | The `https://` address of the repo, e.g. `https://github.com/kiwix/kiwix-tools`. | +| Name | Optional. Defaults to the last part of the URL (`kiwix-tools`). | +| Tag | Optional. A tagged repo is saved under its Tag in the Repos Directory. | +| Download Frequency | How often WROLPi checks for updates. Weekly by default. | +| Mode | **Full history** (default) or **Snapshot**. See [Modes](#modes). | +| Branch | Optional. Empty follows the repo's default branch (usually `main` or `master`). | + +WROLPi clones the repo in the background; its progress is on the Downloads page. Any git host that supports +`https://` works (GitHub, GitLab, Codeberg, Gitea, sourcehut, and others). SSH URLs, and repos that require a password, +are not supported. + +You can also paste the URL of a branch, e.g. `https://github.com/kiwix/kiwix-tools/tree/dev`, and WROLPi will follow +that branch. + +If two repos have the same name (e.g. two repos named `utils` from different owners), WROLPi asks you to choose another +name, such as `owner-utils`. + +## Modes + +| Mode | What is saved | Size | +|--------------|-------------------------------------------------------|-------------------------------------| +| Full history | Every branch, tag and commit. | Can be many times larger than the files. | +| Snapshot | Only the latest commit of one branch. | About the size of the files. | + +Full history is the default because storage is cheaper than a project you can no longer download. You can change the +mode of a repo at any time on its Edit page; switching from Snapshot to Full history downloads the rest of the history +on the next update. + +## Updates + +Each repo is checked for updates on its Download Frequency. Click **Update Now** on a repo's page to check right away. + +A repo is a **mirror** of its source: + +* Any change you make to a repo's files is discarded by the next update. +* Branches deleted by the source are kept. +* If the source rewrites its history (a "force push"), the commit WROLPi had is kept under + `refs/wrolpi/backup/` in the repo, so nothing is lost. + +If the source disappears (deleted, made private, or no Internet), **your copy is kept**. The repo's page shows +"The last update failed" with git's error, and the update is tried again later. + +**Warning!** Git LFS files are not downloaded; they appear as small text files. + +## Browsing a Repo + +> To view a repo, click its name in the Repos table. + +The repo's page shows its source, branch, size, and latest commit, then its files and its README. Click a directory to +open it, or a file to preview it. The README of each directory is shown below its files. + +![Repo Page](repo-page.png) + +## Search + +A repo's name, owner, and README are searchable. Matching repos appear under the **Other** tab of the search results, +with the part of the README that matched. Repos are also suggested by name as you type a search. + +The individual files in a repo are **not** searchable; see [Repos Directory](#repos-directory). + +## Tagging and Deleting + +> To tag a repo, click **Edit** on the repo's page, then click **Tag**. + +Tagging a repo can move it into the Tag's directory, like other [Collections](../../system/collections/index.md). + +> To delete a repo, click **Edit** on the repo's page, then click **Delete**. + +WROLPi stops updating the repo. Its files are kept, unless you check **Also delete its files**. + +## Repos Directory + +Repos are saved in the `repos` directory of your media directory, using the **Repos Directory** setting on the +Settings page. The default is `repos/%(repo_tag)s/%(repo_name)s`. + +| Variable | Value | +|-------------------|-------------------------------------------| +| `%(repo_name)s` | The repo's name. | +| `%(repo_tag)s` | The repo's Tag (empty if it has no Tag). | +| `%(repo_owner)s` | The owner from the URL (e.g. `kiwix`). | +| `%(repo_host)s` | The host from the URL (e.g. `github.com`). | + +The `repos` directory is always ignored when WROLPi refreshes your files, so the many files of a repo do not crowd your +file search. If you change the Repos Directory to somewhere outside `repos`, those repos' files will be indexed. + +## Config + +Your repos are saved in `repos.yaml` in the [config directory](../../system/configs.md). If your database is lost, your +repos (and their download schedules) are restored from this file. diff --git a/docs/modules/repos/new-repo-form.png b/docs/modules/repos/new-repo-form.png new file mode 100644 index 0000000..e362572 Binary files /dev/null and b/docs/modules/repos/new-repo-form.png differ diff --git a/docs/modules/repos/repo-page.png b/docs/modules/repos/repo-page.png new file mode 100644 index 0000000..c9936dd Binary files /dev/null and b/docs/modules/repos/repo-page.png differ diff --git a/docs/modules/repos/repos-page.png b/docs/modules/repos/repos-page.png new file mode 100644 index 0000000..cd2d689 Binary files /dev/null and b/docs/modules/repos/repos-page.png differ diff --git a/docs/system/configs.md b/docs/system/configs.md index e483cf6..01103af 100644 --- a/docs/system/configs.md +++ b/docs/system/configs.md @@ -19,6 +19,7 @@ All configs are stored in the `config` directory of your media directory: `/medi | `channels.yaml` | Your video Channels and their download schedules. | | `domains.yaml` | Your Archive Domains. | | `playlists.yaml` | Your Playlists, including the order of their items. | +| `repos.yaml` | Your [Repos](../modules/repos/index.md) and their download schedules. | | `download_manager.yaml` | All of your Downloads, and the skip list of deleted URLs. | | `videos_downloader.yaml` | Video downloader (yt-dlp) settings: resolutions, file name format, etc. | | `archives_downloader.yaml` | Archive downloader settings. | @@ -70,8 +71,9 @@ Channels at once). Keep in mind: Imports are designed to be safe, but you should know the rules: * A **missing or empty** config file never deletes anything from the database. -* Removing a Channel or Playlist from a non-empty `channels.yaml` / `playlists.yaml` **will delete it** (and a - Channel's videos) on the next import — this is how you can remove Collections by editing configs. +* Removing a Channel, Playlist or Repo from a non-empty `channels.yaml` / `playlists.yaml` / `repos.yaml` **will + delete it** (and a Channel's videos; a Repo's files are kept) on the next import — this is how you can remove + Collections by editing configs. * Removing a Tag from `tags.yaml` only deletes it if it is no longer used by anything. ## Backup and Restore diff --git a/docs/system/special-directories.md b/docs/system/special-directories.md index 8321e41..60a66c4 100644 --- a/docs/system/special-directories.md +++ b/docs/system/special-directories.md @@ -5,6 +5,7 @@ | `/media/wrolpi/` | The [Media Directory](#media-directory). | | `/media/wrolpi/config/` | The [configuration files](#wrolpi-config) of the WROLPi. | | `/media/wrolpi/tags/` | The [Tags Directory](#tags-directory) contains links to files that have been tagged. | +| `/media/wrolpi/repos/` | The [Repos Directory](#repos-directory) contains your git repos. | | `/opt/wrolpi-blobs/` | Contains files necessary to repair a WROLPi. | | `/opt/wrolpi-help/` | These help files. | | `/opt/wrolpi/` | The [source code](#wrolpi-source-directory) of WROLPi. | @@ -35,6 +36,16 @@ files. See [Tags](tags.md). **Warning!** Do not store files in this directory, they will be automatically deleted. +## Repos Directory + +`/media/wrolpi/repos` + +This directory contains the git repos of the [Repos](../modules/repos/index.md) module. Its files are never indexed, so +they do not appear in file search. + +**Warning!** WROLPi keeps each repo identical to its source; changes you make to a repo's files are discarded by the +next update. + ## WROLPi Blobs `/opt/wrolpi-blobs/`