diff --git a/docs/modules/files/files-page.png b/docs/modules/files/files-page.png index 494f90c..b756b60 100644 Binary files a/docs/modules/files/files-page.png and b/docs/modules/files/files-page.png differ diff --git a/docs/modules/files/index.md b/docs/modules/files/index.md index e758aa1..89ccf73 100644 --- a/docs/modules/files/index.md +++ b/docs/modules/files/index.md @@ -11,8 +11,8 @@ that have recently been created, and are not yet in the database. ## Refreshing -WROLPi can search any file in your media directory, this can only be accomplished after a full files refresh has been -completed. +WROLPi can search any file in your media directory, this can only be accomplished after a full refresh of your files has +been completed. > To refresh all your files, click the **Files** link in the top navigation bar, then click **Refresh** found at the > bottom of the files table. @@ -38,6 +38,23 @@ The tool buttons are as displayed: 5. **New Directory** button * Allows you to create a new directory in the directory selected (or, the top directory if no directory is selected.) +6. **Ignore** button + * Allows you to skip any directories (and their files) from indexing and searching. (Useful if you have a directory + with many thousands of small files, such as a git repo) + +## Tags + +WROLPi creates and manages a `tags` directory, this directory contains links to files you have tagged. This is the +[secondary](../../system/primary-secondary-tertiary.md) method to access your tags. **You should never manually add +files to this directory, they will be deleted.** + +If a file is tagged with multiple tags, the file link will be placed in a directory containing the names of the tags, +separated by commas (alphabetically). For example, if you tagged a video at +`videos/WROLPi/WROLPi_20230909_0xfMLNVFq2Y_WROLPi v0.11 beta demo.mp4` with the tags "WROL" and "Computers", +then a link to the video will be created in +`tags/Computers, WROL/WROLPi_20230909_0xfMLNVFq2Y_WROLPi v0.11 beta demo.mp4` + +All files in the **FileGroup** will be linked in the same directory. ## File Preview diff --git a/docs/modules/videos/channel-download-modal.png b/docs/modules/videos/channel-download-modal.png new file mode 100644 index 0000000..3217d17 Binary files /dev/null and b/docs/modules/videos/channel-download-modal.png differ diff --git a/docs/modules/videos/channel-edit-page.png b/docs/modules/videos/channel-edit-page.png index 64283b7..ca1bb17 100644 Binary files a/docs/modules/videos/channel-edit-page.png and b/docs/modules/videos/channel-edit-page.png differ diff --git a/docs/modules/videos/channel-new-page.png b/docs/modules/videos/channel-new-page.png index d1bb0d4..c49c353 100644 Binary files a/docs/modules/videos/channel-new-page.png and b/docs/modules/videos/channel-new-page.png differ diff --git a/docs/modules/videos/channel-tag-modal.png b/docs/modules/videos/channel-tag-modal.png new file mode 100644 index 0000000..2dc3809 Binary files /dev/null and b/docs/modules/videos/channel-tag-modal.png differ diff --git a/docs/modules/videos/index.md b/docs/modules/videos/index.md index c41e3c4..96991cb 100644 --- a/docs/modules/videos/index.md +++ b/docs/modules/videos/index.md @@ -13,19 +13,46 @@ download. A single video file is also supported. ## Video Format As of the writing of this article, WROLPi favors MP4 files due to their small size and universal support. -WROLPi will index many other video formats, but it will always download an MP4 (this may be changed in the -future). +WROLPi will index many other video formats. -## Downloading +## Downloading Videos -WROLPi will download a video (and related files) to that video's channel's directory. If you downloaded a +WROLPi will download a video (and related files) to that video's channel's directory. For example, if you downloaded a video from the WROLPi channel, then your video will be stored in `videos/WROLPi` by default. -It is possible to override the default download directory by opening the Advanced accordion on the Videos -download modal: - ![Videos Download Modal](video-download-modal.png) +### Download Options + +* **Videos Tags** + * All selected Tags will be applied to every video downloaded. +* **Destination** + * Override the default download directory by choosing a directory using the **Destination** input. +* **Video Resolutions** + * Download each video in the first resolution matched. If you select "1080p, 720p" and a video has both resolutions + available, it will use + 1080p, if no 1080p is available, then 720p will be downloaded, if neither resolutions is available an error will + be reported. +* **Video Format** + * The file format the video will be encapsulated in. + * `mp4` is the most universally compatible (but proprietary). + * `mkv` is an open-standard media format. +* **Channel Tag** + * The selected Tag will be applied to any new Channel created in the process of downloading these videos. The + Channel will be placed in the respective tag directory in the `videos` directory. + +## Download Channels/Playlists + +WROLPi can download all videos from a channel, or playlist. This is done using the `Download > Channel/Playlist` +download button. + +![img.png](channel-download-modal.png) + +### Download Options + +* **Videos Tags** + * All selected Tags will be applied to every video downloaded. + ## Channel A Channel contains all videos downloaded from the origin channel (the third-party site from which the video @@ -38,27 +65,64 @@ that you have not downloaded. ### Creating a Channel There are two methods to create a channel: the simplest is by downloading a video, which will automatically create the -channel for you. Alternatively, you can manually create a channel by navigating to the New Channel Page +channel for you. + +Alternatively, you can manually create a channel by navigating to the New Channel Page (`/videos/channel/new`). Access this page by clicking **Videos**, selecting the **Channels** tab, and then clicking **New Channel**. ![New Channel page](channel-new-page.png) -Enter the new channel's name and its directory. All other fields are optional (be sure to enable -`Create this directory, if it doesn't exist` if necessary). - -* Note: When entering a Channel's URL, you may need to append `/videos` to ensure the videos are downloaded to the - correct location. +Enter the new channel's name and its directory. ### Editing a Channel To edit a Channel, navigate to the Channels (`/videos/channels`) page and click the 'Edit' button on the right. You will be directed to the following page: -![Channel Edit Page](channel-edit-page.png) +![img.png](channel-edit-page.png) -On this page, you can modify the download frequency, the channel name, or the URL. Currently, it is not possible to -change the channel's directory. +On this page, you can modify the channel name, or the directory. + +**Warning:** If you change the Channel's directory, all files in the directory (video or not) will be moved to the new +directory you specified. If the automatic move fails somehow, you will need to manually move the files to the correct +directory, then refresh your files. You will probably need to manually re-apply any tags to your videos again. If you +are unsure what your previous tags were, open the `tags.yaml` config of the previous day (`/media/wrolpi/tags/backup`) +to see what tags you had applied. + +#### Details + +Under the **Details** dropdown, you will see the Channel's URL and RSS URL (if available). + +**Tip:** You can use the RSS URL to download new videos from a Channel quickly after they are released. + +#### Channel Tag + +You can also tag a Channel here. Only one Tag can be applied to a Channel. After clicking the **Tag** button, the +following modal will suggest moving your Channel (and it's files) under the respective tag directory in the `videos` +directory. You can uncheck "Move to directory" if you do not want to use the suggested directory. + +![img.png](channel-tag-modal.png) + +#### Buttons + +**Delete** + +* Used to delete the Channel. The files in the Channel's directory will not be deleted. + +**Refresh** + +* Refresh the files in the Channel's directory. Useful if you manually alter files in the directory. + +**Tag** + +* Add or replace the Tag of the Channel. + +#### Downloads + +You can add or update Downloads for this Channel. + +#### Statistics At the bottom of this page you will see the statistics of the channel's video files. This is useful if you want to be sure that all videos were found during a refresh. @@ -79,7 +143,7 @@ working. VLC also displays captions when the caption file is named like the vide ### Raspberry Pi -VLC can be launched by navigating: `Applicaions Menu > Sound & Video > VLC Media Player` +VLC can be launched by navigating: `Applications Menu > Sound & Video > VLC Media Player` ![wrolpi-launch-vlc.png](wrolpi-launch-vlc.png) diff --git a/docs/modules/videos/video-download-modal.png b/docs/modules/videos/video-download-modal.png index f8f2f2b..60f1cd8 100644 Binary files a/docs/modules/videos/video-download-modal.png and b/docs/modules/videos/video-download-modal.png differ diff --git a/docs/system/backup.md b/docs/system/backup.md index 0651041..d5eb0a8 100644 --- a/docs/system/backup.md +++ b/docs/system/backup.md @@ -33,6 +33,7 @@ echo >> ${logfile} echo start $(date +"%Y-%m-%d %H:%M:%S") >> ${logfile} /usr/bin/rsync \ --exclude="lost+found" \ + --exclude="tags" \ -avue ssh \ wrolpi@PRIMARY_IP_ADDRESS:/media/wrolpi/* \ /media/wrolpi/ \ diff --git a/docs/system/databases.md b/docs/system/databases.md new file mode 100644 index 0000000..0b66aaa --- /dev/null +++ b/docs/system/databases.md @@ -0,0 +1,42 @@ +# Databases + +WROLPi has two major Postgresql databases, the main (API) database, and the map database. + +## Main Database + +This contains the tables for the API. + +### Service + +`postgresql@15-main.service` + +### Tables + +| Name | Purpose | Relations | `/opt/wrolpi` File | +|------------------|---------------------------------------------------------|---------------------------|-------------------------------| +| alembic_version | Tracks Alembic migration. | | `alembic` | +| archive | Data about a FileGroup Archive. | file_group_id, domain_id | `modules/archive/models.py` | +| channel | Data about a video Channel. | | `modules/videos/models.py` | +| directory | Directories in the media directory. | | `wrolpi/files/models.py` | +| domains | Groups Archives by shared URL domain. | | `modules/archive/models.py` | +| download | Data about what should be downloaded and when. | | `wrolpi/downloader.py` | +| ebook | Data about FileGroups that are books. | file_group_id | `wrolpi/files/ebooks.py` | +| file_group | Each record is a group of files that share a stem. | | `wrolpi/files/models.py` | +| inventory | A collection of items. | | `modules/inventory/models.py` | +| item | An item in an inventory. | inventory_id | `modules/inventory/models.py` | +| map_file | Data about PBF files (if they have been imported, etc.) | | `modules/map/models.py` | +| tag | Data about a Tag (name, color, etc.) | | `wrolpi/tags.py` | +| tag_file | A FileGroup that has been tagged. | tag_id, file_group_id | `wrolpi/tags.py` | +| tag_zim | An entry (article) that was tagged in a Zim file. | tag_id, zim_id, zim_entry | `modules/zim/models.py` | +| video | Data about a FileGroup Video | file_group_id, channel_id | `modules/videos/models.py` | +| wrolpi_flag | Long-term flags that have occurred in the API. | | `wrolpi/flags.py` | +| zim | A Zim file | | `modules/zim/models.py` | +| zim_subscription | A recurring download of a Zim file. | | `modules/zim/models.py` | + +## Map Database + +This contains tables for the OpenStreetMap. + +### Service + +`postgresql@15-map.service` \ No newline at end of file diff --git a/docs/system/getting-started.md b/docs/system/getting-started.md index c805a55..b77859e 100644 --- a/docs/system/getting-started.md +++ b/docs/system/getting-started.md @@ -33,15 +33,23 @@ on a new Raspberry Pi. 1. Unplug your Raspberry Pi 2. Insert the micro SD card from step 2 into your Raspberry Pi -3. Connect any peripherals to your Raspberry Pi, then turn it on. -4. Login as the **pi** user with the password **wrolpi** -5. Launch the Terminal application - 1. Change your password with: `passwd` - 2. Modify fstab to mount your external drive to **/media/wrolpi** +3. Connect any peripherals to your Raspberry Pi. +4. Boot your Raspberry Pi. +5. Choose your Country/Language/Timezone. +6. Choose your own username and password (do not use the wrolpi user). +7. Skip "Wi-Fi Selection", it is not required. +8. Choose your favorite browser. +9. Skip "Raspberry Pi Connect". +10. Skip "Update Software". +11. Restart. +12. Login using the user you created above. +13. Launch the Terminal application + 1. Modify fstab to mount your external drive to **/media/wrolpi** * `echo '/dev/sda1 /media/wrolpi auto defaults,nofail 0 0' | sudo tee -a /etc/fstab` - 3. Finish the installation with the repair script: - * `sudo bash /opt/wrolpi/repair.sh` - 4. Reboot: `sudo reboot` + 2. Finish the installation with the repair script: + * `/opt/wrolpi/repair.sh` + 3. Reboot: `sudo reboot` +14. Join the Hotspot, browse to https://wrolpi.local or, the IP address of your WROLPi! ## Debian diff --git a/docs/system/primary-secondary-tertiary.md b/docs/system/primary-secondary-tertiary.md index 35a9df5..8e7e2b6 100644 --- a/docs/system/primary-secondary-tertiary.md +++ b/docs/system/primary-secondary-tertiary.md @@ -7,7 +7,8 @@ and tertiary options to use your WROLPi may be available. |----------|-------------------------------------------|--------------------------------------------------------------------------------|-------------------------------------------------------------------| | Videos | [React App](../modules/videos/index.md) | [Open video files with VLC.](../modules/videos/index.md#videos-without-wrolpi) | Open subtitle files with a text editor. | | Archives | [React App](../modules/archives/index.md) | Open Archive files directly with Chromium | Readability files can be opened with a text editor. | -| Map | [React App](../modules/map/index.md) | Map is directly accessible on port `:8084` | None | -| Zim | [React App](../modules/zim/index.md) | Kiwix server is directly accessible on port `:8085` | [Kiwix application](../modules/zim/index.md#kiwix-without-wrolpi) | -| Help | React App | Help server is directly accessible on port `:8086` | Markdown files are available in `/opt/wrolpi-help/docs` | -| Tags | React App | Browse the Tags Directory: `/media/wrolpi/tags` | Read the config: `/media/wrolpi/config/tags.yaml` | \ No newline at end of file +| Map | [React App](../modules/map/index.md) | Map is accessible on port `:8084` | None | +| Zim | [React App](../modules/zim/index.md) | Kiwix server is accessible on port `:8085` | [Kiwix application](../modules/zim/index.md#kiwix-without-wrolpi) | +| Help | React App | Help server is accessible on port `:8086` | Markdown files are available in `/opt/wrolpi-help/docs` | +| Tags | React App | Browse the Tags Directory: `/media/wrolpi/tags` | Read the config: `/media/wrolpi/config/tags.yaml` | +| Files | [React App](../modules/files/index.md) | Browse the files on port `:8443` path `/media` (https://localhost:8443/media) | Browse the files using the desktop file browser | diff --git a/docs/system/special-directories.md b/docs/system/special-directories.md index ff1a2f7..d62cbe7 100644 --- a/docs/system/special-directories.md +++ b/docs/system/special-directories.md @@ -38,7 +38,7 @@ files. `/opt/wrolpi-blobs/` -This directory contains the files necessary to repair a WROLPi. For example, the `gis-map.dump.gz` file contains a +This directory contains the files necessary to repair a WROLPi. For example, the `map-db-gis.dump` file contains a Postgresql dump of the map database. **It is best to leave this directory as-is.** ## WROLPi Help diff --git a/docs/system/systemd.md b/docs/system/systemd.md index 7cb1b9c..d3db06a 100644 --- a/docs/system/systemd.md +++ b/docs/system/systemd.md @@ -4,9 +4,11 @@ WROLPi services are controlled by `systemd`. You can view and control the WROLPi services using these commands: -| Service | Status | Logs | Restart | Stop | Start | -|-----------|---------------------------------|------------------------------|----------------------------------|-------------------------------|--------------------------------| -| API | `systemctl status wrolpi-api` | `journalctl -u wrolpi-api` | `systemctl restart wrolpi-api` | `systemctl stop wrolpi-api` | `systemctl start wrolpi-api` | -| React App | `systemctl status wrolpi-app` | `journalctl -u wrolpi-app` | `systemctl restart wrolpi-app` | `systemctl stop wrolpi-app` | `systemctl start wrolpi-app` | -| Map | `systemctl status renderd` | `journalctl -u renderd` | `systemctl restart renderd` | `systemctl stop renderd` | `systemctl start renderd` | -| Zim | `systemctl status wrolpi-kiwix` | `journalctl -u wrolpi-kiwix` | `systemctl restart wrolpi-kiwix` | `systemctl stop wrolpi-kiwix` | `systemctl start wrolpi-kiwix` | +| Service | Status | Logs | Restart | Stop | Start | +|-----------|-----------------------------------------------|------------------------------------------------|------------------------------------------------|---------------------------------------------|----------------------------------------------| +| API | `systemctl status wrolpi-api` | `journalctl -u wrolpi-api` | `systemctl restart wrolpi-api` | `systemctl stop wrolpi-api` | `systemctl start wrolpi-api` | +| React App | `systemctl status wrolpi-app` | `journalctl -u wrolpi-app` | `systemctl restart wrolpi-app` | `systemctl stop wrolpi-app` | `systemctl start wrolpi-app` | +| Map | `systemctl status renderd` | `journalctl -u renderd` | `systemctl restart renderd` | `systemctl stop renderd` | `systemctl start renderd` | +| Zim | `systemctl status wrolpi-kiwix` | `journalctl -u wrolpi-kiwix` | `systemctl restart wrolpi-kiwix` | `systemctl stop wrolpi-kiwix` | `systemctl start wrolpi-kiwix` | +| API DB | `systemctl status postgresql@15-main.service` | `journalctl status postgresql@15-main.service` | `systemctl restart postgresql@15-main.service` | `systemctl stop postgresql@15-main.service` | `systemctl start postgresql@15-main.service` | +| Map DB | `systemctl status postgresql@15-map.service` | `journalctl status postgresql@15-map.service` | `systemctl restart postgresql@15-map.service` | `systemctl stop postgresql@15-map.service` | `systemctl start postgresql@15-map.service` | diff --git a/requirements.txt b/requirements.txt index 30cf187..6b64d7a 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,2 +1 @@ -# Install newer version of mkdocs to avoid navbar bug. -git+https://github.com/mkdocs/mkdocs.git@12d4345a419e06a7edeb2effec7c98c36c907029#egg=mkdocs \ No newline at end of file +mkdocs==1.6.1 \ No newline at end of file