Roland
7d6b7114a7
Adding TRACE level for logging beyond debug. Reworking async_client for tests, not all tests need the App to handle signals, so any test that needs signals can use the new await_switches fixture which primes the app.
2024-12-11 11:52:33 -07:00
Roland
5621d47309
Video downloads have many more options. Downloads are migrated.
...
Storing video settings in session storage to help user start downloads consistently.
Applying channel tag name to new channels when download videos or channels.
Initializing map tiles when importing and resetting map.
2024-11-28 09:48:57 -07:00
Roland
0784f740bb
Reworking Downloader.process_runner to use real stdout/stderr files to avoid issue with pipes being truncated.
2024-10-16 18:57:07 -06:00
Roland
77ad52473a
Fixing upsert_file usage. Adding log details for error.
2024-10-16 18:56:43 -06:00
Roland
d54735c4bb
Configs can be saved in background using switches.
2024-10-16 18:56:38 -06:00
Roland
317f3f3dfb
Top processes are displayed on Status page. Nice of renderd is lowered to avoid clobbering system at startup.
2024-10-16 18:56:27 -06:00
Roland
0f232fd533
Moving FileGroups instead of refreshing files after they have been moved.
2024-10-16 18:56:25 -06:00
Roland
45167e4f32
A Channel can be tagged with a single Tag. The Channel can be moved into a Tag Directory according to the config.
2024-10-16 18:56:19 -06:00
Roland
905042b600
Files which do not have a model can still be uploaded, they are indexed instead.
2024-10-16 18:56:15 -06:00
Roland
a98aebfdda
A video that is uploaded is attached to its Channel. Saving Channels config with both download url and frequency in case downloads config is corrupted.
2024-10-16 18:56:11 -06:00
Roland
0e99266379
A directory containing files that have been tagged is maintained automatically by WROLPi.
2024-10-16 18:56:05 -06:00
Roland
c839acb270
Converting Tag methods to use tag id or name.
2024-10-16 18:56:01 -06:00
Roland
2ad363281f
Delaying map startup because it prevents the API from starting.
2024-10-16 18:56:00 -06:00
Roland
af8bd7ba34
Cleaning up some extra data from video info json.
2024-10-16 18:55:59 -06:00
Roland
82af9a0d3c
Fixing bug where download.downloader would be replaced for all Channel Downloads.
2024-10-16 18:55:58 -06:00
Roland
7933d9625c
Downloads without a frequency cannot be a Channel Download.
2024-10-16 18:55:57 -06:00
Roland
63738bd70f
Removing ChannelDownload in favor of Download.channel_id.
2024-10-16 18:55:54 -06:00
Roland
287beabb9c
Creating missing ChannelDownload when importing channels config.
2024-10-16 18:55:49 -06:00
Roland
50d95dad19
Channels now have Channel.channel_downloads. A Channel can have many Downloads.
2024-10-16 18:55:46 -06:00
Roland
0fd545d2e8
Setting FileGroup.censored when refreshing channel files.
2024-06-17 14:48:41 -06:00
Roland
0bee458b79
Sorting keys when saving json file.
2024-06-13 14:08:07 -06:00
Roland
0fbbd36297
Creating perpetual funciton which checks if the internet is accessible. This is used to prevent downloads from starting when no internet exists.
2024-05-31 23:54:00 -06:00
Roland
58de8c7c5b
Upgrading Sanic and yt-dlp. Major changes occurred because Sanic changed how multiprocessing works.
...
Python 3.11 is now required, old versions were causing unpredictability in tests. (Sanic does not yet
support 3.12)
Sanic has been upgraded to 23.6.0, which is the latest version that avoids this bug:
https://github.com/sanic-org/sanic/issues/2921
New strategy for multiprocessing is to create all multiprocessing tools in one process, then fork
to other processes. The previous strategy was to declare multiprocessing tools at the top of every
file, or wherever they were needed at import/creation. Now all multiprocessing tools are attached to the
app.shared_ctx. This means `api_app` is imported in many, many places.
This forced a change in how the DownloadManager works. Previously, it would continually run
download workers which would pull downloads from a multiprocessing.Queue. Now, a single worker checks
for new downloads and sends a Sanic signal.
Flags have been reworked to use the `api_app`. I removed the `which` flag functionality because the
`which` are called at import and needed their own multiprocessing.Event.
2024-04-30 10:55:25 -06:00
Roland
a665677ecc
Adding more lower-quality video formats to download when no other is available. Getting video ffprobe json when downloading.
2024-03-16 19:58:58 -06:00
Roland
758ff8e810
Creating background process that fetches comments for Videos that do not yet have comments.
...
Displaying star for Uploader's comments, heart for favorited comments.
Adding wrolpi help files desktop shortcut to rpi image.
Using zim file names as FileGroup.title.
Removing accidental output commit.
2024-03-01 17:15:56 -07:00
Roland
df849ffb42
Ordering video/archive results when no published_datetime exists. Adding Video.location property which returns the App location of the video.
2024-02-26 13:40:39 -07:00
Roland
0d6edc05d0
Duplicate videos are deleted when a video is downloaded.
2024-02-21 12:42:30 -07:00
Roland
f865d9fb0f
Videos display comments. User can download comments using Refresh button.
2024-02-20 13:47:12 -07:00
Roland
1b92d1e580
Global search uses Semantic SearchResultsInput component, other components use SearchInput.
2023-11-21 13:01:52 -07:00
Roland
97b214d387
Adding Total Duration to EditChannel page. Handling when searching channel names, but channels have not been fetched yet.
2023-11-02 10:28:00 -06:00
Roland
440b770a11
Fixing Video.get_surrounding_videos after moving video upload date.
2023-10-29 18:44:50 -06:00
Roland
15687e534a
Moving data from models to FileGroup.
...
* Ebook.creator -> FileGroup.author
* Video.viewed -> FileGroup.viewed
* Video.duration -> FileGroup.length
* Video.url -> FileGroup.url
* Archive.url -> FileGroup.url
2023-10-29 17:31:54 -06:00
Roland
76212cf3de
Created script which searches for videos that have similar names, interacts with the user to choose which to delete.
2023-10-09 08:44:30 -06:00
Roland
6e0f9a999c
Combining install scripts into a new repair script. Removing create_db script in favor of repair script.
2023-10-03 12:15:29 -06:00
Roland
e7582944b9
Channels not in the channels config are deleted on import.
2023-09-12 20:24:02 -06:00
Roland
3e39102a55
ffprobe json may not be valid, the video should still be retrieved.
2023-09-07 13:01:35 -06:00
Roland
da6d6a8094
Allowing starting a download of a channel without frequency. Doing this creates a once-download.
2023-08-30 14:01:58 -06:00
Roland
6959c7a3b4
HTML and JSON files are reformatted when downloaded.
2023-08-26 21:49:24 -06:00
Roland
9dbd43cfdd
Videos near a video file without and upload date are returned as the prev/next videos.
2023-08-17 19:32:59 -06:00
Roland
14017d43bb
Creating Video/Archive get_by_id/find_by_id methods. find_by_id will raise an exception when not found, get_by_id will not.
2023-08-17 18:46:42 -06:00
Roland
2821d6d7b8
Removing unused column.
2023-08-15 14:14:37 -06:00
Roland
e8e5b5727f
Creating Video.validated which is used by find_corrupt_videos.py to search.
2023-08-08 13:42:55 -06:00
Roland
36083200e6
Creating Video.ffprobe_json column which stores ffprobe information about the video file. Creating find_corrupt_videos.py script which automatically re-downloads videos which do not have both video and audio streams.
2023-08-03 21:40:25 -06:00
Roland
fdaaa510c4
Splitting errors into their modules. Removing API_ERRORS dict in favor of storing error data on the class.
2023-07-17 14:21:27 -06:00
Roland
ce34721c4c
Videos are disowned when their Channel is deleted. A Channel directory cannot be in another Channel's directory.
2023-07-14 13:56:23 -06:00
Roland
03a2df217d
Zim/Kiwix (Wikipedia) feature
...
* A new zim module has been added, this module allows several things to interact with Zim files.
* Allow searching of Zim files using libzim.
* Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
* Creation of a new zim docker container which allows viewing of the Zim file contents.
* This commit also adds move/rename/mkdir features in the File Browser.
* File tags are preserved when moving/renaming.
2023-07-10 21:30:40 -06:00
Roland
8de805b6d1
Updating tests to match searching for Channel and Domain names for directories.
2023-05-03 12:59:47 -06:00
Roland
be3e8375a9
Creating FileGroup.merge to be used in FileGroup.from_paths.
2023-05-03 08:28:53 -06:00
Roland
83f9f6f085
Trusting Video.file_group.primary_path instead of checking mimetype each time.
2023-04-28 13:56:21 -06:00
Roland
6fe536558a
You cannot delete a tagged FileGroup.
2023-04-12 11:42:13 -06:00
