Roland
9ee73af61d
Converting APIError to use SanicException as base.
2024-05-01 18:04:25 -06:00
Roland
58de8c7c5b
Upgrading Sanic and yt-dlp. Major changes occurred because Sanic changed how multiprocessing works.
...
Python 3.11 is now required, old versions were causing unpredictability in tests. (Sanic does not yet
support 3.12)
Sanic has been upgraded to 23.6.0, which is the latest version that avoids this bug:
https://github.com/sanic-org/sanic/issues/2921
New strategy for multiprocessing is to create all multiprocessing tools in one process, then fork
to other processes. The previous strategy was to declare multiprocessing tools at the top of every
file, or wherever they were needed at import/creation. Now all multiprocessing tools are attached to the
app.shared_ctx. This means `api_app` is imported in many, many places.
This forced a change in how the DownloadManager works. Previously, it would continually run
download workers which would pull downloads from a multiprocessing.Queue. Now, a single worker checks
for new downloads and sends a Sanic signal.
Flags have been reworked to use the `api_app`. I removed the `which` flag functionality because the
`which` are called at import and needed their own multiprocessing.Event.
2024-04-30 10:55:25 -06:00
Roland
2c127a03f5
Testing channel suggestions being ordered by video count.
2024-04-16 15:48:09 -06:00
Roland
7669cfe361
Ordering suggested channel names by video count.
2024-04-16 10:04:06 -06:00
Roland
ba2dd6dd11
Many fixes split out from the feature/unified-query branch. Installing diagnostic tools in the built images. Displaying video comment statistics. Echoing error when install.sh fails.
2024-04-14 13:13:30 -06:00
Roland
a665677ecc
Adding more lower-quality video formats to download when no other is available. Getting video ffprobe json when downloading.
2024-03-16 19:58:58 -06:00
Roland
947135985d
Reformatting code.
2024-03-12 17:20:51 -06:00
Roland
4b8b897eb3
Unifying YoutubeDL.
2024-03-12 16:52:26 -06:00
Roland
ce0ae66a9e
Storing video download_datetime.
2024-03-12 11:06:30 -06:00
Roland
2ec267a65b
Automatically backing up config files each day in config/backup directory.
2024-03-11 17:47:35 -06:00
Roland
a486a39b73
Channels config no longer supports favorites.
2024-03-10 16:44:08 -06:00
Roland
b05b2e7a6d
Allowing maintainer to change special directories.
2024-03-09 16:51:26 -07:00
Roland
00fe799b8f
User can select from/to years when searching files. Indexing doc and docx files.
2024-03-07 12:12:48 -07:00
Roland
758ff8e810
Creating background process that fetches comments for Videos that do not yet have comments.
...
Displaying star for Uploader's comments, heart for favorited comments.
Adding wrolpi help files desktop shortcut to rpi image.
Using zim file names as FileGroup.title.
Removing accidental output commit.
2024-03-01 17:15:56 -07:00
Roland
df849ffb42
Ordering video/archive results when no published_datetime exists. Adding Video.location property which returns the App location of the video.
2024-02-26 13:40:39 -07:00
Roland
b705e08e48
Handling if multiple videos exist at start of video download.
2024-02-22 13:12:26 -07:00
Roland
2aa30e1674
Getting video location to report errors when getting video metadata again.
2024-02-22 08:47:35 -07:00
Roland
0d6edc05d0
Duplicate videos are deleted when a video is downloaded.
2024-02-21 12:42:30 -07:00
Roland
08fa3af453
Moving video about pane to it's own segment to avoid tabs going out of mobile viewing area.
2024-02-20 15:13:18 -07:00
Roland
5629adc84c
Limiting normal video comment download limit.
2024-02-20 14:06:13 -07:00
Roland
f865d9fb0f
Videos display comments. User can download comments using Refresh button.
2024-02-20 13:47:12 -07:00
Roland
32346dedc8
Fixing download url for Electronics and Unix zim files.
2024-02-19 16:00:51 -07:00
Roland
61a7fdab10
Creating HTMLIndexer which indexes the text of HTML files.
2024-02-19 15:48:17 -07:00
Roland
85a1360a25
Suggesting channels that names match, even without spaces.
2024-02-19 14:55:53 -07:00
Roland
4f4b2bdc72
Fixing Arduino download URL.
2024-02-13 14:22:25 -07:00
Roland
afecb62114
Adding medical wiki projects.
2024-02-13 14:16:16 -07:00
Roland
acaec9eacc
Testing special english stack overflow url.
2024-02-12 13:05:14 -07:00
Roland
80d9b74a03
Fixing english stackoverflow download url.
2024-02-12 12:51:12 -07:00
Roland
971eb4f99a
Adding many more Stack Exchange Zim subscriptions.
2024-02-12 12:40:56 -07:00
Roland
59b23b3134
Creating function which appends tag names sub-select.
2024-02-10 21:11:19 -07:00
Roland
8916600a14
Search suggestions handles any combination of filtering.
2024-02-10 19:42:29 -07:00
Roland
8b67dc87ff
Checking that tests are happening in tmp directory.
2024-02-05 14:52:38 -07:00
Roland
377c69e8a7
Adding search icon button to navbar.
2024-01-01 11:31:17 -07:00
Roland
d87bc7b314
Fetching search estimate with tags separate from suggestions.
2023-11-29 21:09:39 -07:00
Roland
760d949f5e
Adding debugging when finding channel.
2023-11-24 19:23:25 -07:00
Roland
cccdd8fe4e
Removing extra parameters from video urls.
2023-11-24 19:06:32 -07:00
Roland
bbfe76890b
Converting Search Suggetions to a Context. Search estimates is removed and suggestions is used instead.
2023-11-22 12:19:49 -07:00
Roland
1b92d1e580
Global search uses Semantic SearchResultsInput component, other components use SearchInput.
2023-11-21 13:01:52 -07:00
Roland
97b214d387
Adding Total Duration to EditChannel page. Handling when searching channel names, but channels have not been fetched yet.
2023-11-02 10:28:00 -06:00
Roland
73cb142ae1
Reverting to older singlefile container to keep archive screenshot ratio. Using FileGroup.url on ArchivePage.
2023-10-31 10:20:45 -06:00
Roland
9ea2dadde5
Raising specific error when datetime cannot be parsed.
2023-10-29 22:15:57 -06:00
Roland
f3faec9ce7
Removing unused video sort.
2023-10-29 21:17:21 -06:00
Roland
440b770a11
Fixing Video.get_surrounding_videos after moving video upload date.
2023-10-29 18:44:50 -06:00
Roland
15687e534a
Moving data from models to FileGroup.
...
* Ebook.creator -> FileGroup.author
* Video.viewed -> FileGroup.viewed
* Video.duration -> FileGroup.length
* Video.url -> FileGroup.url
* Archive.url -> FileGroup.url
2023-10-29 17:31:54 -06:00
Roland
ae9d188391
Adding FileGroup.published_date/author/etc. Creating script which converts a Singlefile to an Archive FileGroup files.
...
Parsing SingleFile datetime using dates.strpdate.
Adding timezone when no timezone info is available.
Installing python-dateutil to assist in handling diverse datetime strings.
Reworking refresh progress to use idempotency to display more accurate progress. Only logging that files have been invalidated if they used to be indexed.
Clearing archives to force re-index.
Ignoring 'Unknown' author from PDF metadata. Using file_title when pdf metadata title is empty.
2023-10-23 18:09:08 -06:00
Roland
b819b9c916
Adding Amateur Radio zim download.
2023-10-09 16:46:46 -06:00
Roland
76212cf3de
Created script which searches for videos that have similar names, interacts with the user to choose which to delete.
2023-10-09 08:44:30 -06:00
Roland
6e0f9a999c
Combining install scripts into a new repair script. Removing create_db script in favor of repair script.
2023-10-03 12:15:29 -06:00
Roland
b817534c37
Adding Download.is_new/etc. for code suggestions. Fixing async test warnings.
2023-09-20 12:58:35 -06:00
Roland
351351d90c
Raising error when Downloader.do_download is not programmed correctly. Fixing any tests that were not using do_download correctly.
2023-09-20 12:17:07 -06:00
