Commit graph

28 commits

Author SHA1 Message Date
Roland
cdc73d9d16
Trimming video file name to avoid error when file name is too long. 2024-10-16 18:55:52 -06:00
Roland
50d95dad19
Channels now have Channel.channel_downloads. A Channel can have many Downloads. 2024-10-16 18:55:46 -06:00
Roland
9ee73af61d
Converting APIError to use SanicException as base. 2024-05-01 18:04:25 -06:00
Roland
58de8c7c5b
Upgrading Sanic and yt-dlp. Major changes occurred because Sanic changed how multiprocessing works.
Python 3.11 is now required, old versions were causing unpredictability in tests.  (Sanic does not yet
support 3.12)

Sanic has been upgraded to 23.6.0, which is the latest version that avoids this bug:
 https://github.com/sanic-org/sanic/issues/2921

New strategy for multiprocessing is to create all multiprocessing tools in one process, then fork
to other processes.  The previous strategy was to declare multiprocessing tools at the top of every
file, or wherever they were needed at import/creation.  Now all multiprocessing tools are attached to the
app.shared_ctx.  This means `api_app` is imported in many, many places.

This forced a change in how the DownloadManager works.  Previously, it would continually run
download workers which would pull downloads from a multiprocessing.Queue.  Now, a single worker checks
for new downloads and sends a Sanic signal.

Flags have been reworked to use the `api_app`.  I removed the `which` flag functionality because the
`which` are called at import and needed their own multiprocessing.Event.
2024-04-30 10:55:25 -06:00
Roland
b05b2e7a6d
Allowing maintainer to change special directories. 2024-03-09 16:51:26 -07:00
Roland
758ff8e810
Creating background process that fetches comments for Videos that do not yet have comments.
Displaying star for Uploader's comments, heart for favorited comments.

Adding wrolpi help files desktop shortcut to rpi image.

Using zim file names as FileGroup.title.

Removing accidental output commit.
2024-03-01 17:15:56 -07:00
Roland
32346dedc8
Fixing download url for Electronics and Unix zim files. 2024-02-19 16:00:51 -07:00
Roland
4f4b2bdc72
Fixing Arduino download URL. 2024-02-13 14:22:25 -07:00
Roland
afecb62114
Adding medical wiki projects. 2024-02-13 14:16:16 -07:00
Roland
acaec9eacc
Testing special english stack overflow url. 2024-02-12 13:05:14 -07:00
Roland
80d9b74a03
Fixing english stackoverflow download url. 2024-02-12 12:51:12 -07:00
Roland
971eb4f99a
Adding many more Stack Exchange Zim subscriptions. 2024-02-12 12:40:56 -07:00
Roland
8b67dc87ff
Checking that tests are happening in tmp directory. 2024-02-05 14:52:38 -07:00
Roland
377c69e8a7
Adding search icon button to navbar. 2024-01-01 11:31:17 -07:00
Roland
d87bc7b314
Fetching search estimate with tags separate from suggestions. 2023-11-29 21:09:39 -07:00
Roland
bbfe76890b
Converting Search Suggetions to a Context. Search estimates is removed and suggestions is used instead. 2023-11-22 12:19:49 -07:00
Roland
73cb142ae1
Reverting to older singlefile container to keep archive screenshot ratio. Using FileGroup.url on ArchivePage. 2023-10-31 10:20:45 -06:00
Roland
ae9d188391
Adding FileGroup.published_date/author/etc. Creating script which converts a Singlefile to an Archive FileGroup files.
Parsing SingleFile datetime using dates.strpdate.

Adding timezone when no timezone info is available.

Installing python-dateutil to assist in handling diverse datetime strings.

Reworking refresh progress to use idempotency to display more accurate progress.  Only logging that files have been invalidated if they used to be indexed.

Clearing archives to force re-index.

Ignoring 'Unknown' author from PDF metadata. Using file_title when pdf metadata title is empty.
2023-10-23 18:09:08 -06:00
Roland
b819b9c916
Adding Amateur Radio zim download. 2023-10-09 16:46:46 -06:00
Roland
f14df90174
Removing unused imports. 2023-09-12 17:49:26 -06:00
Roland
a0e5c5d550
Wikipedia mini increased in size. 2023-08-24 07:48:35 -06:00
Roland
6458316d1a
Removing unused /videos/tag. Sending events when Channels are created or deleted. 2023-07-17 17:32:29 -06:00
Roland
fdaaa510c4
Splitting errors into their modules. Removing API_ERRORS dict in favor of storing error data on the class. 2023-07-17 14:21:27 -06:00
Roland
551aa50bce
Skipping kiwix restart during testing before flagging. 2023-07-11 17:15:44 -06:00
Roland
6f94dac2e8
Setting Kiwix restart flag when service can't be restarted. 2023-07-11 17:10:43 -06:00
Roland
1fff928d4e
Installing Kiwix packages and service in Raspberry Pi and Debian installer scripts. 2023-07-11 16:52:18 -06:00
Roland
0eb321de9f
Removing accidental commit. 2023-07-10 21:30:42 -06:00
Roland
03a2df217d
Zim/Kiwix (Wikipedia) feature
* A new zim module has been added, this module allows several things to interact with Zim files.
  * Allow searching of Zim files using libzim.
  * Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
  * Creation of a new zim docker container which allows viewing of the Zim file contents.

* This commit also adds move/rename/mkdir features in the File Browser.
  * File tags are preserved when moving/renaming.
2023-07-10 21:30:40 -06:00