Commit graph

71 commits

Author SHA1 Message Date
Roland
d5ca3e7f02
Including Country/Region codes in caption file names. 2024-10-16 18:56:31 -06:00
Roland
7f8b294199
Dashboard needs bytes per second, not total. 2024-10-16 18:56:28 -06:00
Roland
317f3f3dfb
Top processes are displayed on Status page. Nice of renderd is lowered to avoid clobbering system at startup. 2024-10-16 18:56:27 -06:00
Roland
0f232fd533
Moving FileGroups instead of refreshing files after they have been moved. 2024-10-16 18:56:25 -06:00
Roland
35ec6d44fe
Creating setting for changing navbar color. 2024-10-16 18:56:04 -06:00
Roland
63738bd70f
Removing ChannelDownload in favor of Download.channel_id. 2024-10-16 18:55:54 -06:00
Roland
287beabb9c
Creating missing ChannelDownload when importing channels config. 2024-10-16 18:55:49 -06:00
Roland
7a784d7c45
Excluded URLs was not displayed correctly. 2024-10-16 18:55:48 -06:00
Roland
50d95dad19
Channels now have Channel.channel_downloads. A Channel can have many Downloads. 2024-10-16 18:55:46 -06:00
Roland
279fe2af3e
Adding Developer status accordions. 2024-05-31 23:52:55 -06:00
Roland
63381c1d77
Created download retry button. 2024-05-08 17:05:12 -06:00
Roland
58de8c7c5b
Upgrading Sanic and yt-dlp. Major changes occurred because Sanic changed how multiprocessing works.
Python 3.11 is now required, old versions were causing unpredictability in tests.  (Sanic does not yet
support 3.12)

Sanic has been upgraded to 23.6.0, which is the latest version that avoids this bug:
 https://github.com/sanic-org/sanic/issues/2921

New strategy for multiprocessing is to create all multiprocessing tools in one process, then fork
to other processes.  The previous strategy was to declare multiprocessing tools at the top of every
file, or wherever they were needed at import/creation.  Now all multiprocessing tools are attached to the
app.shared_ctx.  This means `api_app` is imported in many, many places.

This forced a change in how the DownloadManager works.  Previously, it would continually run
download workers which would pull downloads from a multiprocessing.Queue.  Now, a single worker checks
for new downloads and sends a Sanic signal.

Flags have been reworked to use the `api_app`.  I removed the `which` flag functionality because the
`which` are called at import and needed their own multiprocessing.Event.
2024-04-30 10:55:25 -06:00
Roland
ed1ed94beb
Replacing download count span with semanticui Label. 2024-04-16 10:14:47 -06:00
Roland
947135985d
Reformatting code. 2024-03-12 17:20:51 -06:00
Roland
b05b2e7a6d
Allowing maintainer to change special directories. 2024-03-09 16:51:26 -07:00
Roland
00ab6401a0
Adding minimum width to ellipsis text for mobile screens. Adding button to clear active tags filter. 2023-11-04 15:57:57 -06:00
Roland
97b214d387
Adding Total Duration to EditChannel page. Handling when searching channel names, but channels have not been fetched yet. 2023-11-02 10:28:00 -06:00
Roland
badfe691de
Using css ellipsis for download rows. 2023-11-02 10:07:49 -06:00
Roland
6e0f9a999c
Combining install scripts into a new repair script. Removing create_db script in favor of repair script. 2023-10-03 12:15:29 -06:00
Roland
95e870d659
Adding range input in settings so the user can change the API log level. 2023-09-08 17:10:15 -06:00
Roland
b92a0d5e41
Rewording retry button to restart. 2023-09-07 21:47:43 -06:00
Roland
9a3796bcd0
Creating download retry button for recurring downloads. 2023-09-07 21:38:19 -06:00
Roland
619a88c429
Reworking default/empty/error states for all pages. Null is pending, undefined is error. 2023-09-07 20:13:36 -06:00
Roland
ee95204d3b
Moving Status segments around so instantaneous statuses are next to eachother. Slower, disk usage is by itself. 2023-09-07 18:53:29 -06:00
Roland
b765f2e440
Keeping the old disk maximum read/write rate so the progress bars mean something. 2023-09-07 13:02:38 -06:00
Roland
6f43f1daa0
Creating RecursiveHTMLDownloader which recursively searches for files in the provided URLs. 2023-08-30 20:19:32 -06:00
Roland
402882ce2e
Restoring singlefile button on Archive page. 2023-08-28 09:47:51 -06:00
Roland
7c238f5dfa
Only one argument is passed. 2023-08-23 20:09:03 -06:00
Roland
b3cec16459
Displaying memory usage progress bar on status page. 2023-08-23 19:08:13 -06:00
Roland
cc84f20072
Moving restart/shutdown buttons back to Settings page. 2023-08-19 20:38:34 -06:00
Roland
98dc34df5e
Moving Restart/Shutdown buttons onto Status page. 2023-08-18 17:47:38 -06:00
Roland
4cc00a9cb1
Delaying reboot/shutdown with asyncio sleep to give API time to shut down. 2023-08-18 11:55:14 -06:00
Roland
2537017039
Creating shutdown and restart buttons. 2023-08-17 20:07:02 -06:00
Roland
027d305abb
Moving download clearing bottons to bottom of table to match style. 2023-08-17 17:52:40 -06:00
Roland
6791e88baf
Stopping/starting renderd during map import. Moving tile cache clearing to import script. 2023-08-17 11:49:19 -06:00
Roland
1e4644e15c
Creating CPU Temperature Icon which displays only when temperature is too high. 2023-08-13 10:54:43 -06:00
Roland
e8e5b5727f
Creating Video.validated which is used by find_corrupt_videos.py to search. 2023-08-08 13:42:55 -06:00
Roland
59c4ab2502
Replacing icon property when icon has been requested.
Creating <APIButton/> which handles success/failure/pending during API calls.
2023-07-21 16:52:15 -06:00
Roland
03a2df217d
Zim/Kiwix (Wikipedia) feature
* A new zim module has been added, this module allows several things to interact with Zim files.
  * Allow searching of Zim files using libzim.
  * Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
  * Creation of a new zim docker container which allows viewing of the Zim file contents.

* This commit also adds move/rename/mkdir features in the File Browser.
  * File tags are preserved when moving/renaming.
2023-07-10 21:30:40 -06:00
Roland
387282b445
Adding my public gpg key.
Moving upload destination input out of drop Segment.

Disabling uploads when downloads are disabled.

Inverting labels for checkboxes in inverted modal.

Deleting file_groups which no longer contain the exclusive primary path.

Adding debug logging for all API errors.

Checking that upload output is not a directory.

Expanding nginx request size.

Adding docs to upload endpoint.

Using dot-circle icon for more image mimetypes.

Using Form.Field to make upload destination required.

Adding border to inverted Segments.

Many files can be uploaded in chunks.

Files can be uploaded, progress is displayed.

Showing white border around share QR code.

Displaying WROLPi colors for download icons.

Creating button to clear DirectorySearch.

Excluding channel/domain directories so other directories can be returned.

Ordering search results.

Ignoring when no channel is found only if we have a destination.

Limiting directory search results.

Displaying categories when searching destination directories.

Handling when live_chat.json file exists.

Allowing refresh of directory directly.

Logging which download was successful.

Removing timeout.

Creating index for directory names.

Sleeping to catch cancel.

Arbitrary files can be downloaded using the new FileDownloader.
2023-04-28 13:56:32 -06:00
Roland
ec8a095699
Using SButton for close buttons to avoid themeing. 2023-04-09 17:17:50 -06:00
Roland
4d5eba59f7
Changing primary button colors to violet. 2023-04-05 10:18:52 -06:00
Roland
c24353edf2
Showing 'now' when next_download time has been passed. 2023-04-02 12:19:15 -06:00
Roland
0689c03866
Displaying downloads link when user submits download. 2023-04-01 16:09:05 -06:00
Roland
55cf76a884
Requiring downloader name to be specified. 2023-04-01 15:34:46 -06:00
Roland
5615ce542d
Making example label a tag.
Tags can be created and deleted in UI.

Adding tag_file using id to avoid session error.

Using UTC in test.

Matching tag_names style in test.

Tag icon is displayed on file rows.

Archives, Videos and Files can be filtered by their tags.

Tags can be added and removed from UI.

Tags are displayed in VideoPlayer.

The tags of a file are returned in it's JSON.

Video favorites are converted to Favorite tag on channel config import.

Updating tests to match frontend usage.

PDF creator and modification time are displayed.

Reworking file search to use Video.__json__ for complex video relations.

Archives can be viewed.

Removing ebook.title.

Reworking alembic migration to be an upgrade rather than a replace.

Skipping test on circleci.

Clearing out video caption index only if extract_subtitles if off.

Channel tests pass.

Some video tests pass.

Missing videos can be found.  A video can be downloaded.

Orphaned files can be found.

Videos can be tagged.

Videos can be searched.

Videos can be modeled.

Converting some tests to use FileGroup.primary_path.

Adding archive and domains to alembic migration.

Testing cascading Archive titles.

Archives now uses FileGroup.

archive.lib tests pass.

Archive is partially converted to FileGroup.

Calculating FileGroup.primary file on refresh, rather than on select.

wrolpi tests pass.

Using FilesType to convert between str/pathlib.Path for FileGroup.files.

Using list of files in FileGroup.files instead of dictionary.

Storing files in dedicated FileGroup.files to avoid overwriting them.

Adding comment explaining CASE.

Not overwriting FileGroup.data on refresh if no change has been detected.

Removing File model in favor of FileGroup.data.

Adding typing to Files methods.

Files are indexed using the primary mimetype of the FileGroup.

files test_lib tests pass.

Removing file_group_file in favor of simple one-to-many relationship.

PDF files are indexed during refresh.

Adding Tag and TagFile to tag files with specific tags.

Only iterating when full_stem_ids has contents.

Testing that FileGroups are related to their Files.

Files and file_groups are created and linked.

Bump webpack from 5.75.0 to 5.76.1 in /app (#160)

Bumps [webpack](https://github.com/webpack/webpack) from 5.75.0 to 5.76.1.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.75.0...v5.76.1)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-25 17:54:41 -06:00
Roland
1812a1690f
Matching return style in all js. 2023-02-17 16:09:24 -07:00
Roland
84802eaf66
Updating dependencies for App testing. 2023-01-07 18:53:22 -07:00
Roland
6a5a73af22
Upgrading Reactjs to version 18. Removing timezone select in favor of using browser's timezone. Switching to @artsy/fresnel for Responsive content. 2023-01-07 18:31:39 -07:00
Roland
2840f9215c
Separating Downloads tables. 2023-01-07 17:22:28 -07:00