Commit graph

113 commits

Author SHA1 Message Date
Roland
b05b2e7a6d
Allowing maintainer to change special directories. 2024-03-09 16:51:26 -07:00
Roland
df849ffb42
Ordering video/archive results when no published_datetime exists. Adding Video.location property which returns the App location of the video. 2024-02-26 13:40:39 -07:00
Roland
61a7fdab10
Creating HTMLIndexer which indexes the text of HTML files. 2024-02-19 15:48:17 -07:00
Roland
59b23b3134
Creating function which appends tag names sub-select. 2024-02-10 21:11:19 -07:00
Roland
1b92d1e580
Global search uses Semantic SearchResultsInput component, other components use SearchInput. 2023-11-21 13:01:52 -07:00
Roland
73cb142ae1
Reverting to older singlefile container to keep archive screenshot ratio. Using FileGroup.url on ArchivePage. 2023-10-31 10:20:45 -06:00
Roland
9ea2dadde5
Raising specific error when datetime cannot be parsed. 2023-10-29 22:15:57 -06:00
Roland
15687e534a
Moving data from models to FileGroup.
* Ebook.creator -> FileGroup.author
* Video.viewed -> FileGroup.viewed
* Video.duration -> FileGroup.length
* Video.url -> FileGroup.url
* Archive.url -> FileGroup.url
2023-10-29 17:31:54 -06:00
Roland
ae9d188391
Adding FileGroup.published_date/author/etc. Creating script which converts a Singlefile to an Archive FileGroup files.
Parsing SingleFile datetime using dates.strpdate.

Adding timezone when no timezone info is available.

Installing python-dateutil to assist in handling diverse datetime strings.

Reworking refresh progress to use idempotency to display more accurate progress.  Only logging that files have been invalidated if they used to be indexed.

Clearing archives to force re-index.

Ignoring 'Unknown' author from PDF metadata. Using file_title when pdf metadata title is empty.
2023-10-23 18:09:08 -06:00
Roland
b817534c37
Adding Download.is_new/etc. for code suggestions. Fixing async test warnings. 2023-09-20 12:58:35 -06:00
Roland
a510960c8c
Fixing syntax error. 2023-09-15 16:32:09 -06:00
Roland
f14df90174
Removing unused imports. 2023-09-12 17:49:26 -06:00
Roland
17b3923747
Do not need to fetch all archives when checking if domain has archives. 2023-09-12 17:49:17 -06:00
Roland
7611cf8ea1
Converted ArchiveDownloader to use Downloader.process_runner so archiving can be canceled.
Moving Archive download functions to ArchiveDownloader.

Wrapping all do_download with cancel_wrapper which allows canceling of downloads.
2023-09-12 17:32:29 -06:00
Roland
6959c7a3b4
HTML and JSON files are reformatted when downloaded. 2023-08-26 21:49:24 -06:00
Roland
14017d43bb
Creating Video/Archive get_by_id/find_by_id methods. find_by_id will raise an exception when not found, get_by_id will not. 2023-08-17 18:46:42 -06:00
Roland
36083200e6
Creating Video.ffprobe_json column which stores ffprobe information about the video file. Creating find_corrupt_videos.py script which automatically re-downloads videos which do not have both video and audio streams. 2023-08-03 21:40:25 -06:00
Roland
59c4ab2502
Replacing icon property when icon has been requested.
Creating <APIButton/> which handles success/failure/pending during API calls.
2023-07-21 16:52:15 -06:00
Roland
fdaaa510c4
Splitting errors into their modules. Removing API_ERRORS dict in favor of storing error data on the class. 2023-07-17 14:21:27 -06:00
Roland
03a2df217d
Zim/Kiwix (Wikipedia) feature
* A new zim module has been added, this module allows several things to interact with Zim files.
  * Allow searching of Zim files using libzim.
  * Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
  * Creation of a new zim docker container which allows viewing of the Zim file contents.

* This commit also adds move/rename/mkdir features in the File Browser.
  * File tags are preserved when moving/renaming.
2023-07-10 21:30:40 -06:00
Roland
bc2ca0e512
Replacing tag filtering with new tag_names_to_sub_select function which uses a single sub-select to filter FileGroups. 2023-05-18 22:39:19 -06:00
Roland
fedbbe31d7
Changing default Survival tag to Primitive. 2023-05-18 17:25:33 -06:00
Roland
ac06a9b8b2
Joining when sorting by size. 2023-05-16 14:59:35 -06:00
Roland
7c1cc07365
Only joining FileGroup table when searching Archives when necessary. 2023-05-16 14:56:53 -06:00
Roland
e06f846c8c
Updating function names to match their functionality. 2023-05-03 13:05:57 -06:00
Roland
8de805b6d1
Updating tests to match searching for Channel and Domain names for directories. 2023-05-03 12:59:47 -06:00
Roland
387282b445
Adding my public gpg key.
Moving upload destination input out of drop Segment.

Disabling uploads when downloads are disabled.

Inverting labels for checkboxes in inverted modal.

Deleting file_groups which no longer contain the exclusive primary path.

Adding debug logging for all API errors.

Checking that upload output is not a directory.

Expanding nginx request size.

Adding docs to upload endpoint.

Using dot-circle icon for more image mimetypes.

Using Form.Field to make upload destination required.

Adding border to inverted Segments.

Many files can be uploaded in chunks.

Files can be uploaded, progress is displayed.

Showing white border around share QR code.

Displaying WROLPi colors for download icons.

Creating button to clear DirectorySearch.

Excluding channel/domain directories so other directories can be returned.

Ordering search results.

Ignoring when no channel is found only if we have a destination.

Limiting directory search results.

Displaying categories when searching destination directories.

Handling when live_chat.json file exists.

Allowing refresh of directory directly.

Logging which download was successful.

Removing timeout.

Creating index for directory names.

Sleeping to catch cancel.

Arbitrary files can be downloaded using the new FileDownloader.
2023-04-28 13:56:32 -06:00
Roland
f9ad9981db
Testing Archive headline search. 2023-04-12 19:07:16 -06:00
Roland
bba6a341b7
Only fetching headlines if requested to speed up RPi responses.g 2023-04-12 18:51:26 -06:00
Roland
29e562c520
Headlines can be displayed for file search results. 2023-04-12 18:09:10 -06:00
Roland
5e206fd9a2
Adding more details to Archive slow_logger. 2023-04-12 11:51:28 -06:00
Roland
6fe536558a
You cannot delete a tagged FileGroup. 2023-04-12 11:42:13 -06:00
Roland
18273286dc
Removing FileGroup.full_stem in favor of primary_path. This is to handle when two unrelated files share a name ('some text.txt' and 'some text.jpg'). 2023-04-09 16:59:20 -06:00
Roland
bab10a0c96
Creating sort button which allows the user to select a sort order via modal. 2023-04-09 14:13:36 -06:00
Roland
9ca099267b
Refreshing file when attempting to add Tag to file that has not been refreshed. Adding url to skip list when deleting download. 2023-04-05 10:00:08 -06:00
Roland
ede9d2236b
Video and Archive can be tagged when downloading. 2023-04-03 09:34:42 -06:00
Roland
55cf76a884
Requiring downloader name to be specified. 2023-04-01 15:34:46 -06:00
Roland
ae2b6daa33
FileBrowser files can be tagged. A Tag icon is displayed on each file row in the browser. 2023-04-01 11:52:41 -06:00
Roland
a7c4fd1119
Displaying progress bar during refresh. 2023-03-27 20:46:31 -06:00
Roland
b5a9e45806
Increasing logging during refresh. Not setting channel_id in file_group.data in favor of using video.channel_id. 2023-03-27 10:02:47 -06:00
Roland
5615ce542d
Making example label a tag.
Tags can be created and deleted in UI.

Adding tag_file using id to avoid session error.

Using UTC in test.

Matching tag_names style in test.

Tag icon is displayed on file rows.

Archives, Videos and Files can be filtered by their tags.

Tags can be added and removed from UI.

Tags are displayed in VideoPlayer.

The tags of a file are returned in it's JSON.

Video favorites are converted to Favorite tag on channel config import.

Updating tests to match frontend usage.

PDF creator and modification time are displayed.

Reworking file search to use Video.__json__ for complex video relations.

Archives can be viewed.

Removing ebook.title.

Reworking alembic migration to be an upgrade rather than a replace.

Skipping test on circleci.

Clearing out video caption index only if extract_subtitles if off.

Channel tests pass.

Some video tests pass.

Missing videos can be found.  A video can be downloaded.

Orphaned files can be found.

Videos can be tagged.

Videos can be searched.

Videos can be modeled.

Converting some tests to use FileGroup.primary_path.

Adding archive and domains to alembic migration.

Testing cascading Archive titles.

Archives now uses FileGroup.

archive.lib tests pass.

Archive is partially converted to FileGroup.

Calculating FileGroup.primary file on refresh, rather than on select.

wrolpi tests pass.

Using FilesType to convert between str/pathlib.Path for FileGroup.files.

Using list of files in FileGroup.files instead of dictionary.

Storing files in dedicated FileGroup.files to avoid overwriting them.

Adding comment explaining CASE.

Not overwriting FileGroup.data on refresh if no change has been detected.

Removing File model in favor of FileGroup.data.

Adding typing to Files methods.

Files are indexed using the primary mimetype of the FileGroup.

files test_lib tests pass.

Removing file_group_file in favor of simple one-to-many relationship.

PDF files are indexed during refresh.

Adding Tag and TagFile to tag files with specific tags.

Only iterating when full_stem_ids has contents.

Testing that FileGroups are related to their Files.

Files and file_groups are created and linked.

Bump webpack from 5.75.0 to 5.76.1 in /app (#160)

Bumps [webpack](https://github.com/webpack/webpack) from 5.75.0 to 5.76.1.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.75.0...v5.76.1)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-25 17:54:41 -06:00
Roland
b373992fed
Increasing archive file name character limit. 2023-02-27 14:42:58 -07:00
Roland
95c8610df2
File.d_text is reformatted to break up long lines. 2022-12-27 15:41:55 -07:00
Roland
93f0e7494f
Creating Event and Flag systems. Events are used to send and display a toast to the App. Flags are used to signal if some action has been performed, or if a package is installed. 2022-12-23 16:04:09 -07:00
Roland
a149cc8f69
Reworking modelers to only fetch matching models once to avoid making many queries per group. 2022-12-14 18:52:42 -07:00
Roland
ff292b6e3f
Increasing Singlefile header size because some URLs may be very long. 2022-12-14 15:14:50 -07:00
Roland
11bd59cec0
Changing Archive Sync to Update and Alternatives to History. 2022-11-14 15:42:57 -07:00
Roland
aef7f1162d
Moving like methods near eachother. 2022-11-03 16:52:14 -06:00
Roland
50f598558c
Only skipping read_singlefile_data when no paths are available. 2022-11-02 16:18:17 -06:00
Roland
7db4ea73f2
Archive alternatives must have a URL. 2022-11-02 15:12:48 -06:00