Roland
4080b7a641
Refactoring ArchiveDownloader so that it is more testable.
2026-05-07 15:54:53 -06:00
Roland
ba63170e55
Consolidating deletion endpoints.
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-05-03 19:14:30 -06:00
Roland
09a967535b
Models like Archive/Video/Doc can be deleted, even if they are tagged. User is shown a warning and must confirm deletion of tagged files.
2026-05-03 19:14:15 -06:00
Roland
755b133fa4
Fixing bug where configs could be overwritten with empty configs on first refresh. Fixing bug where configs were displayed as JSON in admin settings.
2026-03-22 12:56:37 -06:00
Roland
1c4f57226f
Adding skip_viewed to archive and video fetching endpoints.
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-03-20 20:17:27 -06:00
Roland
66a891c759
Refactoring videos and archives UI paths for consistent scheme.
2026-03-20 15:38:32 -06:00
Roland
da6e62ee45
Backup configs can be restored in the UI. User can choose between Merge or Overwrite, the UI displays a comparison. Configs are compared when user opens the restore modal, configs which won't cause changes are marked.
2026-03-10 20:30:46 -06:00
Roland
fa5c116b62
Detecting collection directories using collection formatting rather than relying on parent directory of file groups, because those are now longer always in the collection directory.
2026-02-07 17:16:08 -07:00
Roland
2c8f765f2b
Adding statistics page for Archives.
2026-02-02 15:15:28 -07:00
Roland
74cb5171da
FileGroups in Collections can be reorganized in the Collection's directory.
...
* Batch reorganization is supported for both Domain and Channel Collections.
* Batch reorganization happens when any Collection does not match the respective file format. For example, a Channel Collection file format that does not match the `yt_dlp_options.file_name_format` will show `needs_reorganization`.
* Individual Collections can always be reorganized. The reorganization is a little more aggressive. Useful for when some files may have been missed in the batch reorganization.
2026-01-31 18:08:41 -07:00
Roland
2b0b161dad
Changing archive special directory default to include domain tag.
2025-12-25 15:42:35 -07:00
Roland
4ba5226c68
Adding ext to archive file format to match video file format.
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2025-12-23 19:11:07 -07:00
Roland
80decbd560
Marking archives_downloader config as imported when it is imported.
2025-12-23 17:42:46 -07:00
Roland
1ebcae0965
Allowing user to configure collection download destination directories, as well as directories in the file structure. This will allow sorting archives by destination year, for example.
2025-12-23 16:46:15 -07:00
Roland
a8ff1a1c38
Updating archive downloader to use download.destination.
2025-12-17 16:56:58 -07:00
Roland
344fe9035f
Removing optional_session, passing session down from request middleware. Moving session param to the front of functions.
CodeQL / Analyze (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2025-12-11 16:03:18 -07:00
Roland
8a7f0737f0
Reformatting code.
2025-12-06 15:11:37 -07:00
Roland
5b29144d70
Testing that nearly-empty configs are ignored on import. Adding setting to disable automatic upgrade checking.
2025-12-06 15:11:37 -07:00
Roland
47044cdd1f
Ensuring domain config saves when tagging/untagging a domain.
2025-12-06 15:11:37 -07:00
Roland
d7a02b5735
Speeding up channels config import.
2025-12-06 15:11:37 -07:00
Roland
79e0700609
Fixing local imports.
2025-12-06 15:11:37 -07:00
Roland
a1e0fb4e92
Creating a new Collection model for use in organizing FileGroups.
...
Replacing Channels and Domains with Collection.
Moving downloading relationships from Channel to Collection.
Created domains.yaml config to manage tagged domain collections.
Adding turn-key upgrade solution in the UI.
2025-12-06 15:11:37 -07:00
Roland
1ed0d63bac
Creating Generate Screenshot button for Archives which are missing a screenshot.
2025-12-06 15:11:37 -07:00
Roland
338d961284
Adding file_group.effective_datetime to speed up video and archive browsing.
CodeQL / Analyze (push) Waiting to run
2025-08-16 11:38:22 -06:00
Roland
76b9ead174
Handling rare error when parsing articles.
CodeQL / Analyze (push) Waiting to run
2025-08-06 15:01:40 -06:00
Roland
00c86e1a1a
Searching wrolpi user's home directory for browser profiles.
2025-05-02 17:24:51 -06:00
Roland
13407af64c
Fixing Archive download sorting by moving nulls to the end.
2025-02-04 17:11:11 -07:00
Roland
531ec3d65e
Creating new /api/archive/upload which accepts a file from the SingleFile browser extension and converts it to an Archive.
2025-01-30 18:41:32 -07:00
Roland
82acca00b6
Changing aiohttp function to be context managers to simplify calls.
2024-10-16 18:56:49 -06:00
Roland
0f232fd533
Moving FileGroups instead of refreshing files after they have been moved.
2024-10-16 18:56:25 -06:00
Roland
45167e4f32
A Channel can be tagged with a single Tag. The Channel can be moved into a Tag Directory according to the config.
2024-10-16 18:56:19 -06:00
Roland
53d2402d32
Only files that have been tagged will be returned when new 'Any' tag button is used to filter file search results.
2024-10-16 18:55:56 -06:00
Roland
cdc73d9d16
Trimming video file name to avoid error when file name is too long.
2024-10-16 18:55:52 -06:00
Roland
0bee458b79
Sorting keys when saving json file.
2024-06-13 14:08:07 -06:00
Roland
b05b2e7a6d
Allowing maintainer to change special directories.
2024-03-09 16:51:26 -07:00
Roland
df849ffb42
Ordering video/archive results when no published_datetime exists. Adding Video.location property which returns the App location of the video.
2024-02-26 13:40:39 -07:00
Roland
61a7fdab10
Creating HTMLIndexer which indexes the text of HTML files.
2024-02-19 15:48:17 -07:00
Roland
59b23b3134
Creating function which appends tag names sub-select.
2024-02-10 21:11:19 -07:00
Roland
1b92d1e580
Global search uses Semantic SearchResultsInput component, other components use SearchInput.
2023-11-21 13:01:52 -07:00
Roland
73cb142ae1
Reverting to older singlefile container to keep archive screenshot ratio. Using FileGroup.url on ArchivePage.
2023-10-31 10:20:45 -06:00
Roland
9ea2dadde5
Raising specific error when datetime cannot be parsed.
2023-10-29 22:15:57 -06:00
Roland
15687e534a
Moving data from models to FileGroup.
...
* Ebook.creator -> FileGroup.author
* Video.viewed -> FileGroup.viewed
* Video.duration -> FileGroup.length
* Video.url -> FileGroup.url
* Archive.url -> FileGroup.url
2023-10-29 17:31:54 -06:00
Roland
ae9d188391
Adding FileGroup.published_date/author/etc. Creating script which converts a Singlefile to an Archive FileGroup files.
...
Parsing SingleFile datetime using dates.strpdate.
Adding timezone when no timezone info is available.
Installing python-dateutil to assist in handling diverse datetime strings.
Reworking refresh progress to use idempotency to display more accurate progress. Only logging that files have been invalidated if they used to be indexed.
Clearing archives to force re-index.
Ignoring 'Unknown' author from PDF metadata. Using file_title when pdf metadata title is empty.
2023-10-23 18:09:08 -06:00
Roland
b817534c37
Adding Download.is_new/etc. for code suggestions. Fixing async test warnings.
2023-09-20 12:58:35 -06:00
Roland
7611cf8ea1
Converted ArchiveDownloader to use Downloader.process_runner so archiving can be canceled.
...
Moving Archive download functions to ArchiveDownloader.
Wrapping all do_download with cancel_wrapper which allows canceling of downloads.
2023-09-12 17:32:29 -06:00
Roland
6959c7a3b4
HTML and JSON files are reformatted when downloaded.
2023-08-26 21:49:24 -06:00
Roland
14017d43bb
Creating Video/Archive get_by_id/find_by_id methods. find_by_id will raise an exception when not found, get_by_id will not.
2023-08-17 18:46:42 -06:00
Roland
36083200e6
Creating Video.ffprobe_json column which stores ffprobe information about the video file. Creating find_corrupt_videos.py script which automatically re-downloads videos which do not have both video and audio streams.
2023-08-03 21:40:25 -06:00
Roland
03a2df217d
Zim/Kiwix (Wikipedia) feature
...
* A new zim module has been added, this module allows several things to interact with Zim files.
* Allow searching of Zim files using libzim.
* Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
* Creation of a new zim docker container which allows viewing of the Zim file contents.
* This commit also adds move/rename/mkdir features in the File Browser.
* File tags are preserved when moving/renaming.
2023-07-10 21:30:40 -06:00
Roland
bc2ca0e512
Replacing tag filtering with new tag_names_to_sub_select function which uses a single sub-select to filter FileGroups.
2023-05-18 22:39:19 -06:00
