Commit graph

354 commits

Author SHA1 Message Date
Roland
a6d6aaf9b4
Adding .ffprobe.json for video file groups to speed up re-indexing in the future; adding save_ffprobe_json to the WROLPiConfig to disable this feature. 2026-02-07 18:40:56 -07:00
Roland
fa5c116b62
Detecting collection directories using collection formatting rather than relying on parent directory of file groups, because those are now longer always in the collection directory. 2026-02-07 17:16:08 -07:00
Roland
8a4f9201f6
Using Channel name when reorganizing channel's files, fallback to info json uploader name (which may be outdated).
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-02-05 11:48:25 -07:00
Roland
f5a48e3e7c
Falling back to video file name parser if we don't have enough info in the video's info json. 2026-02-05 08:52:29 -07:00
Roland
9005c0deff
Adding toggle to disable the tags directory feature. 2026-01-31 18:08:42 -07:00
Roland
74cb5171da
FileGroups in Collections can be reorganized in the Collection's directory.
* Batch reorganization is supported for both Domain and Channel Collections.
* Batch reorganization happens when any Collection does not match the respective file format. For example, a Channel Collection file format that does not match the `yt_dlp_options.file_name_format` will show `needs_reorganization`.
* Individual Collections can always be reorganized. The reorganization is a little more aggressive. Useful for when some files may have been missed in the batch reorganization.
2026-01-31 18:08:41 -07:00
Roland
31af5120b8
Handling edge case when configs have absolute directories. 2026-01-23 15:29:20 -07:00
Roland
173284699d
FileWorker: Async queue-based file synchronization system
Introduces FileWorker for background file operations with dual-queue
architecture (public/private). Handles refresh, move, and rename tasks
with progress tracking, job completion monitoring, and batch processing.
Compares DB FileGroups to filesystem state using FileGroupDiff to detect
new/modified/deleted files. Designed for memory efficiency on Raspberry Pi.
2026-01-23 15:29:15 -07:00
Roland
1d9b794fca
Converting JSON columns to JSONB for performance.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-01-06 06:23:32 -07:00
Roland
2a27d354a5
Creating AutoAwaitTestClient for tests which automatically runs background tasks and expires db objects.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-01-05 18:26:49 -07:00
Roland
3733eb4efd
Always refreshing channel in background task. 2025-12-26 13:43:51 -07:00
Roland
150ad6ee78
Deleting existing orphaned collections when no channel exists in the config. 2025-12-26 11:28:06 -07:00
Roland
b408d11ea1
Deleting orphaned collections when no channel is in the channels config. 2025-12-26 10:00:32 -07:00
Roland
166d6a47c3
Removing PYTEST from channel move to avoid missing session.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2025-12-24 17:42:17 -07:00
Roland
5d50562768
Reworking Cards in the App so that they use the new FileGroup.files and .data. 2025-12-24 12:30:04 -07:00
Roland
298e852484 Creating FileGroup.directory, converting paths to relative to avoid updating all the paths on move. 2025-12-24 11:26:45 -07:00
Roland
641fb368eb
Adding upload_year and upload_month to the video file format option. 2025-12-23 18:49:33 -07:00
Roland
1ebcae0965
Allowing user to configure collection download destination directories, as well as directories in the file structure. This will allow sorting archives by destination year, for example. 2025-12-23 16:46:15 -07:00
Roland
3e3694869b
Fixing bug where not all archives were modeled. 2025-12-18 17:19:57 -07:00
Roland
344fe9035f
Removing optional_session, passing session down from request middleware. Moving session param to the front of functions.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2025-12-11 16:03:18 -07:00
Roland
1747442db3 Migrating move_channel functionality into move_collection so domain files are moved.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2025-12-09 16:44:22 -07:00
Roland
8a7f0737f0 Reformatting code. 2025-12-06 15:11:37 -07:00
Roland
5b29144d70 Testing that nearly-empty configs are ignored on import. Adding setting to disable automatic upgrade checking. 2025-12-06 15:11:37 -07:00
Roland
47044cdd1f Ensuring domain config saves when tagging/untagging a domain. 2025-12-06 15:11:37 -07:00
Roland
d7a02b5735 Speeding up channels config import. 2025-12-06 15:11:37 -07:00
Roland
1d8738ace2 Running alembic check in cicd. 2025-12-06 15:11:37 -07:00
Roland
a1e0fb4e92 Creating a new Collection model for use in organizing FileGroups.
Replacing Channels and Domains with Collection.

Moving downloading relationships from Channel to Collection.

Created domains.yaml config to manage tagged domain collections.

Adding turn-key upgrade solution in the UI.
2025-12-06 15:11:37 -07:00
Roland
c635267eda
Fixing bug where Archive screenshots would not be tracked when a user uploads a screenshot.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
2025-10-12 17:19:54 -06:00
Roland
2bca298ed3
Ignoring videos with no duration when filtering by min/max duration.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
2025-09-14 09:23:21 -06:00
Roland
eb8124b391
Moving bestvideo/bestaudio to end of format selection so the preferred formats are downloaded first.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
2025-09-13 17:56:20 -06:00
Roland
cd2fdce90a
Adding best matching video resolutions at the start of video download format lists. 2025-08-26 16:24:42 -06:00
Roland
b14c153664
Setting default value for channel trigger columns. 2025-08-22 11:02:47 -06:00
Roland
25f9568154 Adding summary columns to Channel table. Adding triggers to keep columns updated. 2025-08-17 10:50:55 -06:00
Roland
338d961284 Adding file_group.effective_datetime to speed up video and archive browsing.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
2025-08-16 11:38:22 -06:00
Roland
76a454ea34
Linking to uploaded file on download after a user uploads a file with the same URL.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
2025-08-11 16:54:10 -06:00
Roland
57081e327e Converting singlefile_upload_switch_handler to use multiprocessing.Queue to enable uploading of multiple singlefile archives at once. 2025-08-06 14:05:21 -06:00
Roland
3725f0f39d Adding edit button so user can edit downloads, currently supports videos and archives. 2025-08-06 13:03:32 -06:00
Roland
f08790af05
Updating test_get_channel_videos_pagination to use channel_factory.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
2025-07-20 16:40:24 -06:00
Roland
2dd182a10e
Adding vertical video formats to video downloader. 2025-07-20 16:17:43 -06:00
Roland
41c52b9c1e
Not using browser profile if it is disabled, even if always_user_browser_profile is enabled. This enables the user to selectively disable cookies. 2025-05-05 12:56:24 -06:00
Roland
deaeae9e9c
Adding more video formats to video resolutions. 2025-05-05 11:13:50 -06:00
Roland
00c86e1a1a
Searching wrolpi user's home directory for browser profiles. 2025-05-02 17:24:51 -06:00
Roland
4900554a1d
Fixing some tests to work on MacOS for development. 2025-04-14 20:21:01 -06:00
Roland
12262b0802
Adding more video format versions now that more have been put out. 2025-03-28 10:27:22 -06:00
Roland
399cb0f107
Testing that downloads have absolute destinations. Handling 'undefined', 'null', etc, dates. 2025-03-21 17:14:53 -06:00
Roland
581d5eaf6e
Adding toggle to Channel page to stop downloading comments for certain channels. 2025-03-01 18:23:29 -07:00
Roland
15f6e7ed5c
Handling new issue with youtube caption file names. Adding QR Code calculator. 2025-02-07 11:44:17 -07:00
Roland
531ec3d65e
Creating new /api/archive/upload which accepts a file from the SingleFile browser extension and converts it to an Archive. 2025-01-30 18:41:32 -07:00
Roland
dcbac00f6e
Updating VideoDownloader formats selection based off new AI language audio. 2025-01-17 11:56:49 -07:00
Roland
b34f5dd594
Deleting temporary video files after video has been downloaded. 2025-01-13 19:14:45 -07:00