Commit graph

365 commits

Author SHA1 Message Date
Roland
71f20d8c59
Using shared context to sync cookies between processes. 2026-02-22 14:32:27 -07:00
Roland
0c059865e2
Updating download process to prevent rate limiting.
* Added encrypted cookies feature for use with yt-dlp. Decrypted cookies never touch the disk.
* Adding sleep between all downloads to prevent rapid requests.
* Channels are fully refreshed before downloading.
* Removing superseded video browser profile.
2026-02-21 14:08:00 -07:00
Roland
4537c63d89
Adding DEBUS to api systemd services. 2026-02-19 18:01:55 -07:00
Roland
929468821b
Using gnomekeyring for yt-dlp brave cookie parsing. 2026-02-19 17:44:57 -07:00
Roland
a49f67e117
Fetching video duration using api rather than using overpowered yt-dlp. 2026-02-19 16:49:38 -07:00
Roland
d8c51db5a3
Fetching video comments uses video downloader browser profile. 2026-02-19 16:25:35 -07:00
Roland
ae17152e79
Expanding video browser profile selection to match archive feature. Adding toggle to disable downloading missing video comments. 2026-02-19 16:15:18 -07:00
Roland
99a716aa13
Storing parent url for every child download, useful for knowing why WROLPi is downloading something. This is displayed on the Video and Archive download edit page. Updating Archive Download edit modal to use single url input.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-02-18 15:16:10 -07:00
Roland
3f5470d974
Channels are linked to downloads by directory or download source id when downloading, or when importing config. 2026-02-08 14:27:59 -07:00
Roland
a5dee64af2
Changing video view_count to bigint for popular videos.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-02-08 12:33:37 -07:00
Roland
387ef6dfe8
Saving video ffprobe json when video file is uploaded. 2026-02-08 12:29:25 -07:00
Roland
a6d6aaf9b4
Adding .ffprobe.json for video file groups to speed up re-indexing in the future; adding save_ffprobe_json to the WROLPiConfig to disable this feature. 2026-02-07 18:40:56 -07:00
Roland
fa5c116b62
Detecting collection directories using collection formatting rather than relying on parent directory of file groups, because those are now longer always in the collection directory. 2026-02-07 17:16:08 -07:00
Roland
8a4f9201f6
Using Channel name when reorganizing channel's files, fallback to info json uploader name (which may be outdated).
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-02-05 11:48:25 -07:00
Roland
f5a48e3e7c
Falling back to video file name parser if we don't have enough info in the video's info json. 2026-02-05 08:52:29 -07:00
Roland
9005c0deff
Adding toggle to disable the tags directory feature. 2026-01-31 18:08:42 -07:00
Roland
74cb5171da
FileGroups in Collections can be reorganized in the Collection's directory.
* Batch reorganization is supported for both Domain and Channel Collections.
* Batch reorganization happens when any Collection does not match the respective file format. For example, a Channel Collection file format that does not match the `yt_dlp_options.file_name_format` will show `needs_reorganization`.
* Individual Collections can always be reorganized. The reorganization is a little more aggressive. Useful for when some files may have been missed in the batch reorganization.
2026-01-31 18:08:41 -07:00
Roland
31af5120b8
Handling edge case when configs have absolute directories. 2026-01-23 15:29:20 -07:00
Roland
173284699d
FileWorker: Async queue-based file synchronization system
Introduces FileWorker for background file operations with dual-queue
architecture (public/private). Handles refresh, move, and rename tasks
with progress tracking, job completion monitoring, and batch processing.
Compares DB FileGroups to filesystem state using FileGroupDiff to detect
new/modified/deleted files. Designed for memory efficiency on Raspberry Pi.
2026-01-23 15:29:15 -07:00
Roland
1d9b794fca
Converting JSON columns to JSONB for performance.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-01-06 06:23:32 -07:00
Roland
2a27d354a5
Creating AutoAwaitTestClient for tests which automatically runs background tasks and expires db objects.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-01-05 18:26:49 -07:00
Roland
3733eb4efd
Always refreshing channel in background task. 2025-12-26 13:43:51 -07:00
Roland
150ad6ee78
Deleting existing orphaned collections when no channel exists in the config. 2025-12-26 11:28:06 -07:00
Roland
b408d11ea1
Deleting orphaned collections when no channel is in the channels config. 2025-12-26 10:00:32 -07:00
Roland
166d6a47c3
Removing PYTEST from channel move to avoid missing session.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2025-12-24 17:42:17 -07:00
Roland
5d50562768
Reworking Cards in the App so that they use the new FileGroup.files and .data. 2025-12-24 12:30:04 -07:00
Roland
298e852484 Creating FileGroup.directory, converting paths to relative to avoid updating all the paths on move. 2025-12-24 11:26:45 -07:00
Roland
641fb368eb
Adding upload_year and upload_month to the video file format option. 2025-12-23 18:49:33 -07:00
Roland
1ebcae0965
Allowing user to configure collection download destination directories, as well as directories in the file structure. This will allow sorting archives by destination year, for example. 2025-12-23 16:46:15 -07:00
Roland
3e3694869b
Fixing bug where not all archives were modeled. 2025-12-18 17:19:57 -07:00
Roland
344fe9035f
Removing optional_session, passing session down from request middleware. Moving session param to the front of functions.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2025-12-11 16:03:18 -07:00
Roland
1747442db3 Migrating move_channel functionality into move_collection so domain files are moved.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2025-12-09 16:44:22 -07:00
Roland
8a7f0737f0 Reformatting code. 2025-12-06 15:11:37 -07:00
Roland
5b29144d70 Testing that nearly-empty configs are ignored on import. Adding setting to disable automatic upgrade checking. 2025-12-06 15:11:37 -07:00
Roland
47044cdd1f Ensuring domain config saves when tagging/untagging a domain. 2025-12-06 15:11:37 -07:00
Roland
d7a02b5735 Speeding up channels config import. 2025-12-06 15:11:37 -07:00
Roland
1d8738ace2 Running alembic check in cicd. 2025-12-06 15:11:37 -07:00
Roland
a1e0fb4e92 Creating a new Collection model for use in organizing FileGroups.
Replacing Channels and Domains with Collection.

Moving downloading relationships from Channel to Collection.

Created domains.yaml config to manage tagged domain collections.

Adding turn-key upgrade solution in the UI.
2025-12-06 15:11:37 -07:00
Roland
c635267eda
Fixing bug where Archive screenshots would not be tracked when a user uploads a screenshot.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
2025-10-12 17:19:54 -06:00
Roland
2bca298ed3
Ignoring videos with no duration when filtering by min/max duration.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
2025-09-14 09:23:21 -06:00
Roland
eb8124b391
Moving bestvideo/bestaudio to end of format selection so the preferred formats are downloaded first.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
2025-09-13 17:56:20 -06:00
Roland
cd2fdce90a
Adding best matching video resolutions at the start of video download format lists. 2025-08-26 16:24:42 -06:00
Roland
b14c153664
Setting default value for channel trigger columns. 2025-08-22 11:02:47 -06:00
Roland
25f9568154 Adding summary columns to Channel table. Adding triggers to keep columns updated. 2025-08-17 10:50:55 -06:00
Roland
338d961284 Adding file_group.effective_datetime to speed up video and archive browsing.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
2025-08-16 11:38:22 -06:00
Roland
76a454ea34
Linking to uploaded file on download after a user uploads a file with the same URL.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
2025-08-11 16:54:10 -06:00
Roland
57081e327e Converting singlefile_upload_switch_handler to use multiprocessing.Queue to enable uploading of multiple singlefile archives at once. 2025-08-06 14:05:21 -06:00
Roland
3725f0f39d Adding edit button so user can edit downloads, currently supports videos and archives. 2025-08-06 13:03:32 -06:00
Roland
f08790af05
Updating test_get_channel_videos_pagination to use channel_factory.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
2025-07-20 16:40:24 -06:00
Roland
2dd182a10e
Adding vertical video formats to video downloader. 2025-07-20 16:17:43 -06:00