Roland
1951ca548d
Using stubs to simplify tests.
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-05-07 20:07:46 -06:00
Roland
8f0bb78722
Converting RSS and Video Downloaders to new testable strategy.
2026-05-07 20:07:31 -06:00
Roland
f80b3ea769
Fixing bug where Channel Downloader would create empty videos.xml channel if RSS feed was sent to it.
2026-05-07 15:54:53 -06:00
Roland
ba63170e55
Consolidating deletion endpoints.
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-05-03 19:14:30 -06:00
Roland
09a967535b
Models like Archive/Video/Doc can be deleted, even if they are tagged. User is shown a warning and must confirm deletion of tagged files.
2026-05-03 19:14:15 -06:00
Roland
9468448a54
Modifying Video/Channel/RSS Downloaders so that they can download videos, but audio only. These audio files are grouped under their channel directories.
2026-04-14 15:33:01 -06:00
Roland
250575ef87
Delete duplicate video on download, even if it has tags.
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-04-04 19:05:16 -06:00
Roland
105ae377f9
Video download destination can download to the incorrect Channel.
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-04-04 15:52:47 -06:00
Roland
b8a532c8d9
Key changes:
...
- Add MapLibre GL JS viewer with light/dark themes, terrain layer,
and embedded static map page for coordinate sharing
- Add map pin support with persistent map_pins.yaml config
- Add map region subscriptions and downloader with progress tracking,
sourcing pre-extracted PMTiles from a cold CDN
- Add map export to PNG and GPS coordinate copy via context menu
- Add MapViewer React component alongside reworked Map page
- Add map catalog and region-building script (build_map_regions.py)
- Remove OSM import/render scripts, leaflet files, Apache configs,
renderd service, and osm-map-dumper Docker container
- Update Caddyfile to serve PMTiles and map static assets
- Update build/install/repair scripts for Protomaps dependencies
- Drop map_file table, add/drop map_subscription table (alembic)
- Adding button to skip primary drive mounting during onboarding.
2026-04-04 13:48:19 -06:00
Roland
755b133fa4
Fixing bug where configs could be overwritten with empty configs on first refresh. Fixing bug where configs were displayed as JSON in admin settings.
2026-03-22 12:56:37 -06:00
Roland
00cfdc14c6
Trying to get existing channel using more variations of Youtube URLs.
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-03-20 17:26:51 -06:00
Roland
66a891c759
Refactoring videos and archives UI paths for consistent scheme.
2026-03-20 15:38:32 -06:00
Roland
aefc7b1def
Add Docs module for searching and organizing PDFs and EPUBs
...
Create a new Docs module that consolidates PDF and eBook handling into a
unified system with subjects and authors support. This replaces the
separate PDF/eBook modelers with a single Doc model backed by an
alembic migration (rename ebook to doc).
Key changes:
- New modules/docs package with models, API, extractors, and lib
- Subject and author metadata extraction with case-insensitive
normalization for cleaner PDF metadata
- Frontend Docs pages with AuthorsPage, SubjectsPage using
CollectionTable, and doc card deletion
- Global search integration for subjects and authors
- Generic collection item count support
- Frontend file card metadata adjusted to match collection sort
- Bump yt-dlp version
2026-03-20 14:17:06 -06:00
Roland
e6aaab1016
Creating video duration cache to avoid requesting video duration between restarts.
2026-03-20 13:02:46 -06:00
Roland
b1db71f04a
Handling when skip_urls is a None. Logging a warning (and ignoring) when duplicate json files are in a Video FileGroup.
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-03-13 15:27:43 -06:00
Roland
4579562be5
Displaying download progress (speed, eta) for file and video downloads.
2026-03-12 19:31:35 -06:00
Roland
da6e62ee45
Backup configs can be restored in the UI. User can choose between Merge or Overwrite, the UI displays a comparison. Configs are compared when user opens the restore modal, configs which won't cause changes are marked.
2026-03-10 20:30:46 -06:00
Roland
1cf402821c
Renaming VTT files on video download if there are extra characters in the name.
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-03-09 15:00:24 -06:00
Roland
8c294dad39
Downloading only the first video when the VideoDownloader gets a playlist by accident.
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-03-08 11:55:59 -06:00
Roland
b2185ad7c3
Failing all downloads of a domain if cookies errors are received.
2026-03-08 11:08:08 -06:00
Roland
d2d09da296
Video downloads are retried in an hour, if they are live.
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-03-06 15:22:27 -07:00
Roland
0ffbc05e91
Unifying video download settings so that global/channel/rss/video
...
download settings have inheritance. Allowing more granular control of
potentially failed downloads.
2026-03-05 17:40:17 -07:00
Roland
9e52f658ea
Storing parent_download_url in a file group's json file after download has completed.
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-03-03 15:54:22 -07:00
Roland
71f20d8c59
Using shared context to sync cookies between processes.
2026-02-22 14:32:27 -07:00
Roland
0c059865e2
Updating download process to prevent rate limiting.
...
* Added encrypted cookies feature for use with yt-dlp. Decrypted cookies never touch the disk.
* Adding sleep between all downloads to prevent rapid requests.
* Channels are fully refreshed before downloading.
* Removing superseded video browser profile.
2026-02-21 14:08:00 -07:00
Roland
4537c63d89
Adding DEBUS to api systemd services.
2026-02-19 18:01:55 -07:00
Roland
929468821b
Using gnomekeyring for yt-dlp brave cookie parsing.
2026-02-19 17:44:57 -07:00
Roland
d8c51db5a3
Fetching video comments uses video downloader browser profile.
2026-02-19 16:25:35 -07:00
Roland
ae17152e79
Expanding video browser profile selection to match archive feature. Adding toggle to disable downloading missing video comments.
2026-02-19 16:15:18 -07:00
Roland
99a716aa13
Storing parent url for every child download, useful for knowing why WROLPi is downloading something. This is displayed on the Video and Archive download edit page. Updating Archive Download edit modal to use single url input.
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-02-18 15:16:10 -07:00
Roland
3f5470d974
Channels are linked to downloads by directory or download source id when downloading, or when importing config.
2026-02-08 14:27:59 -07:00
Roland
387ef6dfe8
Saving video ffprobe json when video file is uploaded.
2026-02-08 12:29:25 -07:00
Roland
a6d6aaf9b4
Adding .ffprobe.json for video file groups to speed up re-indexing in the future; adding save_ffprobe_json to the WROLPiConfig to disable this feature.
2026-02-07 18:40:56 -07:00
Roland
fa5c116b62
Detecting collection directories using collection formatting rather than relying on parent directory of file groups, because those are now longer always in the collection directory.
2026-02-07 17:16:08 -07:00
Roland
8a4f9201f6
Using Channel name when reorganizing channel's files, fallback to info json uploader name (which may be outdated).
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-02-05 11:48:25 -07:00
Roland
74cb5171da
FileGroups in Collections can be reorganized in the Collection's directory.
...
* Batch reorganization is supported for both Domain and Channel Collections.
* Batch reorganization happens when any Collection does not match the respective file format. For example, a Channel Collection file format that does not match the `yt_dlp_options.file_name_format` will show `needs_reorganization`.
* Individual Collections can always be reorganized. The reorganization is a little more aggressive. Useful for when some files may have been missed in the batch reorganization.
2026-01-31 18:08:41 -07:00
Roland
31af5120b8
Handling edge case when configs have absolute directories.
2026-01-23 15:29:20 -07:00
Roland
173284699d
FileWorker: Async queue-based file synchronization system
...
Introduces FileWorker for background file operations with dual-queue
architecture (public/private). Handles refresh, move, and rename tasks
with progress tracking, job completion monitoring, and batch processing.
Compares DB FileGroups to filesystem state using FileGroupDiff to detect
new/modified/deleted files. Designed for memory efficiency on Raspberry Pi.
2026-01-23 15:29:15 -07:00
Roland
2a27d354a5
Creating AutoAwaitTestClient for tests which automatically runs background tasks and expires db objects.
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-01-05 18:26:49 -07:00
Roland
150ad6ee78
Deleting existing orphaned collections when no channel exists in the config.
2025-12-26 11:28:06 -07:00
Roland
b408d11ea1
Deleting orphaned collections when no channel is in the channels config.
2025-12-26 10:00:32 -07:00
Roland
298e852484
Creating FileGroup.directory, converting paths to relative to avoid updating all the paths on move.
2025-12-24 11:26:45 -07:00
Roland
641fb368eb
Adding upload_year and upload_month to the video file format option.
2025-12-23 18:49:33 -07:00
Roland
3e3694869b
Fixing bug where not all archives were modeled.
2025-12-18 17:19:57 -07:00
Roland
344fe9035f
Removing optional_session, passing session down from request middleware. Moving session param to the front of functions.
CodeQL / Analyze (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2025-12-11 16:03:18 -07:00
Roland
8a7f0737f0
Reformatting code.
2025-12-06 15:11:37 -07:00
Roland
5b29144d70
Testing that nearly-empty configs are ignored on import. Adding setting to disable automatic upgrade checking.
2025-12-06 15:11:37 -07:00
Roland
a1e0fb4e92
Creating a new Collection model for use in organizing FileGroups.
...
Replacing Channels and Domains with Collection.
Moving downloading relationships from Channel to Collection.
Created domains.yaml config to manage tagged domain collections.
Adding turn-key upgrade solution in the UI.
2025-12-06 15:11:37 -07:00
Roland
c635267eda
Fixing bug where Archive screenshots would not be tracked when a user uploads a screenshot.
CodeQL / Analyze (push) Has been cancelled
2025-10-12 17:19:54 -06:00
Roland
76a454ea34
Linking to uploaded file on download after a user uploads a file with the same URL.
CodeQL / Analyze (push) Has been cancelled
2025-08-11 16:54:10 -06:00