Commit graph

75 commits

Author SHA1 Message Date
lrnselfreliance
9a5d1f1630
Add ordered collections (playlists): polymorphic items + item API (#475)
* Add Playlists: ordered, mixed-type collections

A playlist (Collection kind='playlist') is a manually-curated, ordered list that
mixes item types. Squash of the playlists feature branch (24 commits).

Data model & API:
- CollectionItem is polymorphic via item_kind: 'file' (FileGroup), 'zim'
  (zim_id + zim_entry), or 'url' (link to anything the WROLPi browser opens,
  e.g. /map?lat=&lon=&z=). CHECK constraint enforces exactly one kind; per-kind
  unique constraints; migration a7b1c2d3e4f5.
- Endpoints: create playlist; add/remove/reorder items (idempotent adds: 201 new,
  200 existing); GET returns the ordered items. All mutations wrol_mode_check'd.

Config (source of truth):
- playlists.yaml (PlaylistsConfig) stores each playlist's metadata AND ordered
  items (file by relative path, zim by path+entry, url inline); round-trips a
  DB wipe. Supports backup import (DB_CONFIG_FILE_NAMES).
- The redundant generic CollectionsConfig (collections.yaml) is deleted:
  channels/domains own their configs, authors/subjects are re-derived.

On-disk sync (sync_playlists_directory switch):
- Each playlist materializes into the configurable Playlists Directory
  (playlists_destination, Settings UI input, excluded from indexing) with
  position-prefixed entries: NNNN_<file> hard links (copy fallback across
  filesystems) and NNNN_<title>.html stubs for zim/url items.
- Tagged playlists are namespaced like channels: <playlists>/<tag>/<name>;
  a tag change moves the directory and prunes the empty old tag dir.
- Deletion safety: managed files are only removed when they are our marked
  stubs or still have another hard link (st_nlink > 1) -- never a last copy.
  Stray loose files at the playlists root are removed; stub URLs are
  scheme-allowlisted (no javascript:/data:).

Frontend:
- /playlists list page (CollectionTable, name filter, New Playlist modal, Tag
  column) between Files and Zim in the nav.
- View-only detail page (Name link) and edit page (Edit button): themed table
  with model icons/colors + poster thumbnails, items link to their model page
  (video/archive/doc) or preview, up/down reorder, remove, add-URL form,
  single-tag selector; all editing disabled in WROL Mode.
- Reusable green "Add to Playlist" button on Video/Archive/Doc detail pages,
  the FilePreview toolbar, Zim search results, and Map pins (added as a /map
  URL item with a map-marker icon).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Playlist edit page parity: rename, custom directory, tag modal move suggestion

Match the ChannelEditPage/DomainEditPage UX on the playlist edit page:
Back/View buttons, an edit form (name, description, directory), and the
shared CollectionTagModal with the violet "Tag" button.  The tag modal now
suggests moving the playlist to its managed tag location
(playlists/<tag>/<name>); accepting the suggestion stores directory=None so
the sync keeps auto-managing the location.  Tag buttons on the Channel and
Domain edit pages are now violet to match.

Backend: update_collection learns rename (with per-kind validation and
duplicate checks) and playlist custom directories (cleanup of the abandoned
directory, config round-trip, commit-before-switch to avoid the background
sync racing pre-commit state).  tag_collection's duplicated untag/tag
branches are merged, and the per-kind config-save dispatch is consolidated
into _activate_config_save().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Deduplicate Collection error handling and Tag creation

Delete the duplicate UnknownCollection in wrolpi/collections/errors.py and
re-export the canonical one from wrolpi/errors.py, which carries
status_code=NOT_FOUND (the local copy inherited 500).  Replace the repeated
query/raise pattern in reorganize.py with Collection.find_by_id.

Add Tag.get_or_create_tag and use it for collection tagging; flush the new
Tag before returning so callers can read .id immediately -- previously
tagging a collection with a brand-new tag name set tag_id=None.  Regression
test included.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 19:46:34 -06:00
Roland
ad0d93714a
Adding videos page hotkeys to help play and navigate videos. Adding f hotkey for all local search fields (channel/domain/etc). Focusing when opening cookie modals. 2026-02-23 17:29:09 -07:00
Roland
74cb5171da
FileGroups in Collections can be reorganized in the Collection's directory.
* Batch reorganization is supported for both Domain and Channel Collections.
* Batch reorganization happens when any Collection does not match the respective file format. For example, a Channel Collection file format that does not match the `yt_dlp_options.file_name_format` will show `needs_reorganization`.
* Individual Collections can always be reorganized. The reorganization is a little more aggressive. Useful for when some files may have been missed in the batch reorganization.
2026-01-31 18:08:41 -07:00
Roland
a92104728d Converting Theme.js to typescript so we can use shorthands like Card.Icon.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2025-12-07 12:41:10 -07:00
Roland
8ea3ea8a67
Updating code to use new version of react-router. 2025-12-07 11:54:20 -07:00
Roland
8a7f0737f0 Reformatting code. 2025-12-06 15:11:37 -07:00
Roland
47044cdd1f Ensuring domain config saves when tagging/untagging a domain. 2025-12-06 15:11:37 -07:00
Roland
b339c21515 Moving form fields. 2025-12-06 15:11:37 -07:00
Roland
63a77b87e6 Removing collection metadata. 2025-12-06 15:11:37 -07:00
Roland
a1e0fb4e92 Creating a new Collection model for use in organizing FileGroups.
Replacing Channels and Domains with Collection.

Moving downloading relationships from Channel to Collection.

Created domains.yaml config to manage tagged domain collections.

Adding turn-key upgrade solution in the UI.
2025-12-06 15:11:37 -07:00
Roland
581d5eaf6e
Adding toggle to Channel page to stop downloading comments for certain channels. 2025-03-01 18:23:29 -07:00
Roland
11bdbc4dc7
Creating map db cluster in pi-gen build. Removing duplicate pgpass creation. 2025-01-04 10:51:14 -07:00
Roland
7d6b7114a7
Adding TRACE level for logging beyond debug. Reworking async_client for tests, not all tests need the App to handle signals, so any test that needs signals can use the new await_switches fixture which primes the app. 2024-12-11 11:52:33 -07:00
Roland
5621d47309
Video downloads have many more options. Downloads are migrated.
Storing video settings in session storage to help user start downloads consistently.

Applying channel tag name to new channels when download videos or channels.

Initializing map tiles when importing and resetting map.
2024-11-28 09:48:57 -07:00
Roland
b643728b96
Using localStorage to store navbar color and calculator values. 2024-10-16 18:57:02 -06:00
Roland
11dc370ed9
Channel can be created with Tag. 2024-10-16 18:57:01 -06:00
Roland
c05078c513
Fixing Channel page directory selection. 2024-10-16 18:56:58 -06:00
Roland
4b6fb1d37f
Updating DirectorySearch to allow creation of new directories. 2024-10-16 18:56:51 -06:00
Roland
317f3f3dfb
Top processes are displayed on Status page. Nice of renderd is lowered to avoid clobbering system at startup. 2024-10-16 18:56:27 -06:00
Roland
0f232fd533
Moving FileGroups instead of refreshing files after they have been moved. 2024-10-16 18:56:25 -06:00
Roland
6fe8a7b704
Refusing to rename Tag or move Channel when files are refreshing. 2024-10-16 18:56:24 -06:00
Roland
2a83517d93
Using APIButton while tagging channel is processing. 2024-10-16 18:56:22 -06:00
Roland
45167e4f32
A Channel can be tagged with a single Tag. The Channel can be moved into a Tag Directory according to the config. 2024-10-16 18:56:19 -06:00
Roland
53d2402d32
Only files that have been tagged will be returned when new 'Any' tag button is used to filter file search results. 2024-10-16 18:55:56 -06:00
Roland
63738bd70f
Removing ChannelDownload in favor of Download.channel_id. 2024-10-16 18:55:54 -06:00
Roland
37ff1ab2cd
Changing title_match to title_include for clarity. 2024-10-16 18:55:51 -06:00
Roland
287beabb9c
Creating missing ChannelDownload when importing channels config. 2024-10-16 18:55:49 -06:00
Roland
50d95dad19
Channels now have Channel.channel_downloads. A Channel can have many Downloads. 2024-10-16 18:55:46 -06:00
Roland
213c4c0746
Showing top 80% of Channels. 2024-05-18 18:36:20 -06:00
Roland
bbfe76890b
Converting Search Suggetions to a Context. Search estimates is removed and suggestions is used instead. 2023-11-22 12:19:49 -07:00
Roland
00ab6401a0
Adding minimum width to ellipsis text for mobile screens. Adding button to clear active tags filter. 2023-11-04 15:57:57 -06:00
Roland
8ff34f5822
Hiding channels with low video count on Channels page. All channels can be viewed using Show More/Less button. 2023-11-02 16:10:30 -06:00
Roland
472846038b
Auto-submitting global search input, but debouncing with 1 second. 2023-11-02 15:24:07 -06:00
Roland
5b7aec59cc
Adding name query param to Channel filter input so the search can survive a refresh. 2023-11-02 10:45:35 -06:00
Roland
97b214d387
Adding Total Duration to EditChannel page. Handling when searching channel names, but channels have not been fetched yet. 2023-11-02 10:28:00 -06:00
Roland
b2f02d473e
Hiding Size column when editing Tags in mobile. 2023-09-13 10:39:06 -06:00
Roland
12850bdb8b
Using placeholder in SortableTable while data is loading. 2023-09-12 22:05:40 -06:00
Roland
619a88c429
Reworking default/empty/error states for all pages. Null is pending, undefined is error. 2023-09-07 20:13:36 -06:00
Roland
da6d6a8094
Allowing starting a download of a channel without frequency. Doing this creates a once-download. 2023-08-30 14:01:58 -06:00
Roland
59c4ab2502
Replacing icon property when icon has been requested.
Creating <APIButton/> which handles success/failure/pending during API calls.
2023-07-21 16:52:15 -06:00
Roland
6458316d1a
Removing unused /videos/tag. Sending events when Channels are created or deleted. 2023-07-17 17:32:29 -06:00
Roland
fdaaa510c4
Splitting errors into their modules. Removing API_ERRORS dict in favor of storing error data on the class. 2023-07-17 14:21:27 -06:00
Roland
7c5caf1dfd
Summarizing Channel size on Channels page, rather than info age. 2023-05-10 22:59:16 -06:00
Roland
2cbc6e00e0
Handling situation where React Router may not be available for a Tag Label. 2023-05-03 12:46:57 -06:00
Roland
77635b7926
Using Link for channel edit button and Tag label. 2023-05-03 08:28:32 -06:00
Roland
4d5eba59f7
Changing primary button colors to violet. 2023-04-05 10:18:52 -06:00
Roland
1812a1690f
Matching return style in all js. 2023-02-17 16:09:24 -07:00
Roland
81d80238be
Fixing changes before merge.
Moving FilesRefreshButton and DirectoryRefreshButton out of browser file.

Moving file fetching tino FileBrowser.

Removing sub-folders from Files display when any parent folder is clicked.

Using closed captioning for vtt/srt file icon.

Using my own react-semantic-toasts-2 to use toasts in Node 18.

Fixing bug where more than the last page was displayed.

Upgrading yt-dlp.

Fixing sub-folder file selection.

Adding rowKey to SortableTable to give React a key for each row.

Renaming key to path.

Removing duplicate urls.

Unifying file modals.

Displaying modals for image and text files.

Handling refresh loading in FileBrowser.  Creating refreshing_directory flag.

Removing accidental commit.

Sorting by url_count.

Using SortableTable on Domains page.

Using SortableTable for Channels page.

Created new FileBrowser which allows browsing of files in the media directory.

Using Node 18 in pi-gen.

Creating new FileBrowser to replace outdated react-keyed-file-browser.

Returning mimetype with file list.

Do not display 'Do not download' on Channel download in Dashboard.

Allowing null channel download frequency so user can stop downloading a channel.

Bumping version for Node 18.

Using my own semantic-ui-offline to avoid pulling Google fonts.

Upgrading app docker container to use Node 18.  This requires the removal of several npm packages.  File browser will require re-write.  It now pulls fonts from Google, this will need to be fixed.

Testing DownloadsToggle click.

Testing trinary states of DownloadsToggle.

Installing @testing-library/user-event for testing App.
2023-02-08 21:18:51 -07:00
Roland
84802eaf66
Updating dependencies for App testing. 2023-01-07 18:53:22 -07:00
Roland
6a5a73af22
Upgrading Reactjs to version 18. Removing timezone select in favor of using browser's timezone. Switching to @artsy/fresnel for Responsive content. 2023-01-07 18:31:39 -07:00