wrolpi/alembic/versions
lrnselfreliance 20f65e1063
Add ordered collections (playlists): polymorphic items + item API (#475)
* Add Playlists: ordered, mixed-type collections

A playlist (Collection kind='playlist') is a manually-curated, ordered list that
mixes item types. Squash of the playlists feature branch (24 commits).

Data model & API:
- CollectionItem is polymorphic via item_kind: 'file' (FileGroup), 'zim'
  (zim_id + zim_entry), or 'url' (link to anything the WROLPi browser opens,
  e.g. /map?lat=&lon=&z=). CHECK constraint enforces exactly one kind; per-kind
  unique constraints; migration a7b1c2d3e4f5.
- Endpoints: create playlist; add/remove/reorder items (idempotent adds: 201 new,
  200 existing); GET returns the ordered items. All mutations wrol_mode_check'd.

Config (source of truth):
- playlists.yaml (PlaylistsConfig) stores each playlist's metadata AND ordered
  items (file by relative path, zim by path+entry, url inline); round-trips a
  DB wipe. Supports backup import (DB_CONFIG_FILE_NAMES).
- The redundant generic CollectionsConfig (collections.yaml) is deleted:
  channels/domains own their configs, authors/subjects are re-derived.

On-disk sync (sync_playlists_directory switch):
- Each playlist materializes into the configurable Playlists Directory
  (playlists_destination, Settings UI input, excluded from indexing) with
  position-prefixed entries: NNNN_<file> hard links (copy fallback across
  filesystems) and NNNN_<title>.html stubs for zim/url items.
- Tagged playlists are namespaced like channels: <playlists>/<tag>/<name>;
  a tag change moves the directory and prunes the empty old tag dir.
- Deletion safety: managed files are only removed when they are our marked
  stubs or still have another hard link (st_nlink > 1) -- never a last copy.
  Stray loose files at the playlists root are removed; stub URLs are
  scheme-allowlisted (no javascript:/data:).

Frontend:
- /playlists list page (CollectionTable, name filter, New Playlist modal, Tag
  column) between Files and Zim in the nav.
- View-only detail page (Name link) and edit page (Edit button): themed table
  with model icons/colors + poster thumbnails, items link to their model page
  (video/archive/doc) or preview, up/down reorder, remove, add-URL form,
  single-tag selector; all editing disabled in WROL Mode.
- Reusable green "Add to Playlist" button on Video/Archive/Doc detail pages,
  the FilePreview toolbar, Zim search results, and Map pins (added as a /map
  URL item with a map-marker icon).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Playlist edit page parity: rename, custom directory, tag modal move suggestion

Match the ChannelEditPage/DomainEditPage UX on the playlist edit page:
Back/View buttons, an edit form (name, description, directory), and the
shared CollectionTagModal with the violet "Tag" button.  The tag modal now
suggests moving the playlist to its managed tag location
(playlists/<tag>/<name>); accepting the suggestion stores directory=None so
the sync keeps auto-managing the location.  Tag buttons on the Channel and
Domain edit pages are now violet to match.

Backend: update_collection learns rename (with per-kind validation and
duplicate checks) and playlist custom directories (cleanup of the abandoned
directory, config round-trip, commit-before-switch to avoid the background
sync racing pre-commit state).  tag_collection's duplicated untag/tag
branches are merged, and the per-kind config-save dispatch is consolidated
into _activate_config_save().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Deduplicate Collection error handling and Tag creation

Delete the duplicate UnknownCollection in wrolpi/collections/errors.py and
re-export the canonical one from wrolpi/errors.py, which carries
status_code=NOT_FOUND (the local copy inherited 500).  Replace the repeated
query/raise pattern in reorganize.py with Collection.find_by_id.

Add Tag.get_or_create_tag and use it for collection tagging; flush the new
Tag before returning so callers can read .id immediately -- previously
tagging a collection with a brand-new tag name set tag_id=None.  Regression
test included.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 19:37:34 -06:00
..
00f11f309c53_filegroup_censored.py Setting FileGroup.censored when refreshing channel files. 2024-06-17 14:48:41 -06:00
1a70196c05bb_download_location.py DownloadResult.try_again is not necessary. 2022-05-10 21:20:36 -06:00
1c041aeee0fc_doc_indexer.py User can select from/to years when searching files. Indexing doc and docx files. 2024-03-07 12:12:48 -07:00
1f2325523525_download_destination_migrate.py Video downloads have many more options. Downloads are migrated. 2024-11-28 09:48:57 -07:00
2bfb3b50b178_tag_names_migration.py Configs can be saved in background using switches. 2024-10-16 18:56:38 -06:00
2fb39c1861c7_create_mapfile_table.py Squashed commit of the following: 2022-03-08 13:26:03 -07:00
3b6918aeca4b_file_group.py Zim/Kiwix (Wikipedia) feature 2023-07-10 21:30:40 -06:00
3c33b1dae069_video_validated.py Feature/video validate (#74) 2022-01-14 15:54:51 -07:00
4b54ad052e2e_reindex_archives.py Adding FileGroup.published_date/author/etc. Creating script which converts a Singlefile to an Archive FileGroup files. 2023-10-23 18:09:08 -06:00
4c8da5502c8a_drop_channel_skip_download_videos.py Combining install scripts into a new repair script. Removing create_db script in favor of repair script. 2023-10-03 12:15:29 -06:00
4f03b9548f6e_file_group_effective_date_trigger.py Adding db trigger for file_group.effective_datetime. 2025-08-17 11:19:25 -06:00
5ad9efbf3570_filegroup_length.py Converting VideoPlayer to use FileGroup.url. Fixing downgrade to use FileGroup.url. 2023-10-29 21:13:38 -06:00
6b0b249a4f81_archive_published_datetime.py Moving data from models to FileGroup. 2023-10-29 17:31:54 -06:00
8bdb48a486b9_video_ffprobe_json.py Creating Video.ffprobe_json column which stores ffprobe information about the video file. Creating find_corrupt_videos.py script which automatically re-downloads videos which do not have both video and audio streams. 2023-08-03 21:40:25 -06:00
8d0d81bc9c34_channel_channel_downloads.py Fixing alembic migration that imported non-existent code. 2026-05-29 11:15:32 -06:00
8df411232c17_video_indexes.py Creating indexes on video columns to speedup video search. 2021-12-24 14:53:47 -07:00
9ec4c765ef8d_remove_inventories_version.py Configs are imported manually in main. Configs now have dump/save/import methods. Configs will refuse to save if they have not been imported. 2024-10-16 18:56:44 -06:00
26c5a62ec683_make_filegroup_directory_not_null.py Creating FileGroup.directory, converting paths to relative to avoid updating all the paths on move. 2025-12-24 11:26:45 -07:00
32d96eb9122d_video_censored.py Creating indexes on video columns to speedup video search. 2021-12-24 14:53:47 -07:00
36cae041ad31_files.py Refresh files removed files that were deleted. 2022-02-02 08:50:03 -07:00
053f17f07c4e_init_inventory_tables.py A new start 2021-11-01 16:43:09 -06:00
53f6780aa599_drop_urls.py Fixing alembic revisions to not use external models. 2025-12-06 15:11:37 -07:00
54c19aa847da_channel_summary_indexes.py Fixing channel summary trigger. 2025-08-22 11:03:33 -06:00
62acddc55091_archive_textsearch_coalesce_fix.py Applying textsearch coalesce fix to archive.textsearch. 2022-01-04 11:14:15 -07:00
69e9754d1a63_filegroup_effective_datetime.py Adding file_group.effective_datetime to speed up video and archive browsing. 2025-08-16 11:38:22 -06:00
70db5ba8763d_files_rework.py Creating video_channel_id_idx index. 2022-09-23 12:31:11 -06:00
75d34a2b442d_archive_datetime_index.py Only joining FileGroup table when searching Archives when necessary. 2023-05-16 14:56:53 -06:00
76add63ed225_json_to_jsonb.py Converting JSON columns to JSONB for performance. 2026-01-06 06:23:32 -07:00
84b6993a6091_drop_channel_next_download.py Zim/Kiwix (Wikipedia) feature 2023-07-10 21:30:40 -06:00
90ac16f9140a_remove_file_directory_and_stem.py Removing unused File columns and indexes. 2022-12-16 10:00:46 -07:00
170d1be52bc7_download_destination.py Video downloads have many more options. Downloads are migrated. 2024-11-28 09:48:57 -07:00
365c985893b8_ebook_creator.py Moving data from models to FileGroup. 2023-10-29 17:31:54 -06:00
577e414f1bc4_add_archive_collection_id_index.py Batch processing in get_collections to speed up collection pages. 2025-12-07 17:42:04 -07:00
717f36a4da94_ebook.py Storing example ebooks for testing. 2022-12-11 12:36:53 -07:00
751aecf46b88_zim.py Zim/Kiwix (Wikipedia) feature 2023-07-10 21:30:40 -06:00
930b9b7adb79_wrolpi_flags.py Creating Event and Flag systems. Events are used to send and display a toast to the App. Flags are used to signal if some action has been performed, or if a package is installed. 2022-12-23 16:04:09 -07:00
1870bec7c81e_initialize_archive_tables.py A new start 2021-11-01 16:43:09 -06:00
6489cd50c889_file_stem_and_directory.py Bumping version for new feature. 2022-12-07 18:15:28 -07:00
13542a230c78_archive_url_index.py Creating index for archive.url. 2022-11-02 15:21:34 -06:00
31896cc29b71_archive_contents_search.py Archives can be searched by their contents. 2021-12-31 20:14:14 -07:00
66407d145b76_.py Creating a new Collection model for use in organizing FileGroups. 2025-12-06 15:11:37 -07:00
158973fda20f_lower_filegroup_primary_path.py Adding file_group.effective_datetime to speed up video and archive browsing. 2025-08-16 11:38:22 -06:00
184329ed5137_downloader.py Arbitrary videos can be downloaded. (#63) 2021-12-20 16:09:11 -07:00
0400005fe2dc_video_textsearch_coalesce_fix.py Applying textsearch coalesce fix to archive.textsearch. 2022-01-04 11:14:15 -07:00
467235201c70_download_sub_downloader.py RSSDownloader can schedule downloads for other Downloaders using sub_downloader. 2022-06-08 11:56:42 -06:00
a1b2c3d4e5f6_add_collection_file_format.py FileGroups in Collections can be reorganized in the Collection's directory. 2026-01-31 18:08:41 -07:00
a2b3c4d5e6f7_drop_map_file_table.py Key changes: 2026-04-04 13:48:19 -06:00
a7b1c2d3e4f5_collection_item_polymorphic.py Add ordered collections (playlists): polymorphic items + item API (#475) 2026-06-09 19:37:34 -06:00
aac864072193_increase_file_size_column.py Zim/Kiwix (Wikipedia) feature 2023-07-10 21:30:40 -06:00
add_unique_collection_name_kind.py Creating a new Collection model for use in organizing FileGroups. 2025-12-06 15:11:37 -07:00
af4527503bdf_add_filegroup_directory_column.py Creating FileGroup.directory, converting paths to relative to avoid updating all the paths on move. 2025-12-24 11:26:45 -07:00
b2b9700f5048_fix_corrupted_media_paths.py FileGroups in Collections can be reorganized in the Collection's directory. 2026-01-31 18:08:41 -07:00
b3c4d5e6f7a8_create_map_subscription_table.py Key changes: 2026-04-04 13:48:19 -06:00
b43f70f369d0_remove_duplicate_channel_fields_.py Creating a new Collection model for use in organizing FileGroups. 2025-12-06 15:11:37 -07:00
b97c1313f986_video_modification_datetime.py Zim/Kiwix (Wikipedia) feature 2023-07-10 21:30:40 -06:00
ba98bd360b7a_add_collection_id_to_channel_model.py Removing model dependency in migration. 2026-01-31 19:29:02 -07:00
bddb5e95a91e_video_textsearch_index.py Changing weight to lowest. 2021-12-31 22:36:16 -07:00
bf704baa185d_zim_flag.py Zim/Kiwix (Wikipedia) feature 2023-07-10 21:30:40 -06:00
c4d5e6f7a8b9_drop_map_subscription_table.py Key changes: 2026-04-04 13:48:19 -06:00
c72d305cf11e_download_settings.py Adding Download.settings column in alembic migration. 2022-10-14 13:08:39 -06:00
c9561bda3cf0_file_group_indexes.py Adding default to TagFile.created_at. 2023-04-10 08:41:35 -06:00
d0bbe45e56c3_reindex_pdfs.py Storing example ebooks for testing. 2022-12-11 12:36:53 -07:00
d2eace02f5b9_file_path_on_cascade.py Zim/Kiwix (Wikipedia) feature 2023-07-10 21:30:40 -06:00
d4e446637f18_channel_tag.py Making Channel.tag_id migration lazier to avoid rare errors. 2024-10-16 18:57:04 -06:00
d5e6f7a8b9c0_doc_section.py Adding simple search to epub viewer. 2026-04-13 17:23:20 -06:00
d8fcdb7773c0_drop_channel_link.py Moving non-video functions to wrolpi.common. Handling Channel/Downlaod relationship in Channel.update. 2022-04-29 19:09:25 -06:00
d96a04eb2785_video_have_comments.py Creating background process that fetches comments for Videos that do not yet have comments. 2024-03-01 17:15:56 -07:00
d371333159ab_video_view_count_bigint.py Changing video view_count to bigint for popular videos. 2026-02-08 12:33:37 -07:00
db0a1560d923_downloader_info.py Squashed commit of the following: 2021-12-30 20:45:11 -07:00
dbf05c8e8dcc_invalidate_videos_for_orphaned_video_.py Invalidating video files for orphaned video meta files. 2022-12-07 18:27:45 -07:00
dc6637ced17b_zim_auto_search.py Adding feature which adds toggles to Zim files to enable/disable auto search. 2025-02-02 12:43:04 -07:00
e6f7a8b9c0d1_reindex_epubs_for_spine_order.py Adding simple search to epub viewer. 2026-04-13 17:23:20 -06:00
e7f8a9b0c1d2_rename_ebook_to_doc.py Add Docs module for searching and organizing PDFs and EPUBs 2026-03-20 14:17:06 -06:00
f0d0086a73d1_init_videos_tables.py A new start 2021-11-01 16:43:09 -06:00
f2c6326cad29_video_source_id_index.py Zim/Kiwix (Wikipedia) feature 2023-07-10 21:30:40 -06:00
f8c65e7e6802_channel_refresh.py Zim/Kiwix (Wikipedia) feature 2023-07-10 21:30:40 -06:00
f38cfe4b5cb2_collections_feature.py Running alembic check in cicd. 2025-12-06 15:11:37 -07:00
f7229a67e333_channel_download_missing_data.py Adding toggle to Channel page to stop downloading comments for certain channels. 2025-03-01 18:23:29 -07:00
f9928a3b6bc5_file_directories.py Zim/Kiwix (Wikipedia) feature 2023-07-10 21:30:40 -06:00
migrate_domains_to_collections.py Creating domain collection for archives that have no domain. 2026-02-01 16:19:40 -07:00
migrate_download_channel_to_collection.py Removing collection metadata. 2025-12-06 15:11:37 -07:00