Commit graph

77 commits

Author SHA1 Message Date
Roland
15787104fd
videos: address review — tighten auto-caption detection
- Require `line === 'auto'` in the auto-caption signature so a cue that
  is deliberately placed (e.g. `line:0` for top-edge) is left alone even
  if it is also `align:start position:0%`. YouTube auto-captions never
  set `line`.
- For lazily-loaded language tracks, only detach the cuechange listener
  after the cue list is actually populated, in case a cuechange fires
  before cues load.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 18:29:18 -06:00
Roland
16c457ebcb videos: center YouTube auto-generated captions
YouTube auto-generated captions (downloaded via yt-dlp) stamp every cue
with `align:start position:0%`, which pins the caption text to the
bottom-left of the player and makes it hard to read. Properly-authored
VTT files have no cue settings and already render centered.

Detect that auto-caption signature and reset only those cues to the
browser's centered default (align:center / position:auto / line:auto).
This mutates the in-memory VTTCue objects the browser parsed; the .vtt
file on disk is never modified, and deliberately-positioned cues are
left untouched.

Wired via each <track>'s onLoad. Non-default language tracks are
`disabled` at load (no cues yet), so they are normalized lazily on the
first cuechange when the user selects them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 18:29:08 -06:00
lrnselfreliance
9a5d1f1630
Add ordered collections (playlists): polymorphic items + item API (#475)
* Add Playlists: ordered, mixed-type collections

A playlist (Collection kind='playlist') is a manually-curated, ordered list that
mixes item types. Squash of the playlists feature branch (24 commits).

Data model & API:
- CollectionItem is polymorphic via item_kind: 'file' (FileGroup), 'zim'
  (zim_id + zim_entry), or 'url' (link to anything the WROLPi browser opens,
  e.g. /map?lat=&lon=&z=). CHECK constraint enforces exactly one kind; per-kind
  unique constraints; migration a7b1c2d3e4f5.
- Endpoints: create playlist; add/remove/reorder items (idempotent adds: 201 new,
  200 existing); GET returns the ordered items. All mutations wrol_mode_check'd.

Config (source of truth):
- playlists.yaml (PlaylistsConfig) stores each playlist's metadata AND ordered
  items (file by relative path, zim by path+entry, url inline); round-trips a
  DB wipe. Supports backup import (DB_CONFIG_FILE_NAMES).
- The redundant generic CollectionsConfig (collections.yaml) is deleted:
  channels/domains own their configs, authors/subjects are re-derived.

On-disk sync (sync_playlists_directory switch):
- Each playlist materializes into the configurable Playlists Directory
  (playlists_destination, Settings UI input, excluded from indexing) with
  position-prefixed entries: NNNN_<file> hard links (copy fallback across
  filesystems) and NNNN_<title>.html stubs for zim/url items.
- Tagged playlists are namespaced like channels: <playlists>/<tag>/<name>;
  a tag change moves the directory and prunes the empty old tag dir.
- Deletion safety: managed files are only removed when they are our marked
  stubs or still have another hard link (st_nlink > 1) -- never a last copy.
  Stray loose files at the playlists root are removed; stub URLs are
  scheme-allowlisted (no javascript:/data:).

Frontend:
- /playlists list page (CollectionTable, name filter, New Playlist modal, Tag
  column) between Files and Zim in the nav.
- View-only detail page (Name link) and edit page (Edit button): themed table
  with model icons/colors + poster thumbnails, items link to their model page
  (video/archive/doc) or preview, up/down reorder, remove, add-URL form,
  single-tag selector; all editing disabled in WROL Mode.
- Reusable green "Add to Playlist" button on Video/Archive/Doc detail pages,
  the FilePreview toolbar, Zim search results, and Map pins (added as a /map
  URL item with a map-marker icon).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Playlist edit page parity: rename, custom directory, tag modal move suggestion

Match the ChannelEditPage/DomainEditPage UX on the playlist edit page:
Back/View buttons, an edit form (name, description, directory), and the
shared CollectionTagModal with the violet "Tag" button.  The tag modal now
suggests moving the playlist to its managed tag location
(playlists/<tag>/<name>); accepting the suggestion stores directory=None so
the sync keeps auto-managing the location.  Tag buttons on the Channel and
Domain edit pages are now violet to match.

Backend: update_collection learns rename (with per-kind validation and
duplicate checks) and playlist custom directories (cleanup of the abandoned
directory, config round-trip, commit-before-switch to avoid the background
sync racing pre-commit state).  tag_collection's duplicated untag/tag
branches are merged, and the per-kind config-save dispatch is consolidated
into _activate_config_save().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Deduplicate Collection error handling and Tag creation

Delete the duplicate UnknownCollection in wrolpi/collections/errors.py and
re-export the canonical one from wrolpi/errors.py, which carries
status_code=NOT_FOUND (the local copy inherited 500).  Replace the repeated
query/raise pattern in reorganize.py with Collection.find_by_id.

Add Tag.get_or_create_tag and use it for collection tagging; flush the new
Tag before returning so callers can read .id immediately -- previously
tagging a collection with a brand-new tag name set tag_id=None.  Regression
test included.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 19:46:34 -06:00
Roland
ba63170e55
Consolidating deletion endpoints.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-05-03 19:14:30 -06:00
Roland
09a967535b Models like Archive/Video/Doc can be deleted, even if they are tagged. User is shown a warning and must confirm deletion of tagged files. 2026-05-03 19:14:15 -06:00
Roland
9468448a54
Modifying Video/Channel/RSS Downloaders so that they can download videos, but audio only. These audio files are grouped under their channel directories. 2026-04-14 15:33:01 -06:00
Roland
bf5b787fb3
Unifying Files tab appearance for model pages.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
Controller Tests / controller-tests (push) Has been cancelled
Jest Tests / jest-tests (push) Has been cancelled
2026-03-28 14:19:09 -06:00
Roland
81f1456fed
Adding fullscreen keyboard shortcut. 2026-03-27 17:59:36 -06:00
Roland
282b0f63e8
Adding links next to video captions to allow scrubbing of video by the captions, adding filter to make finding quick. 2026-03-21 16:50:27 -06:00
Roland
66a891c759
Refactoring videos and archives UI paths for consistent scheme. 2026-03-20 15:38:32 -06:00
Roland
ebcb7b0e5d
Fixing bug where refreshing a video would download a copy, instead of just the metadata.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Controller Tests / controller-tests (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2026-03-19 15:02:32 -06:00
Roland
19a0e7e1c5
Diplaying video poster on lock screen. 2026-03-08 09:22:50 -06:00
Roland
ad0d93714a
Adding videos page hotkeys to help play and navigate videos. Adding f hotkey for all local search fields (channel/domain/etc). Focusing when opening cookie modals. 2026-02-23 17:29:09 -07:00
Roland
6b939bb195
Displaying more informative error when video files are missing. 2026-02-08 11:02:07 -07:00
Roland
3cfba32c29
Linking to new ffprobe json file on VideoPage. 2026-02-08 10:29:08 -07:00
Roland
a92104728d Converting Theme.js to typescript so we can use shorthands like Card.Icon.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
Jest Tests / jest-tests (push) Waiting to run
2025-12-07 12:41:10 -07:00
Roland
8ea3ea8a67
Updating code to use new version of react-router. 2025-12-07 11:54:20 -07:00
Roland
8a7f0737f0 Reformatting code. 2025-12-06 15:11:37 -07:00
Roland
531ec3d65e
Creating new /api/archive/upload which accepts a file from the SingleFile browser extension and converts it to an Archive. 2025-01-30 18:41:32 -07:00
Roland
5621d47309
Video downloads have many more options. Downloads are migrated.
Storing video settings in session storage to help user start downloads consistently.

Applying channel tag name to new channels when download videos or channels.

Initializing map tiles when importing and resetting map.
2024-11-28 09:48:57 -07:00
Roland
c1cfe103cb
Video comments' children sorted by time. 2024-10-16 18:57:17 -06:00
Roland
79e88d1bc6
Hiding port for /media link when there is no port. Using 'ago' popup for VideoPage date. 2024-10-16 18:57:03 -06:00
Roland
4b6fb1d37f
Updating DirectorySearch to allow creation of new directories. 2024-10-16 18:56:51 -06:00
Roland
a3e9952c4d
Using human readable number for comment like count. 2024-10-16 18:56:32 -06:00
Roland
c5e86c19e8
Centering video player in VideoPage. 2024-10-16 18:56:14 -06:00
Roland
62c3fe6b3c
Compressing video about segment. Moving about segment above comments. 2024-06-17 14:48:43 -06:00
Roland
758ff8e810
Creating background process that fetches comments for Videos that do not yet have comments.
Displaying star for Uploader's comments, heart for favorited comments.

Adding wrolpi help files desktop shortcut to rpi image.

Using zim file names as FileGroup.title.

Removing accidental output commit.
2024-03-01 17:15:56 -07:00
Roland
f179c5cbd9
Converting comment timestamp to javascript timestamp. 2024-02-21 12:44:21 -07:00
Roland
32877c5ada
Disabling video refresh button during wrol mode, or when video does not have url. 2024-02-20 15:15:23 -07:00
Roland
08fa3af453
Moving video about pane to it's own segment to avoid tabs going out of mobile viewing area. 2024-02-20 15:13:18 -07:00
Roland
f865d9fb0f
Videos display comments. User can download comments using Refresh button. 2024-02-20 13:47:12 -07:00
Roland
73cb142ae1
Reverting to older singlefile container to keep archive screenshot ratio. Using FileGroup.url on ArchivePage. 2023-10-31 10:20:45 -06:00
Roland
0629c8b464
Converting VideoPlayer to use FileGroup.url. Fixing downgrade to use FileGroup.url. 2023-10-29 21:13:38 -06:00
Roland
15687e534a
Moving data from models to FileGroup.
* Ebook.creator -> FileGroup.author
* Video.viewed -> FileGroup.viewed
* Video.duration -> FileGroup.length
* Video.url -> FileGroup.url
* Archive.url -> FileGroup.url
2023-10-29 17:31:54 -06:00
Roland
369da8a9b7
Converting hashtags in video descriptions to search links. 2023-09-07 13:00:54 -06:00
Roland
17a1aa903c
Modifying Archive page to match Video page, specifically using tabs for the about/files. 2023-08-26 20:28:34 -06:00
Roland
9dbd43cfdd
Videos near a video file without and upload date are returned as the prev/next videos. 2023-08-17 19:32:59 -06:00
Roland
e8e5b5727f
Creating Video.validated which is used by find_corrupt_videos.py to search. 2023-08-08 13:42:55 -06:00
Roland
85d4925ecc
Using correct caption track for each language. 2023-08-03 22:36:27 -06:00
Roland
36083200e6
Creating Video.ffprobe_json column which stores ffprobe information about the video file. Creating find_corrupt_videos.py script which automatically re-downloads videos which do not have both video and audio streams. 2023-08-03 21:40:25 -06:00
Roland
59c4ab2502
Replacing icon property when icon has been requested.
Creating <APIButton/> which handles success/failure/pending during API calls.
2023-07-21 16:52:15 -06:00
Roland
03a2df217d
Zim/Kiwix (Wikipedia) feature
* A new zim module has been added, this module allows several things to interact with Zim files.
  * Allow searching of Zim files using libzim.
  * Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
  * Creation of a new zim docker container which allows viewing of the Zim file contents.

* This commit also adds move/rename/mkdir features in the File Browser.
  * File tags are preserved when moving/renaming.
2023-07-10 21:30:40 -06:00
Roland
2cbc6e00e0
Handling situation where React Router may not be available for a Tag Label. 2023-05-03 12:46:57 -06:00
Roland
7a855b57cc
Fetching tags globally every 30 seconds, rather than for each usage of Tags. 2023-04-05 14:48:13 -06:00
Roland
a893d44094
Modifying AddTagsButton with an internal state to track tags. Video captions are displayed again. 2023-04-02 16:30:01 -06:00
Roland
ae2b6daa33
FileBrowser files can be tagged. A Tag icon is displayed on each file row in the browser. 2023-04-01 11:52:41 -06:00
Roland
44ebc876fa
Renaming examples to avoid grouping of pdf with ebooks. Reworking URLs for media files to avoid escaping url safe characters. 2023-03-29 13:46:39 -06:00
Roland
5615ce542d
Making example label a tag.
Tags can be created and deleted in UI.

Adding tag_file using id to avoid session error.

Using UTC in test.

Matching tag_names style in test.

Tag icon is displayed on file rows.

Archives, Videos and Files can be filtered by their tags.

Tags can be added and removed from UI.

Tags are displayed in VideoPlayer.

The tags of a file are returned in it's JSON.

Video favorites are converted to Favorite tag on channel config import.

Updating tests to match frontend usage.

PDF creator and modification time are displayed.

Reworking file search to use Video.__json__ for complex video relations.

Archives can be viewed.

Removing ebook.title.

Reworking alembic migration to be an upgrade rather than a replace.

Skipping test on circleci.

Clearing out video caption index only if extract_subtitles if off.

Channel tests pass.

Some video tests pass.

Missing videos can be found.  A video can be downloaded.

Orphaned files can be found.

Videos can be tagged.

Videos can be searched.

Videos can be modeled.

Converting some tests to use FileGroup.primary_path.

Adding archive and domains to alembic migration.

Testing cascading Archive titles.

Archives now uses FileGroup.

archive.lib tests pass.

Archive is partially converted to FileGroup.

Calculating FileGroup.primary file on refresh, rather than on select.

wrolpi tests pass.

Using FilesType to convert between str/pathlib.Path for FileGroup.files.

Using list of files in FileGroup.files instead of dictionary.

Storing files in dedicated FileGroup.files to avoid overwriting them.

Adding comment explaining CASE.

Not overwriting FileGroup.data on refresh if no change has been detected.

Removing File model in favor of FileGroup.data.

Adding typing to Files methods.

Files are indexed using the primary mimetype of the FileGroup.

files test_lib tests pass.

Removing file_group_file in favor of simple one-to-many relationship.

PDF files are indexed during refresh.

Adding Tag and TagFile to tag files with specific tags.

Only iterating when full_stem_ids has contents.

Testing that FileGroups are related to their Files.

Files and file_groups are created and linked.

Bump webpack from 5.75.0 to 5.76.1 in /app (#160)

Bumps [webpack](https://github.com/webpack/webpack) from 5.75.0 to 5.76.1.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.75.0...v5.76.1)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-25 17:54:41 -06:00
Roland
b6226a0212
Using FilePreview in VideoPlayer. 2023-02-17 16:30:25 -07:00
Roland
1812a1690f
Matching return style in all js. 2023-02-17 16:09:24 -07:00