Roland
8a7f0737f0
Reformatting code.
2025-12-06 15:11:37 -07:00
Roland
d7a02b5735
Speeding up channels config import.
2025-12-06 15:11:37 -07:00
Roland
1d8738ace2
Running alembic check in cicd.
2025-12-06 15:11:37 -07:00
Roland
a1e0fb4e92
Creating a new Collection model for use in organizing FileGroups.
...
Replacing Channels and Domains with Collection.
Moving downloading relationships from Channel to Collection.
Created domains.yaml config to manage tagged domain collections.
Adding turn-key upgrade solution in the UI.
2025-12-06 15:11:37 -07:00
Roland
c635267eda
Fixing bug where Archive screenshots would not be tracked when a user uploads a screenshot.
CodeQL / Analyze (push) Has been cancelled
2025-10-12 17:19:54 -06:00
Roland
b14c153664
Setting default value for channel trigger columns.
2025-08-22 11:02:47 -06:00
Roland
25f9568154
Adding summary columns to Channel table. Adding triggers to keep columns updated.
2025-08-17 10:50:55 -06:00
Roland
581d5eaf6e
Adding toggle to Channel page to stop downloading comments for certain channels.
2025-03-01 18:23:29 -07:00
Roland
531ec3d65e
Creating new /api/archive/upload which accepts a file from the SingleFile browser extension and converts it to an Archive.
2025-01-30 18:41:32 -07:00
Roland
e49ffc8a6c
Creating Channel.info_json_path property to avoid redefining channel's info json path.
2024-12-24 14:41:51 -07:00
Roland
7d6b7114a7
Adding TRACE level for logging beyond debug. Reworking async_client for tests, not all tests need the App to handle signals, so any test that needs signals can use the new await_switches fixture which primes the app.
2024-12-11 11:52:33 -07:00
Roland
5621d47309
Video downloads have many more options. Downloads are migrated.
...
Storing video settings in session storage to help user start downloads consistently.
Applying channel tag name to new channels when download videos or channels.
Initializing map tiles when importing and resetting map.
2024-11-28 09:48:57 -07:00
Roland
0784f740bb
Reworking Downloader.process_runner to use real stdout/stderr files to avoid issue with pipes being truncated.
2024-10-16 18:57:07 -06:00
Roland
77ad52473a
Fixing upsert_file usage. Adding log details for error.
2024-10-16 18:56:43 -06:00
Roland
d54735c4bb
Configs can be saved in background using switches.
2024-10-16 18:56:38 -06:00
Roland
317f3f3dfb
Top processes are displayed on Status page. Nice of renderd is lowered to avoid clobbering system at startup.
2024-10-16 18:56:27 -06:00
Roland
0f232fd533
Moving FileGroups instead of refreshing files after they have been moved.
2024-10-16 18:56:25 -06:00
Roland
45167e4f32
A Channel can be tagged with a single Tag. The Channel can be moved into a Tag Directory according to the config.
2024-10-16 18:56:19 -06:00
Roland
905042b600
Files which do not have a model can still be uploaded, they are indexed instead.
2024-10-16 18:56:15 -06:00
Roland
a98aebfdda
A video that is uploaded is attached to its Channel. Saving Channels config with both download url and frequency in case downloads config is corrupted.
2024-10-16 18:56:11 -06:00
Roland
0e99266379
A directory containing files that have been tagged is maintained automatically by WROLPi.
2024-10-16 18:56:05 -06:00
Roland
c839acb270
Converting Tag methods to use tag id or name.
2024-10-16 18:56:01 -06:00
Roland
2ad363281f
Delaying map startup because it prevents the API from starting.
2024-10-16 18:56:00 -06:00
Roland
af8bd7ba34
Cleaning up some extra data from video info json.
2024-10-16 18:55:59 -06:00
Roland
82af9a0d3c
Fixing bug where download.downloader would be replaced for all Channel Downloads.
2024-10-16 18:55:58 -06:00
Roland
7933d9625c
Downloads without a frequency cannot be a Channel Download.
2024-10-16 18:55:57 -06:00
Roland
63738bd70f
Removing ChannelDownload in favor of Download.channel_id.
2024-10-16 18:55:54 -06:00
Roland
287beabb9c
Creating missing ChannelDownload when importing channels config.
2024-10-16 18:55:49 -06:00
Roland
50d95dad19
Channels now have Channel.channel_downloads. A Channel can have many Downloads.
2024-10-16 18:55:46 -06:00
Roland
0fd545d2e8
Setting FileGroup.censored when refreshing channel files.
2024-06-17 14:48:41 -06:00
Roland
0bee458b79
Sorting keys when saving json file.
2024-06-13 14:08:07 -06:00
Roland
0fbbd36297
Creating perpetual funciton which checks if the internet is accessible. This is used to prevent downloads from starting when no internet exists.
2024-05-31 23:54:00 -06:00
Roland
58de8c7c5b
Upgrading Sanic and yt-dlp. Major changes occurred because Sanic changed how multiprocessing works.
...
Python 3.11 is now required, old versions were causing unpredictability in tests. (Sanic does not yet
support 3.12)
Sanic has been upgraded to 23.6.0, which is the latest version that avoids this bug:
https://github.com/sanic-org/sanic/issues/2921
New strategy for multiprocessing is to create all multiprocessing tools in one process, then fork
to other processes. The previous strategy was to declare multiprocessing tools at the top of every
file, or wherever they were needed at import/creation. Now all multiprocessing tools are attached to the
app.shared_ctx. This means `api_app` is imported in many, many places.
This forced a change in how the DownloadManager works. Previously, it would continually run
download workers which would pull downloads from a multiprocessing.Queue. Now, a single worker checks
for new downloads and sends a Sanic signal.
Flags have been reworked to use the `api_app`. I removed the `which` flag functionality because the
`which` are called at import and needed their own multiprocessing.Event.
2024-04-30 10:55:25 -06:00
Roland
a665677ecc
Adding more lower-quality video formats to download when no other is available. Getting video ffprobe json when downloading.
2024-03-16 19:58:58 -06:00
Roland
758ff8e810
Creating background process that fetches comments for Videos that do not yet have comments.
...
Displaying star for Uploader's comments, heart for favorited comments.
Adding wrolpi help files desktop shortcut to rpi image.
Using zim file names as FileGroup.title.
Removing accidental output commit.
2024-03-01 17:15:56 -07:00
Roland
df849ffb42
Ordering video/archive results when no published_datetime exists. Adding Video.location property which returns the App location of the video.
2024-02-26 13:40:39 -07:00
Roland
0d6edc05d0
Duplicate videos are deleted when a video is downloaded.
2024-02-21 12:42:30 -07:00
Roland
f865d9fb0f
Videos display comments. User can download comments using Refresh button.
2024-02-20 13:47:12 -07:00
Roland
1b92d1e580
Global search uses Semantic SearchResultsInput component, other components use SearchInput.
2023-11-21 13:01:52 -07:00
Roland
97b214d387
Adding Total Duration to EditChannel page. Handling when searching channel names, but channels have not been fetched yet.
2023-11-02 10:28:00 -06:00
Roland
440b770a11
Fixing Video.get_surrounding_videos after moving video upload date.
2023-10-29 18:44:50 -06:00
Roland
15687e534a
Moving data from models to FileGroup.
...
* Ebook.creator -> FileGroup.author
* Video.viewed -> FileGroup.viewed
* Video.duration -> FileGroup.length
* Video.url -> FileGroup.url
* Archive.url -> FileGroup.url
2023-10-29 17:31:54 -06:00
Roland
76212cf3de
Created script which searches for videos that have similar names, interacts with the user to choose which to delete.
2023-10-09 08:44:30 -06:00
Roland
6e0f9a999c
Combining install scripts into a new repair script. Removing create_db script in favor of repair script.
2023-10-03 12:15:29 -06:00
Roland
e7582944b9
Channels not in the channels config are deleted on import.
2023-09-12 20:24:02 -06:00
Roland
3e39102a55
ffprobe json may not be valid, the video should still be retrieved.
2023-09-07 13:01:35 -06:00
Roland
da6d6a8094
Allowing starting a download of a channel without frequency. Doing this creates a once-download.
2023-08-30 14:01:58 -06:00
Roland
6959c7a3b4
HTML and JSON files are reformatted when downloaded.
2023-08-26 21:49:24 -06:00
Roland
9dbd43cfdd
Videos near a video file without and upload date are returned as the prev/next videos.
2023-08-17 19:32:59 -06:00
Roland
14017d43bb
Creating Video/Archive get_by_id/find_by_id methods. find_by_id will raise an exception when not found, get_by_id will not.
2023-08-17 18:46:42 -06:00