Commit graph

91 commits

Author SHA1 Message Date
Roland
4058f22428
Warning when validation fails on video upsert. 2022-04-29 19:30:53 -06:00
Roland
8318224db6
Removing troubleshooting print. 2022-04-29 19:19:12 -06:00
Roland
d75ddf097d
Removing unused function. 2022-04-29 19:09:49 -06:00
Roland
aa203cc102
Moving non-video functions to wrolpi.common. Handling Channel/Downlaod relationship in Channel.update.
Removing default download freqeuncy so channels will not be downloaded without a url and frequency.

Adding channel unique url constraint.

Handling when a Channel's directory may still be relative in DB.

Adding TODOs to remove old favorites.

Removing outdated favorites from config after favorites have been created.

Testing that old-style favorited videos in the config are imported.

Removing channels_by_link.

Using list to store channels in config.

Moving channel import function into lib.

Supporting old channel import with links.

Bumping version for channel.link removal.

Removing all reference to Channel.link in app and api.

Adding relative property to MediaPathType.

Testing that a channel can be downloaded after it has a frequency added.

All tests pass.

Video and Channel tests pass without channel.link.

Removing channel.link usage in channel api.

Changing video Statistics page to About.
2022-04-29 19:09:25 -06:00
Roland
372056eefd
Removing troubleshooting print. 2022-04-08 09:34:18 -06:00
Roland
7656890835
Testing that video size is gathered during validation. 2022-04-07 13:42:01 -06:00
lrnselfreliance
35f5662571
Bugfix/channel download on import (#116)
* A Download record is created for every Channel imported.

* Downloads must have a URL.

* Handling when channel has download frequency but no url.

* Fetching downloads after clearing.

* Setting next download when importing channels.

* Not deleting recurring completed downloads.

* Fixing comments.

* Skipping some service tests in CircleCI because they shouldn't be tested off a local machine.

Signed-off-by: Roland <roland@learningselfreliance.com>
2022-04-07 11:18:15 -06:00
Roland
43709ac9e7
Creatind download manager config which will store skipped urls. 2022-04-02 16:01:43 -06:00
Roland
b1d526fb0b
Adding __init__ to map/test. 2022-03-09 20:10:58 -07:00
Roland
aabfe6a695
Downloading bestvideo+bestaudio as last resort. 2022-03-02 15:48:06 -07:00
Roland
5342d573c2
Using yt-dlp's match-filter to skip live videos. 2022-02-20 10:45:53 -07:00
Roland
f2af5e048b
Configs are initialized on import so Manager is started for all processes. 2022-02-16 23:05:43 -07:00
Roland
08065155cf
Squashing config rework. 2022-02-16 21:57:27 -07:00
StruckCroissant
9a1fe7323a Added channel whitespace test 2022-02-13 00:06:42 -06:00
StruckCroissant
d04a9c05d0
Merge branch 'lrnselfreliance:master' into master 2022-02-10 20:29:39 -06:00
Roland
59bd675afd
Adding video format 18 for 360p format. 2022-02-08 10:00:32 -07:00
StruckCroissant
8bfa2cdba9
Update api.py 2022-02-07 13:47:17 -06:00
StruckCroissant
d26ddd2ee0 Stripping whitespace from posted channel names 2022-02-07 12:04:38 -06:00
Roland
f2e4fcfa9c
Importing modules so its easier to see the relation between an api and its dependencies. 2022-02-03 12:39:14 -07:00
Roland
b4c1565fbb
Using new openapi.definition to show validation in docs. 2022-02-03 11:36:26 -07:00
Roland
8d3f65425a
Creating method that returns all Video files. Using this new method to delete the video files. 2022-02-02 16:30:58 -07:00
Roland
f655924c82
Fixing bug with DirectoriesRequest. 2022-02-02 16:18:08 -07:00
Roland
cd2ce1d691
Testing bad captions file parsing. 2022-02-01 09:56:59 -07:00
Roland
c79e28ff02
Upgrading base packages. Fixing failing test from merge. 2022-01-31 21:45:46 -07:00
Roland
c277a75a1a
Video tests pass with updated Sanic. 2022-01-31 21:30:07 -07:00
Roland
b1ed453466
Upgrading Sanic. Updating tests and schemas. 2022-01-31 21:30:07 -07:00
Roland
dc888c6392
Squashed commit of the following:
commit 54d06bf8045fceaa2684348d110792e6833b170c
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Jan 29 16:46:14 2022 -0700

    Can't delete files when WROL Mode is enabled.

commit 2d91f1983aae47bdb3f6dc7fc2bd2ae642c1126a
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Jan 29 16:37:42 2022 -0700

    Fixing bug where files could not be deleted in series.

commit eed818c50682b19de9f18da2d4daedc54408cc49
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Jan 29 16:26:31 2022 -0700

    Fixing fake_now to reset now() when complete.

commit 183e98cbec25784720ebaeabfac8af4a3fc87da8
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Jan 29 16:08:14 2022 -0700

    Files can be deleted and downloaded.

commit c5bcd25a3da041f42b9e11eae804aa359ab9e985
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Jan 29 14:41:07 2022 -0700

    Files can be deleted or downloaded.

commit 690b332bb4f89aa4e97ce2590b35bd4a71fefd87
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Jan 29 14:11:05 2022 -0700

    File is opened in new tab when file is selected.

commit ba8ead6f3185d0a398c4f89cecb5958f5e7731a3
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Jan 29 13:32:32 2022 -0700

    Removing files sort because React FE sorts them itself.

commit 4e04753234b11d615c5992bc08e9dd9fb7226c05
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Jan 29 13:29:52 2022 -0700

    Duplicate directories are ignored.

commit c2f7d7929f37655e5c2d11767a512571263b35ef
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Jan 29 12:22:38 2022 -0700

    Multiple directories can be requested.

commit 31678139c0799d0034d564cc1f8ae448234df90c
Author: Roland <roland@learningselfreliance.com>
Date:   Fri Jan 28 17:21:33 2022 -0700

    Files are displayed with the correct modified time.  Files can be recursively viewed.

commit a2d10a1dd792e518c8c6ad8f699b5d416c3d07e4
Author: Roland <roland@learningselfreliance.com>
Date:   Thu Jan 27 22:37:19 2022 -0700

    Files can be viewed.  Folders display their children.
2022-01-29 16:50:22 -07:00
Roland
83b9e3fc8f
Adding more video name parse tests. Using limiting match. 2022-01-27 21:09:22 -07:00
Roland
30b235a04e
Using greedy regex match on source_id. 2022-01-27 21:06:30 -07:00
Roland
8b0196ede3
Using new methods in download_channel. 2022-01-27 20:47:52 -07:00
Roland
8e47d2a9a0
Removing spread_channel_downloads function in favor of DownloadManager managing next_download. 2022-01-27 20:36:34 -07:00
Roland
b0465cb7f2
Restoring pytest fixture so PyCharm can find fixtures. 2022-01-26 17:15:35 -07:00
Roland
93204a0b2a
Displaying captions on video page. 2022-01-25 10:16:29 -07:00
Roland
0315def174
Videos can be sorted by their modification date. 2022-01-25 08:51:55 -07:00
Roland
f305d7912f
channel may be None. 2022-01-16 13:37:44 -07:00
Roland
1d9fe2f649
Setting channel id rather than channel because the channel may be in a different session. 2022-01-16 10:27:24 -07:00
Roland
89e3edb157
Creating functions to unify finding/making video directories. 2022-01-16 10:17:06 -07:00
Roland
a70e049823
Creating channel directory if it does not exist. 2022-01-16 09:28:22 -07:00
Roland
ca96bce6b7
Finding yt-dlp bin on startup. 2022-01-15 20:07:38 -07:00
Roland
e0b5966717
Creating videos and archives directory when they don't exist. 2022-01-15 19:58:38 -07:00
Roland
474e68cc3d
Fixing a few bugs in preparation for sneak peek. 2022-01-15 18:05:51 -07:00
Roland
fa0ccedd8b
Fetching next_download for each channel. 2022-01-15 13:52:43 -07:00
Roland
4f770bd215
Not counting empty videos in Channel statistics. 2022-01-15 13:31:53 -07:00
Roland
4a2fb577da
Creating channel directory if requested. 2022-01-15 13:23:57 -07:00
Roland
7d23a914e6
Info_json may not yet be a MediaPath. 2022-01-14 16:13:17 -07:00
Roland
f9a1f736ca
Not adding video when it was already in the session. 2022-01-14 16:11:14 -07:00
Roland
8b9d44df5c
Using str to check if video file is relative to channel directory. 2022-01-14 16:06:23 -07:00
lrnselfreliance
86893ef7f1
Feature/video validate (#74)
* Videos are marked as validated so they will not be tried again.  All previous bulk methods are combined into one.

* Passing json to fixture factory to avoid writing the files in every test.

* Adding log to explain if refresh is slow.

* Generating video poster if one cannot be found.

* Generating video poster if no poster can be found.

* Videos without a video file (yet to be downloaded) are not validated.

* Video title is extracted from the video file name as a last resort.

* Keeping existing view count because it may be from the channel, which will be more recent.

* Downloads should only be downloaded after they are due, not the same day.

* Setting manager on new downloads.

* Checking if downloading is disabled in loop.

* Adding log when refresh is complete.

* Not generating posters by default.  This is to avoid deleting files.

* Not converting posters unless channel has that enabled.

* Not logging on valid because all posters will be checked.

* Validating videos in upsert.

* Using video name parser in upsert.

* Using validate_video in upsert so that data is only read once from disk.

* Attaching manager in download_manager.get_download.

* Adding 22 to preferred video format for odd 720p resolution.

Signed-off-by: Roland <roland@learningselfreliance.com>
2022-01-14 15:54:51 -07:00
Roland
f21dfa4195
Deferring large video columns from select. 2022-01-11 08:56:27 -07:00
Roland
8f48e1b0d2
Commiting using channel session. 2022-01-07 14:46:03 -07:00