wrolpi/modules/videos/models.py

388 lines
14 KiB
Python
Raw Normal View History

import json
2022-01-14 16:13:17 -07:00
import pathlib
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
from datetime import datetime
from pathlib import Path
from typing import Optional, Generator
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
from sqlalchemy import Column, Integer, String, Boolean, JSON, Date, ARRAY, ForeignKey, Computed
from sqlalchemy.orm import relationship, Session, deferred
from sqlalchemy.orm.collections import InstrumentedList
from wrolpi.common import Base, tsvector, ModelHelper, logger, get_media_directory
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
from wrolpi.dates import now, TZDateTime
from wrolpi.db import get_db_curs
from wrolpi.downloader import Download, download_manager
2022-01-27 20:47:52 -07:00
from wrolpi.errors import UnknownVideo, UnknownFile, UnknownDirectory, InvalidDownload
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
from wrolpi.media_path import MediaPathType, MediaPath
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
logger = logger.getChild(__name__)
class Video(ModelHelper, Base):
__tablename__ = 'video'
id = Column(Integer, primary_key=True)
channel_id = Column(Integer, ForeignKey('channel.id'))
channel = relationship('Channel', primaryjoin='Video.channel_id==Channel.id', back_populates='videos')
idempotency = Column(String)
validated = Column(Boolean, default=False)
# File paths
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
caption_path = Column(MediaPathType)
description_path = Column(MediaPathType)
ext = Column(String)
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
info_json_path = Column(MediaPathType)
poster_path = Column(MediaPathType)
video_path = Column(MediaPathType)
caption = deferred(Column(String)) # slow to fetch
censored = Column(Boolean, default=False)
duration = Column(Integer)
favorite = Column(TZDateTime)
modification_datetime = Column(TZDateTime)
size = Column(Integer)
source_id = Column(String)
title = Column(String)
upload_date = Column(TZDateTime)
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
url = Column(String)
view_count = Column(Integer)
viewed = Column(TZDateTime)
textsearch = deferred(
Column(tsvector, Computed('''to_tsvector('english'::regconfig,
((COALESCE(title, ''::text) || ' '::text) ||
COALESCE(caption, ''::text)))''')))
def __repr__(self):
video_path = self.video_path
if self.video_path and isinstance(self.video_path, MediaPath):
video_path = self.video_path.path.relative_to(get_media_directory())
return f'<Video id={self.id} title={self.title} path={video_path} channel={self.channel_id} ' \
f'source_id={self.source_id}>'
def dict(self) -> dict:
d = super().dict()
if self.channel_id:
d['channel'] = self.channel.dict()
d['info_json'] = self.get_info_json()
return d
def get_minimize(self) -> dict:
"""
Get a dictionary representation of this Video suitable for sending out the API.
"""
from .common import minimize_video
return minimize_video(self.dict())
def _clear_paths(self):
self.caption_path = None
self.description_path = None
self.ext = None
self.info_json_path = None
self.poster_path = None
self.video_path = None
def my_paths(self) -> Generator[Path, None, None]:
"""Return all paths of this Video. Returns nothing if all paths are None.""" # noqa
if self.description_path: # noqa
yield self.description_path.path
if self.poster_path:
yield self.poster_path.path
if self.caption_path:
yield self.caption_path.path
if self.info_json_path:
yield self.info_json_path.path
if self.video_path:
yield self.video_path.path
def delete(self):
"""
Remove all files related to this video. Add it to it's Channel's skip list.
"""
for path in self.my_paths():
path.unlink(missing_ok=True)
self._clear_paths()
needs_save = bool(self.favorite)
self.add_to_skip_list()
self.favorite = None
session = Session.object_session(self)
session.commit()
if needs_save:
# Save the Channels config. This video was a favorite and that needs to be removed.
from modules.videos.lib import save_channels_config
save_channels_config(preserve_favorites=False)
def add_to_skip_list(self):
"""
Add this video to it's Channel's skip list.
"""
2022-01-25 10:16:29 -07:00
if self.channel and self.source_id:
self.channel.add_video_to_skip_list(self.source_id)
def set_favorite(self, favorite: bool) -> Optional[datetime]:
self.favorite = now() if favorite else None
return self.favorite
def set_viewed(self):
self.viewed = now()
def get_info_json(self) -> Optional[JSON]:
"""
If this Video has an info_json file, return it's contents. Otherwise, return None.
"""
try:
2022-01-14 16:13:17 -07:00
if isinstance(self.info_json_path, pathlib.Path):
with open(self.info_json_path, 'rb') as fh:
contents = json.load(fh)
return contents
elif self.info_json_path:
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
with open(self.info_json_path.path, 'rb') as fh:
contents = json.load(fh)
return contents
except UnknownFile:
pass
except UnknownDirectory:
pass
except Exception as e:
logger.warning(f'Unable to parse info json {self.info_json_path}', exc_info=e)
return None
def get_video_description(self) -> Optional[str]:
"""
Get the Video description from the file system.
"""
# First try to get description from info_json file.
info_json = self.get_info_json()
if info_json:
description = info_json.get('description')
if description:
return description
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
if self.description_path:
with open(self.description_path, 'rt') as fh:
contents = fh.read()
return contents
def get_surrounding_videos(self):
"""
Get the previous and next videos around this video. The videos must be in the same channel.
Example:
>>> vid1 = Video(id=1, upload_date=10)
>>> vid2 = Video(id=2, upload_date=20)
>>> vid3 = Video(id=3, upload_date=30)
>>> vid4 = Video(id=4)
>>> vid1.get_surrounding_videos()
(None, vid2)
>>> vid2.get_surrounding_videos()
(vid1, vid3)
>>> vid3.get_surrounding_videos()
(vid2, None)
Video 4 has no upload date, so we can't place it in order.
>>> vid4.get_surrounding_videos()
(None, None)
"""
session = Session.object_session(self)
if not self.upload_date:
# We can't place a video that has no upload date.
return None, None
with get_db_curs() as curs:
query = '''
WITH numbered_videos AS (
SELECT id,
ROW_NUMBER() OVER (ORDER BY upload_date ASC) AS row_number
FROM video
WHERE
channel_id = %(channel_id)s
AND upload_date IS NOT NULL
)
SELECT id
FROM numbered_videos
WHERE row_number IN (
SELECT row_number+i
FROM numbered_videos
CROSS JOIN (SELECT -1 AS i UNION ALL SELECT 0 UNION ALL SELECT 1) n
WHERE
id = %(video_id)s
)
'''
curs.execute(query, dict(channel_id=self.channel_id, video_id=self.id))
results = [i[0] for i in curs.fetchall()]
# Assign the returned ID's to their respective positions relative to the ID that matches the video_id.
previous_id = next_id = None
for index, id_ in enumerate(results):
if id_ == self.id:
if index > 0:
previous_id = results[index - 1]
if index + 1 < len(results):
next_id = results[index + 1]
break
# Fetch the videos by id, if they exist.
previous_video = session.query(Video).filter_by(id=previous_id).one() if previous_id else None
next_video = session.query(Video).filter_by(id=next_id).one() if next_id else None
return previous_video, next_video
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
def __json__(self):
from modules.videos.common import minimize_video_info_json
info_json = minimize_video_info_json(self.get_info_json()) if self.info_json_path else None
stem = self.video_path.path.stem if self.video_path else None
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
d = dict(
caption_path=self.caption_path,
channel=self.channel,
channel_id=self.channel_id,
modification_datetime=self.modification_datetime,
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
duration=self.duration,
favorite=self.favorite,
id=self.id,
info_json=info_json,
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
poster_path=self.poster_path,
size=self.size,
source_id=self.source_id,
stem=stem,
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
title=self.title,
upload_date=self.upload_date,
url=self.url,
video_path=self.video_path,
view_count=self.view_count,
viewed=self.viewed,
)
return d
class Channel(ModelHelper, Base):
__tablename__ = 'channel'
id = Column(Integer, primary_key=True)
name = Column(String)
link = Column(String, nullable=False)
idempotency = Column(String)
url = Column(String)
match_regex = Column(String)
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
directory: MediaPath = Column(MediaPathType)
skip_download_videos = Column(ARRAY(String))
generate_posters = Column(Boolean, default=False) # generating posters may delete files, and can be slow.
calculate_duration = Column(Boolean, default=True)
download_frequency = Column(Integer)
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
source_id = Column(String)
refreshed = Column(Boolean, default=False)
info_json = Column(JSON)
info_date = Column(Date)
videos: InstrumentedList = relationship('Video', primaryjoin='Channel.id==Video.channel_id')
def __repr__(self):
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
return f'<Channel id={self.id}, name={repr(self.name)}>'
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
def add_video_to_skip_list(self, source_id: str):
if not source_id:
raise UnknownVideo(f'Cannot skip video with empty source id: {source_id}')
skip_download_videos = {i for i in self.skip_download_videos or [] if i}
skip_download_videos.add(source_id)
self.skip_download_videos = skip_download_videos
def delete_with_videos(self):
"""
Delete all Video records (but not video files) related to this Channel. Then delete the Channel.
"""
session = Session.object_session(self)
# Delete the video records, but not the video files!
session.query(Video).filter_by(channel_id=self.id).delete()
if self.url and (download := self.get_download()):
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
session.delete(download)
session.delete(self)
def update(self, data: dict):
"""
Update the values of this Channel. Will also update the Channel's Download, if it has one.
"""
for key, value in data.items():
setattr(self, key, value)
if not self.url:
return
download = self.get_download()
if download:
session = Session.object_session(self)
download.frequency = self.download_frequency
download.next_download = download_manager.get_next_download(download, session)
def config_view(self) -> dict:
"""
Retrieve the data about this Channel that should be stored in a config file.
"""
config = dict(
calculate_duration=self.calculate_duration,
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
directory=str(self.directory.path),
download_frequency=self.download_frequency,
favorites={},
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
generate_posters=self.generate_posters,
match_regex=self.match_regex or '',
name=self.name,
skip_download_videos=self.skip_download_videos or [],
source_id=self.source_id,
url=self.url or '',
)
return config
def get_relative_path(self, path: Path, exists: bool = True):
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
path = self.directory / path
if exists and not path.exists():
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
raise FileNotFoundError(f'{path} does not exist!')
return path
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
def get_download(self) -> Optional[Download]:
"""
Get the Download row for this Channel. If there isn't a Download, return None.
"""
if not self.url:
2022-01-27 20:47:52 -07:00
raise InvalidDownload(f'Channel {self.name} does not have a URL to download!')
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
session = Session.object_session(self)
download = session.query(Download).filter_by(url=self.url).one_or_none()
return download
def __json__(self):
d = dict(
id=self.id,
name=self.name,
directory=self.directory,
url=self.url,
link=self.link,
)
return d
def get_statistics(self):
"""
Get statistics about this channel.
"""
with get_db_curs() as curs:
stmt = '''
SELECT
SUM(size),
MAX(size),
COUNT(id)
FROM video
WHERE channel_id = %(id)s AND video_path IS NOT NULL
'''
curs.execute(stmt, dict(id=self.id))
size, largest_video, video_count = curs.fetchone()
statistics = dict(
video_count=video_count,
size=size,
largest_video=largest_video,
)
return statistics