wrolpi/app/src/components/Download.js

1467 lines
48 KiB
JavaScript
Raw Normal View History

import React, {useState} from "react";
import {fetchVideoDownloaderConfig, getDownloaders, postDownload, putDownload} from "../api";
import {
APIButton,
DirectorySearch,
InfoHeader,
InfoPopup,
mergeDeep,
RequiredAsterisk,
useLocalStorage
} from "./Common";
import Icon from "semantic-ui-react/dist/commonjs/elements/Icon";
import Message from "semantic-ui-react/dist/commonjs/collections/Message";
import {Button, Form, FormInput, Header} from "./Theme";
import {Form as SForm, FormDropdown} from "semantic-ui-react";
import {Link} from "react-router";
import {TagsSelector} from "../Tags";
import Grid from "semantic-ui-react/dist/commonjs/collections/Grid";
import {
commaSeparatedValidator,
InputForm,
NumberInputForm,
ToggleForm,
UrlInput,
UrlsTextarea,
useForm
} from "../hooks/useForm";
import {
channelFrequencyOptions,
days30Option,
defaultVideoFormatOption,
defaultVideoResolutionOptions,
Downloaders,
downloadFormatOptions,
downloadOrderOptions,
downloadResolutionOptions,
extendedFrequencyOptions,
frequencyOptions,
weeklyOption
} from "./Vars";
export function DepthInputForm({form, required, name = 'depth', path = 'settings.depth'}) {
return <NumberInputForm
form={form}
required={required}
name={name}
path={path}
label='Depth'
max={4}
helpContent='Search the URLs provided, and any URLs they contain up to this depth. Warning: This can be exponential!'
/>
}
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
export function MaximumPagesInputForm({form, required, name = 'max_pages', path = 'settings.max_pages'}) {
return <NumberInputForm
form={form}
required={required}
name={name}
path={path}
label='Maximum Pages'
max={100000}
helpContent='Stop searching for files if this many pages have been searched.'
/>
}
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
export function DestinationForm({
form,
infoContent,
label = 'Destination',
name = 'destination',
path = 'destination',
required = false,
}) {
const [inputProps, inputAttrs] = form.getCustomProps({name, path, required});
const {disabled, value, onChange} = inputProps;
const infoPopup = infoContent ? <InfoPopup content={infoContent}/> : null;
return <SForm.Field>
<label>{label} {required && <RequiredAsterisk/>}{infoPopup}</label>
<DirectorySearch
required={required}
value={value}
onSelect={onChange}
disabled={disabled}
id='destination_search_form'
/>
</SForm.Field>
}
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
export function DownloadTagsSelector({form, limit, path = 'tag_names', name = 'tag_names'}) {
const [inputProps, inputAttrs] = form.getCustomProps({name, path});
const {value, onChange} = inputProps;
return <TagsSelector
disabled={form.disabled}
selectedTagNames={value}
limit={limit}
onChange={onChange}
closeAfterAdd={!!limit}
/>;
}
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
export function DownloadFrequencySelector({
form,
freqOptions = frequencyOptions,
name = 'frequency',
path = 'frequency',
}) {
const [inputProps, inputAttrs] = form.getSelectionProps({name: 'frequency', path});
return <FormDropdown
required
selection
label='Download Frequency'
placeholder='Frequency'
options={freqOptions}
name={name}
id='download_frequency_selector'
{...inputProps}
/>
}
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
export function DownloaderSelector({form, name = 'sub_downloader', path = 'sub_downloader'}) {
const [downloaders, setDownloaders] = React.useState([]);
const [inputProps, inputAttrs] = form.getSelectionProps({name, path, required: true});
const fetchDownloaders = async () => {
let {downloaders: downloaders_} = await getDownloaders();
downloaders_ = downloaders_.map((i) => {
return {key: i.name, text: i.pretty_name || i.name, value: i.name}
})
setDownloaders(downloaders_);
}
React.useEffect(() => {
fetchDownloaders();
}, []);
return <FormDropdown selection required
label='Downloader'
placeholder='Select a downloader'
options={downloaders}
id='downloader_selector'
{...inputProps}
/>
}
export function ExcludedUrls({form, name = 'excluded_urls', path = 'settings.excluded_urls'}) {
return <InputForm
form={form}
type='text'
name={name}
path={path}
helpContent="Comma-separated list of keywords that will be ignored if they are in any link's URL"
label='Excluded URLs'
placeholder='prize,gift'
validator={commaSeparatedValidator}
/>
}
export function TitleInclusionInput({form, path = 'settings.title_include'}) {
const [inputProps, inputAttrs] = form.getInputProps({
name: 'title_include',
path,
validator: commaSeparatedValidator,
});
return <>
<InfoHeader
headerSize='h4'
headerContent='Title Match Words'
popupContent='List of words, separated by commas, that titles must contain to be downloaded.'
/>
<FormInput
placeholder='Shelter,Solar Power'
error={inputProps.error}
>
<input {...inputProps}/>
</FormInput>
</>
}
export function TitleExclusionInput({form, path = 'settings.title_exclude'}) {
const [inputProps, inputAttrs] = form.getInputProps({
name: 'title_exclude',
path,
validator: commaSeparatedValidator,
});
return <>
<InfoHeader
headerSize='h4'
headerContent='Title Exclusion Words'
popupContent='List of words, separated by commas, that may not appear in titles to be downloaded.'
/>
<FormInput
placeholder='Giveaway,Prize'
error={inputProps.error}
>
<input {...inputProps}/>
</FormInput>
</>
}
export function DownloadFormButtons({onCancel, form}) {
return <Grid columns={2}>
<Grid.Row>
<Grid.Column textAlign='left'>
<Button content='Cancel' onClick={onCancel} type='button'/>
</Grid.Column>
<Grid.Column textAlign='right'>
<APIButton
disabled={form.disabled || !form.ready}
type='submit'
style={{marginTop: '0.5em'}}
onClick={form.onSubmit}
id='download_form_download_button'
>Download</APIButton>
</Grid.Column>
</Grid.Row>
</Grid>
}
export function EditDownloadFormButtons({onDelete, onCancel, form}) {
return <Grid columns={2}>
<Grid.Row>
<Grid.Column textAlign='left'>
<APIButton
confirmButton='Delete'
confirmContent='Delete this Download?'
confirmHeader='Delete'
color='red'
onClick={onDelete}
>Delete</APIButton>
</Grid.Column>
<Grid.Column textAlign='right'>
<Button content='Cancel' onClick={onCancel}/>
<APIButton
disabled={form.disabled || !form.ready}
type='submit'
style={{marginTop: '0.5em'}}
onClick={form.onSubmit}
>Save</APIButton>
</Grid.Column>
</Grid.Row>
</Grid>
}
function VideoDownloadOrder({form, path = 'settings.download_order'}) {
const [inputProps, inputAttrs] = form.getSelectionProps({
name: 'download_order',
path,
});
return <FormDropdown selection
label='Download Order'
options={downloadOrderOptions}
{...inputProps}
/>
}
function VideoDownloadCountLimit({form, name = 'video_count_limit', path = 'settings.video_count_limit'}) {
return <NumberInputForm
form={form}
helpContent='Stop downloading videos from this channel/playlist when this many have been downloaded.'
helpPosition='top right'
name={name}
path={path}
label='Video Count Limit'
placeholder='100'
/>
}
2025-12-06 14:49:46 -07:00
export function VideoResolutionSelectorForm({
form,
name = 'video_resolutions',
path = 'settings.video_resolutions',
onChange
}) {
const [inputProps, inputAttrs] = form.getSelectionProps({
name,
path,
type: 'array',
afterChange: onChange
});
return <>
<InfoHeader
headerSize='h5'
headerContent='Video Resolutions'
popupContent='Videos will be downloaded in the first available resolution from the list you select.'
for_='video_resolutions_input'
/>
<FormDropdown selection multiple
id='video_resolutions_input'
options={downloadResolutionOptions}
{...inputProps}
/>
</>
}
function VideoFormatSelectorForm({form, name = 'video_format', path = 'settings.video_format'}) {
const [inputProps, inputAttrs] = form.getSelectionProps({
name,
path,
defaultValue: defaultVideoFormatOption,
});
return <>
<InfoHeader
headerSize='h5'
headerContent='Video Format'
popupContent='Videos will be downloaded in this format, or transcoded if not available.'
for_='video_format_input'
/>
<FormDropdown selection
id='video_format_input'
options={downloadFormatOptions}
{...inputProps}
/>
</>
}
function VideoDurationLimit({form, name, path, label, helpContent, placeholder, helpPosition}) {
return <NumberInputForm
form={form}
helpContent={helpContent}
helpPosition={helpPosition}
placeholder={placeholder}
name={name}
path={path}
label={label}
/>
}
export function VideoTagsForm({form}) {
return <>
<InfoHeader
headerSize='h4'
headerContent='Videos Tags'
popupContent='Tag all Videos with these Tags.'
/>
<DownloadTagsSelector form={form}/>
</>
}
export function ChannelTagNameForm({form}) {
return <>
<InfoHeader
headerSize='h4'
headerContent='Channel Tag'
popupContent='If the Channel is new, apply this Tag.'
/>
<DownloadTagsSelector
form={form}
limit={1}
name='channel_tag_name'
path='settings.channel_tag_name'
/>
</>
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
}
export function UseBrowserProfile({form}) {
const [config, setConfig] = useState(null);
const localFetchConfig = async () => {
const result = await fetchVideoDownloaderConfig();
setConfig(result);
if (result?.always_use_browser_profile === true) {
// Enable the toggle if "always_use_browser_profile" is set.
form.setValue('settings.use_browser_profile', true);
}
}
React.useEffect(() => {
localFetchConfig();
}, []);
const popupContent = <>Use the browser profile to download videos.
This is useful for downloading videos that require a login. See: <i>Videos > Settings</i></>;
const label = <InfoHeader
headerSize='h4'
headerContent='Use Browser Profile'
popupContent={popupContent}
/>;
return <ToggleForm
form={form}
label={label}
disabled={!config?.browser_profile}
name='use_browser_profile'
path='settings.use_browser_profile'
/>
}
2025-12-06 14:49:46 -07:00
export function VideosDownloadForm({
singleDownload = true,
onCancel,
onSuccess: propOnSuccess,
download,
submitter: propSubmitter,
actions
}) {
const [showMessage, setShowMessage] = React.useState(false);
const [userChangedResolutions, setUserChangedResolutions] = React.useState(false);
const [config, setConfig] = React.useState(null);
// Keep video format in session to help user start downloads consistently.
const [defaultVideoFormat, setDefaultVideoFormat] = useLocalStorage('video_format', defaultVideoFormatOption);
// Use config video resolutions if available, otherwise use default
const configResolutions = config && config.video_resolutions ? config.video_resolutions : defaultVideoResolutionOptions;
const defaultFormData = {
urls: '', // Textarea, one URL per line.
destination: '',
tag_names: [],
downloader: Downloaders.Video,
settings: {
use_browser_profile: false,
video_format: defaultVideoFormat,
video_resolutions: configResolutions,
}
}
const submitter = propSubmitter || (async (formData) => {
const downloadData = {
destination: formData.destination,
downloader: formData.downloader,
settings: formData.settings,
tag_names: formData.tag_names,
urls: formData.urls.split(/r?\n/),
}
await postDownload(downloadData);
});
const onSuccess = () => {
setShowMessage(true);
if (propOnSuccess) {
propOnSuccess();
} else {
form.reset();
}
}
const form = useForm({
submitter,
defaultFormData: download ? mergeDeep(defaultFormData, download) : defaultFormData,
onSuccess,
});
// Fetch video downloader config when the modal opens
React.useEffect(() => {
const fetchConfig = async () => {
try {
const result = await fetchVideoDownloaderConfig();
setConfig(result);
// Update form with video resolutions from config only if user hasn't changed them
// and if no download object was provided (this is a new download, not an edit)
if (result && result.video_resolutions && !userChangedResolutions && !download) {
form.setValue('settings.video_resolutions', result.video_resolutions);
}
} catch (error) {
console.error('Failed to fetch video downloader config:', error);
}
};
fetchConfig();
}, []);
React.useEffect(() => {
const {video_format} = form.formData.settings;
if (video_format && video_format !== defaultVideoFormat) {
setDefaultVideoFormat(video_format);
}
}, [form.formData]);
const localOnCancel = (e) => {
if (e) e.preventDefault();
if (onCancel) {
onCancel();
}
}
// This form can handle a single Video download, or multiple video downloads.
const urlInput = singleDownload ?
<UrlInput required form={form} path='urls'/>
: <UrlsTextarea required form={form}/>;
return <Form>
<Header as='h3'>
<Icon name='film' color='blue'/>
Videos
</Header>
<p>Download each video at the URLs provided below.</p>
<Grid stackable columns={1}>
<Grid.Row>
<Grid.Column>{urlInput}</Grid.Column>
</Grid.Row>
<Grid.Row>
<Grid.Row columns={1}>
<Grid.Column>
<VideoTagsForm form={form}/>
</Grid.Column>
</Grid.Row>
</Grid.Row>
<Grid.Row>
<Grid.Column>
<DestinationForm
form={form}
helpContent="Videos download into their Channel's directory, by default. If this is provided, then videos in this Channel/Playlist will download to this directory instead."
/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={2}>
<Grid.Column width={11}>
<VideoResolutionSelectorForm
form={form}
onChange={() => setUserChangedResolutions(true)}
/>
</Grid.Column>
<Grid.Column width={4}>
<VideoFormatSelectorForm form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row>
<Grid.Column>
<UseBrowserProfile form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={1}>
<Grid.Column>
<ChannelTagNameForm form={form}/>
</Grid.Column>
</Grid.Row>
{showMessage && <SuccessfulDownloadSubmitMessage/>}
<Grid.Row>
<Grid.Column>
2025-12-06 14:49:46 -07:00
{actions ? actions({onCancel: localOnCancel, form}) :
<DownloadFormButtons onCancel={localOnCancel} form={form}/>}
</Grid.Column>
</Grid.Row>
</Grid>
</Form>
}
export function VideoMinimumDurationForm({form}) {
return <VideoDurationLimit
form={form}
path='settings.minimum_duration'
label='Minimum Duration'
name='minimum_duration'
helpContent='Download only Videos this many seconds long, or greater.'
placeholder='60'
/>
}
export function EditVideosDownloadForm({
2025-12-06 14:49:46 -07:00
download,
onCancel,
onSuccess,
onDelete,
actions = EditDownloadFormButtons,
}) {
const submitter = async (formData) => {
// Create a copy of settings without channel_url and channel_id
const settings = {...formData.settings};
// These settings are created when downloading the video, we can't submit them again.
delete settings.channel_url;
delete settings.channel_id;
const downloadData = {
destination: formData.destination,
downloader: formData.downloader,
settings: settings,
tag_names: formData.tag_names,
urls: formData.urls.split(/r?\n/),
}
await putDownload(download.id, downloadData);
}
return <VideosDownloadForm
singleDownload={true}
onCancel={onCancel}
onSuccess={onSuccess}
download={download}
submitter={submitter}
actions={(props) => actions({...props, onDelete})}
/>
}
export function VideoMaximumDurationForm({form}) {
return <VideoDurationLimit
form={form}
path='settings.maximum_duration'
label='Maximum Duration'
name='maximum_duration'
helpContent='Do not download videos longer than this many seconds.'
placeholder='3600'
/>
}
export function ChannelDownloadForm({
download,
onCancel,
onSuccess,
onDelete = async () => {
},
submitter,
actions = null,
clearOnSuccess = true,
channel_id = null,
}) {
const [showMessage, setShowMessage] = React.useState(false);
const [config, setConfig] = React.useState(null);
const [isConfigLoaded, setIsConfigLoaded] = React.useState(false);
const [userChangedResolutions, setUserChangedResolutions] = React.useState(false);
// May have received submitter from EditChannelDownloadForm.
submitter = submitter || (async (formData) => {
const downloadData = {
destination: formData.destination,
downloader: formData.downloader,
frequency: formData.frequency,
settings: formData.settings,
sub_downloader: formData.sub_downloader,
tag_names: formData.tag_names || [],
urls: [formData.url,],
}
await postDownload(downloadData);
});
const localOnSuccess = async () => {
setShowMessage(true);
if (onSuccess) {
onSuccess();
}
}
// Keep video format in session to help user start downloads consistently.
const [defaultVideoFormat, setDefaultVideoFormat] = useLocalStorage('video_format', defaultVideoFormatOption);
// Use config video resolutions if available, otherwise use default
const configResolutions = config && config.video_resolutions ? config.video_resolutions : defaultVideoResolutionOptions;
const emptyFormData = {
destination: '',
downloader: Downloaders.VideoChannel,
frequency: days30Option.value,
url: '',
settings: {
channel_id,
channel_tag_name: [],
download_order: 'newest',
maximum_duration: null,
minimum_duration: null,
title_exclude: null,
title_include: null,
use_browser_profile: false,
video_count_limit: null,
video_format: defaultVideoFormat,
video_resolutions: configResolutions,
},
sub_downloader: Downloaders.Video,
tag_names: [],
};
const form = useForm({
submitter,
defaultFormData: mergeDeep(emptyFormData, download),
emptyFormData,
onSuccess: localOnSuccess,
clearOnSuccess,
});
// Fetch video downloader config when the modal opens
React.useEffect(() => {
const fetchConfig = async () => {
try {
const result = await fetchVideoDownloaderConfig();
setConfig(result);
setIsConfigLoaded(true);
// Update form with video resolutions from config only if user hasn't changed them
// and if no download object was provided (this is a new download, not an edit)
if (result && result.video_resolutions && !userChangedResolutions && !download) {
form.setValue('settings.video_resolutions', result.video_resolutions);
}
} catch (error) {
console.error('Failed to fetch video downloader config:', error);
setIsConfigLoaded(true);
}
};
fetchConfig();
}, []);
React.useEffect(() => {
const {video_format} = form.formData.settings;
if (video_format && video_format !== defaultVideoFormat) {
setDefaultVideoFormat(video_format);
}
}, [form.formData]);
const onceMessage = <Message>
<Message.Header>Download Once</Message.Header>
<Message.Content>You have selected a frequency of Once, this is useful when you want to download
all videos in a Playlist, and when you do not want to download any videos added to the playlist
in the future.</Message.Content>
</Message>;
// Default to "new" download buttons.
actions = actions || DownloadFormButtons;
const actionsElm = actions({onDelete, onCancel, form});
return <Form>
<Header as='h3'><Icon name='film' color='blue'/> Channel / Playlist</Header>
<Grid stackable columns={1}>
<Grid.Row>
<Grid.Column>
<UrlInput required form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={1}>
<Grid.Column>
<VideoTagsForm form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={2}>
<Grid.Column mobile={4} tablet={4}>
<DownloadFrequencySelector form={form} freqOptions={channelFrequencyOptions}/>
</Grid.Column>
<Grid.Column mobile={4} tablet={12}>
<DestinationForm
form={form}
helpContent='Destination is not required. Videos will download into the automatically created Channel directory.'
/>
</Grid.Column>
</Grid.Row>
{form.formData.frequency === 0 && <Grid.Row><Grid.Column>{onceMessage}</Grid.Column></Grid.Row>}
<Grid.Row columns={2}>
<Grid.Column>
<TitleInclusionInput form={form}/>
</Grid.Column>
<Grid.Column>
<TitleExclusionInput form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={2}>
<Grid.Column mobile={4} tablet={5}>
<VideoDownloadOrder form={form}/>
</Grid.Column>
<Grid.Column mobile={4} tablet={5}>
<VideoDownloadCountLimit form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={2}>
<Grid.Column width={11}>
<VideoResolutionSelectorForm
form={form}
onChange={() => setUserChangedResolutions(true)}
/>
</Grid.Column>
<Grid.Column width={4}>
<VideoFormatSelectorForm form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={2}>
<Grid.Column tablet={8} computer={4}>
<VideoMinimumDurationForm form={form}/>
</Grid.Column>
<Grid.Column tablet={8} computer={4}>
<VideoMaximumDurationForm form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={1}>
<Grid.Column>
<ChannelTagNameForm form={form}/>
</Grid.Column>
</Grid.Row>
{showMessage && <SuccessfulDownloadSubmitMessage/>}
<Grid.Row>
<Grid.Column textAlign='right'>
{actionsElm}
</Grid.Column>
</Grid.Row>
</Grid>
</Form>
}
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
export function EditChannelDownloadForm({
download,
onCancel,
onSuccess,
onDelete,
withTags = true,
actions = EditDownloadFormButtons,
}) {
const submitter = async (formData) => {
const downloadData = {
destination: formData.destination,
downloader: formData.downloader,
frequency: formData.frequency,
settings: formData.settings,
tag_names: formData.tag_names || [],
urls: [formData.url,],
}
await putDownload(download.id, downloadData);
}
return <ChannelDownloadForm
download={download}
submitter={submitter}
onCancel={onCancel}
onSuccess={onSuccess}
onDelete={onDelete}
withTags={withTags}
actions={actions}
clearOnSuccess={false}
/>
}
function SuccessfulDownloadSubmitMessage() {
return <Grid.Row>
<Grid.Column><Message positive>
<Message.Header>Download Submitted</Message.Header>
<Message.Content>
<Link to='/admin'><Icon name='checkmark'/> View downloads</Link>
</Message.Content>
</Message>
</Grid.Column>
</Grid.Row>
}
export function EditArchiveDownloadForm({
2025-12-06 14:49:46 -07:00
download,
onCancel,
onSuccess,
onDelete,
actions = EditDownloadFormButtons,
}) {
const submitter = async (formData) => {
const downloadData = {
downloader: formData.downloader,
tag_names: formData.tag_names,
urls: formData.urls.split(/\r?\n/),
}
await putDownload(download.id, downloadData);
}
return <ArchiveDownloadForm
download={download}
onCancel={onCancel}
onSuccess={onSuccess}
submitter={submitter}
actions={(props) => actions({...props, onDelete})}
/>
}
export function ArchiveDownloadForm({download, onCancel, onSuccess: propOnSuccess, submitter: propSubmitter, actions}) {
const [showMessage, setShowMessage] = React.useState(false);
const submitter = propSubmitter || (async (formData) => {
const downloadData = {
downloader: formData.downloader,
tag_names: formData.tag_names,
urls: formData.urls.split(/\r?\n/),
}
await postDownload(downloadData);
});
const emptyFormData = {
downloader: Downloaders.Archive,
urls: '',
tag_names: [],
};
const onSuccess = () => {
setShowMessage(true);
if (propOnSuccess) {
propOnSuccess();
}
}
const form = useForm({
submitter,
defaultFormData: mergeDeep(emptyFormData, download),
emptyFormData,
clearOnSuccess: !propOnSuccess,
onSuccess,
});
const localOnCancel = (e) => {
if (e) e.preventDefault();
if (onCancel) {
onCancel();
}
}
return <Form>
<Header as='h3'><Icon name='file text' color='green'/> Archives</Header>
<p>Create a Singlefile Archive for each of the URLs provided below.</p>
<Grid stackable columns={1}>
<Grid.Row>
<Grid.Column>
<UrlsTextarea required form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row>
<Grid.Column>
<DownloadTagsSelector form={form}/>
</Grid.Column>
</Grid.Row>
{showMessage && <SuccessfulDownloadSubmitMessage/>}
<Grid.Row>
<Grid.Column textAlign='right'>
2025-12-06 14:49:46 -07:00
{actions ? actions({onCancel: localOnCancel, form}) :
<DownloadFormButtons onCancel={localOnCancel} form={form}/>}
</Grid.Column>
</Grid.Row>
</Grid>
</Form>
}
export function RSSDownloadForm({download, submitter, onDelete, onCancel, actions, clearOnSuccess = true}) {
const [showMessage, setShowMessage] = React.useState(false);
const [config, setConfig] = React.useState(null);
const [isConfigLoaded, setIsConfigLoaded] = React.useState(false);
const [userChangedResolutions, setUserChangedResolutions] = React.useState(false);
const [defaultVideoFormat, setDefaultVideoFormat] = useLocalStorage('video_format', defaultVideoFormatOption);
submitter = submitter || (async (formData) => {
const downloadData = {
destination: formData.destination,
downloader: formData.downloader,
frequency: formData.frequency,
settings: formData.settings,
sub_downloader: formData.sub_downloader,
tag_names: formData.tag_names || [],
urls: [formData.url],
}
await postDownload(downloadData);
});
// Use config video resolutions if available, otherwise use default
const configResolutions = config && config.video_resolutions ? config.video_resolutions : defaultVideoResolutionOptions;
const emptyFormData = {
destination: null,
downloader: Downloaders.RSS,
frequency: weeklyOption.value,
sub_downloader: null,
settings: {
excluded_urls: null,
title_exclude: null,
title_include: null,
video_resolutions: configResolutions,
video_format: defaultVideoFormat,
},
tag_names: [],
url: '',
};
const form = useForm({
submitter,
defaultFormData: mergeDeep(emptyFormData, download),
emptyFormData,
clearOnSuccess,
onSuccess: async () => setShowMessage(true),
});
// Fetch video downloader config when the modal opens
React.useEffect(() => {
const fetchConfig = async () => {
try {
const result = await fetchVideoDownloaderConfig();
setConfig(result);
setIsConfigLoaded(true);
// Update form with video resolutions from config only if user hasn't changed them
// and if no download object was provided (this is a new download, not an edit)
if (result && result.video_resolutions && !userChangedResolutions && !download) {
form.setValue('settings.video_resolutions', result.video_resolutions);
}
} catch (error) {
console.error('Failed to fetch video downloader config:', error);
setIsConfigLoaded(true);
}
};
fetchConfig();
}, []);
React.useEffect(() => {
const {video_format} = form.formData.settings;
if (video_format && video_format !== defaultVideoFormat) {
setDefaultVideoFormat(video_format);
}
}, [form.formData]);
// Default to "new" download buttons.
actions = actions || DownloadFormButtons;
const actionsElm = actions({onDelete, onCancel, form});
let downloaderRows;
if (form.formData.sub_downloader === Downloaders.Video) {
downloaderRows = <>
<Grid.Row columns={1}>
<Grid.Column>
<DestinationForm
form={form}
helpContent="Videos download into their Channel's directory, by default. If this is provided, then videos in this feed will download to this directory instead."
/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={2}>
<Grid.Column mobile={16} computer={8}>
<TitleInclusionInput form={form}/>
</Grid.Column>
<Grid.Column mobile={16} computer={8}>
<TitleExclusionInput form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={2}>
<Grid.Column mobile={16} computer={11}>
<VideoResolutionSelectorForm
form={form}
onChange={() => setUserChangedResolutions(true)}
/>
</Grid.Column>
<Grid.Column mobile={8} computer={3}>
<VideoFormatSelectorForm form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={2}>
<Grid.Column tablet={8} computer={4}>
<VideoMinimumDurationForm form={form}/>
</Grid.Column>
<Grid.Column tablet={8} computer={4}>
<VideoMaximumDurationForm form={form}/>
</Grid.Column>
</Grid.Row>
</>;
} else if (form.formData.sub_downloader === Downloaders.Archive) {
downloaderRows = <Grid.Row>
<Grid.Column>
<ExcludedUrls form={form}/>
</Grid.Column>
</Grid.Row>;
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
}
return <Form>
<Header as='h3'><Icon name='rss' color='orange'/> RSS Feed</Header>
<p>Download each link provided by this RSS feed using the selected downloader.</p>
<Grid columns={1}>
<Grid.Row>
<Grid.Column>
<UrlInput required form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row>
<Grid.Column>
<DownloadTagsSelector form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={2}>
<Grid.Column width={8}>
<DownloadFrequencySelector form={form} freqOptions={extendedFrequencyOptions}/>
</Grid.Column>
<Grid.Column width={8}>
<DownloaderSelector form={form}/>
</Grid.Column>
</Grid.Row>
{downloaderRows}
{showMessage && <SuccessfulDownloadSubmitMessage/>}
<Grid.Row>
<Grid.Column textAlign='right'>
{actionsElm}
</Grid.Column>
</Grid.Row>
</Grid>
</Form>
Arbitrary videos can be downloaded. (#63) * Squashed commit of the following: commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:16:36 2021 -0700 Only saving a favorite video to the config if it has a source_id. commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:14:26 2021 -0700 Ading description and changing Save button to Download. commit 03a72be6dfdfa383915950a80e3f1ee9febf6705 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:09:44 2021 -0700 Removing uploader for now. commit 5d0231284bf84e459687f6ab903b06bf4e3762bd Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:07:17 2021 -0700 Removing unique data from example info. commit 9cc721a32d4715d9d6433e51db3b35e039c78d68 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 22:02:33 2021 -0700 Updating downloader tests. commit 8e193b03349ae4613f105418cd3e8176c7101304 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:53:31 2021 -0700 Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode. commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:45:06 2021 -0700 Raising error if download exits without a 0. Clearing kill event when starting a new download. commit 54ba3a890d59facc837410d34bf7afb40db3b685 Author: Roland <roland@learningselfreliance.com> Date: Tue Dec 14 21:16:46 2021 -0700 Downloads can be restarted and killed. commit bd60751262247259c3e9ceb5116e017d6314c219 Author: Roland <roland@learningselfreliance.com> Date: Mon Dec 13 16:28:18 2021 -0700 Downloads can be started again. Downloads are sorted on Downloads page. commit ea7f39231b92982ae0b263b76c5bd1902cc8169d Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 21:24:16 2021 -0700 Building Downloads tab. commit d3321872ac3743ff6df781b07ef0fb537f1d4c98 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 20:41:46 2021 -0700 Not scheduling channels withtout a URL/frequency. commit 84f268a0d4f58c85f86b6377900492e0766f6619 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 19:56:08 2021 -0700 Channel downloads will be spread out when each channel is created. commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60 Author: Roland <roland@learningselfreliance.com> Date: Sat Dec 11 13:08:11 2021 -0700 Recurring downloads are renewed on startup. commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 17:03:57 2021 -0700 Fetching download again after do_download. commit fdadad327fc09c17a49eb63b11e632fb72cc3195 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 16:20:51 2021 -0700 Fixing error with test. Mock was not being reset. commit 070dee238e53377acd22097c63cf3794e68ed423 Author: Roland <roland@learningselfreliance.com> Date: Fri Dec 10 15:36:39 2021 -0700 Fixed bug in date handling (big surprise). commit e53a8992268d68620c679455adaec3a590f20602 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 16:48:36 2021 -0700 Using chunks for regular channel refresh. commit 3bd60c58d1bef0226325f321978c0434b36b3be5 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:22:02 2021 -0700 Reformatting. commit 4fdc325f25a32c311ca6543fe321624b2bd564b2 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 15:13:34 2021 -0700 Creating function that refreshes NO CHANNEL videos. commit d67feafb6c2994f5a4a1295920a70db369cc6816 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:35:54 2021 -0700 Removing unused functions. commit 6b3918e4e78fbac64920f85c4de0df873ec366a1 Author: Roland <roland@learningselfreliance.com> Date: Wed Dec 1 14:31:48 2021 -0700 Removing reporter and events. This will be replaced in the future with some sort of SSE. commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:55:43 2021 -0700 Better error handling for archive button. commit c31d124e511b27f626ed2df71486d47302b396f3 Author: Roland <roland@learningselfreliance.com> Date: Sun Nov 28 17:29:43 2021 -0700 Displaying View Count in a human readable format. commit 0ee2bd65588fd93b475cad6144e76f1b45191b39 Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 09:29:41 2021 -0700 Fixing tests. commit e64aff32bab95c311ea3747e1a21b58b99750d5e Author: Roland <roland@learningselfreliance.com> Date: Sat Nov 27 08:59:51 2021 -0700 Handling when a Video has no Channel in UI. commit b9549ecd5aac03c002adca7b7667f0a7189b662a Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 26 08:35:36 2021 -0700 Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried. commit da88798e2584d3e84900a70e34166c8c208c64fb Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 20:40:29 2021 -0700 Downloaders can refuse a download, which will then be deleted. commit b865d856121db5ec8a3660c5d3c13b9846fde688 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:35:43 2021 -0700 Removing relative path handling. commit 6d7757870b02ba030d78b3ef7e43f28b906eea61 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 19:28:27 2021 -0700 Using __json__ for Video and Channel models. commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 18:23:38 2021 -0700 Fixing video links in UI. commit 9d081af12742070b8519bd752812130cb5dbe393 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:42:38 2021 -0700 Fixing docs. commit 1202cbe0f9ae744d7801b53a5071cded8ea39924 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:40:36 2021 -0700 Fixing wrolpi tests. commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 24 14:13:35 2021 -0700 Fixing tests to use MediaPath. commit faec04f2a5e4c267a502c03884940a9c5bcfe387 Author: Roland <roland@learningselfreliance.com> Date: Fri Nov 19 10:21:20 2021 -0700 Testing when a video has no channel. commit cbf22aad7a0b72e61685e0236e843554b57c4af1 Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 16:24:03 2021 -0700 Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download. commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d Author: Roland <roland@learningselfreliance.com> Date: Thu Nov 18 12:43:44 2021 -0700 Channels/Videos downloaders can get gotten. commit 808ef796cec01db6e7009065f9e26d25d9500ce2 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 21:45:19 2021 -0700 Upgrading to Sanic 21.9.1. commit ea32813be28e78393ac456ee782a7245c098188f Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 20:29:01 2021 -0700 Creating Downloader and DownloaderManager classes which handle downloading URLs. commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66 Author: Roland <roland@learningselfreliance.com> Date: Wed Nov 17 12:41:39 2021 -0700 Creating UI segement which allows the download/upload of media. * Displaying all recurring downloads. * Not downloading channels with empty url. * Showing positive row for recurring downloads. * Removing default last_successful_download. * Only displaying Last Successful Download when there is one. * Refreshing videos before finding missing videos. * Using short video path name for log. * Removing download history feature in favor of new Downloads page. * Only showing info date if it exists. * Making get_or_create_channel more reliable. * Allowing null for channel download frequency. * Logging when searching for channel. * Storing channel's source_id in config. * Importing using link instead of name. * Getting channel id when importing channels. * Removing redunant parentheses from repr. * Preserving database channel values if there are not values in the config. * Only highlighting pending downloads. * Deleting old downloads periodically. * Sorting recurring by next_download/frequency. * Creating fixtures for channels, media directory and db session. * Removing unused function. Wrapping config for tests. * Deleting channel download when deleting a channel. * Testing that a channel's download is deleted when the channel is deleted. * Updating youtube-dl * Switching from youtube-dl in favor of yt-dlp. * Sorting once-downlaods by last_successful_download then id. * Adding special handling to video_url_resolver for full urls. * Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top. * Getting session if one is not provided. * Adding test for create_downloads. * Not passing format option when getting filename. * Linting Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:08:13 -07:00
}
export function EditRSSDownloadForm({download, onDelete, onCancel, actions = EditDownloadFormButtons}) {
const submitter = async (formData) => {
const downloadData = {
destination: formData.destination,
downloader: formData.downloader,
frequency: formData.frequency,
settings: formData.settings,
sub_downloader: formData.sub_downloader,
tag_names: formData.tag_names || [],
urls: [formData.url,],
}
await putDownload(download.id, downloadData);
}
return <RSSDownloadForm
submitter={submitter}
onDelete={onDelete}
onCancel={onCancel}
download={download}
actions={actions}
clearOnSuccess={false}
/>
}
export function EditZimDownloadForm({download, onDelete, onCancel, actions = EditDownloadFormButtons}) {
const submitter = async (formData) => {
const downloadData = {
downloader: formData.downloader,
frequency: formData.frequency,
settings: formData.settings,
sub_downloader: formData.sub_downloader,
tag_names: formData.tag_names,
urls: [formData.url,],
}
await putDownload(download.id, downloadData);
}
const [showMessage, setShowMessage] = React.useState(false);
const emptyFormData = {
downloader: Downloaders.RSS,
frequency: weeklyOption.value,
sub_downloader: null,
tag_names: [],
url: '',
};
const form = useForm({
submitter,
defaultFormData: mergeDeep(emptyFormData, download),
emptyFormData,
onSuccess: async () => setShowMessage(true),
});
// Default to "new" download buttons.
actions = actions || DownloadFormButtons;
const actionsElm = actions({onDelete, onCancel, form});
return <Form>
<Header as='h3'>Zim File</Header>
<Grid stackable columns={1}>
<Grid.Row>
<Grid.Column>
<UrlInput required form={form} disabled={true}/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={2}>
<Grid.Column width={8}>
<DownloadFrequencySelector form={form} longFrequenciesAvailable={true}/>
</Grid.Column>
</Grid.Row>
{showMessage && <SuccessfulDownloadSubmitMessage/>}
<Grid.Row>
<Grid.Column textAlign='right'>
{actionsElm}
</Grid.Column>
</Grid.Row>
</Grid>
</Form>
}
export function FilesDownloadForm({
download,
submitter,
onCancel,
actions = DownloadFormButtons,
clearOnSuccess = true,
}) {
const [showMessage, setShowMessage] = React.useState(false);
submitter = submitter || (async (formData) => {
const downloadData = {
downloader: Downloaders.File,
tag_names: formData.tag_names,
destination: formData.destination,
urls: formData.urls.split(/\r?\n/),
}
await postDownload(downloadData);
});
const emptyFormData = {
destination: null,
tag_names: [],
urls: '',
};
const form = useForm({
submitter,
defaultFormData: mergeDeep(emptyFormData, download),
emptyFormData,
clearOnSuccess,
onSuccess: async () => setShowMessage(true),
});
const actionsElm = actions({onCancel, form});
return <Form>
<Header as='h3'><Icon name='file'/> Files</Header>
<p>Download each file at the URLs provided below.</p>
<Grid>
<Grid.Row>
<Grid.Column>
<UrlsTextarea form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row>
<Grid.Column>
<DownloadTagsSelector form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row>
<Grid.Column>
<DestinationForm required form={form}/>
</Grid.Column>
</Grid.Row>
{showMessage && <SuccessfulDownloadSubmitMessage/>}
<Grid.Row>
<Grid.Column textAlign='right'>
{actionsElm}
</Grid.Column>
</Grid.Row>
</Grid>
</Form>
}
const suffixValidator = (value) => {
const error = commaSeparatedValidator(value);
if (error) {
return error;
}
const suffixes = value.split(',');
for (const suffix of suffixes) {
if (!suffix.startsWith('.')) {
return 'Suffix must start with .';
}
if (suffix.length === 1) {
return 'Suffix must have characters after .';
}
}
}
export function SuffixFormInput({form, name = 'suffix', path = 'settings.suffix'}) {
return <InputForm
form={form}
label='File Suffixes'
helpContent='Comma-separated list of file suffixes that should be downloaded'
required={true}
placeholder='.pdf,.mp4'
name={name}
path={path}
validator={suffixValidator}
/>
}
export function ScrapeFilesDownloadForm({
download,
submitter,
clearOnSuccess,
onDelete,
onCancel,
onSuccess,
actions = DownloadFormButtons,
singleDownload = false,
}) {
const [showMessage, setShowMessage] = React.useState(false);
submitter = submitter || (async (formData) => {
const downloadData = {
downloader: Downloaders.ScrapeHtml,
sub_downloader: Downloaders.File,
tag_names: formData.tag_names || [],
destination: formData.destination,
urls: formData.urls.split(/\r?\n/),
settings: formData.settings,
}
await postDownload(downloadData);
});
const emptyFormData = {
destination: null,
tag_names: [],
urls: '',
settings: {
depth: 1,
max_pages: 1,
suffix: '',
}
};
const localOnSuccess = async () => {
setShowMessage(true);
if (onSuccess) {
onSuccess();
}
}
const form = useForm({
submitter,
defaultFormData: mergeDeep(emptyFormData, download),
emptyFormData,
clearOnSuccess,
onSuccess: localOnSuccess,
});
const urlInput = singleDownload ?
<UrlInput required form={form} path='url'/>
: <UrlsTextarea required form={form}/>;
// Default to "new" download buttons.
actions = actions || DownloadFormButtons;
const actionsElm = actions({onDelete, onCancel, form});
return <Form>
<Header as='h3'><Icon name='file alternate' color='red'/> Scrape Files</Header>
<p>Search each of the URLs for files matching the suffix (.pdf, etc.).</p>
<Grid>
<Grid.Row>
<Grid.Column>
{urlInput}
</Grid.Column>
</Grid.Row>
<Grid.Row columns={2}>
<Grid.Column width={10}>
<DestinationForm required={true} form={form}/>
</Grid.Column>
<Grid.Column width={6}>
<SuffixFormInput required={true} form={form}/>
</Grid.Column>
</Grid.Row>
<Grid.Row columns={2}>
<Grid.Column width={4}>
<DepthInputForm form={form} required={true}/>
</Grid.Column>
<Grid.Column width={4}>
<MaximumPagesInputForm form={form} required={true}/>
</Grid.Column>
</Grid.Row>
{showMessage && <SuccessfulDownloadSubmitMessage/>}
<Grid.Row>
<Grid.Column>
{actionsElm}
</Grid.Column>
</Grid.Row>
</Grid>
</Form>
}
export function EditScrapeFilesDownloadForm({download, onDelete, onCancel, onSuccess}) {
const submitter = async (formData) => {
const downloadData = {
downloader: Downloaders.ScrapeHtml,
sub_downloader: Downloaders.File,
tag_names: formData.tag_names || [],
destination: formData.destination,
urls: [formData.url],
settings: formData.settings,
}
await postDownload(downloadData);
};
return <ScrapeFilesDownloadForm
download={download}
submitter={submitter}
onDelete={onDelete}
onCancel={onCancel}
onSuccess={onSuccess}
actions={EditDownloadFormButtons}
singleDownload={true}
/>
}
export function DownloadMenu({onOpen, disabled}) {
const [downloader, setDownloader] = useState();
Fixing JS warnings. Handling all combinations of toggle in light and dark. Changing CPUUsageProgress colors to be more readable in dark mode. Moving theme handling into Theme.js. Importing element wrappers from Theme.js to handle element theming. Changing video menu to dark mode. Creating ThemeContext to share Dark Mode setting. Removing duplicate code for search functions. Displaying X when downloads are disabled. Using react context to share settings across components. Refusing to start workers when WROL Mode is enabled. Changing warning to debug. Queuing recurring downloads first. Adding Video.url index. Adding special handling for youtube shorts to videos. Reworking Downloads to have one worker per process. Testing creating downloads and getting downloaders. Testing video validation and poster generation. Testing video statistics. Increasing inventory test coverage. Adding pytest.ini to automatically detect async test. Adding more archive test coverage. Testing Archive CRUD. Catching and logging when encountering error during indexing. Using FileIcon when Archive has no screenshot. A Video will be assigned to a Channel by its info json first, then to its directory. Including suffix in File json. Adding index for File.suffix. Gathering suffix during indexing. Passing entire file down to FileIcon. Moving logging up. Returning File.associated in file json. Associated files are last in file search. Removing duplicate refreshing code by creating new _refresh_directory function. Changing File.idempotency to a datetime to avoid having to clear idempotency. Truncating indexes to avoid error with large files. Don't overwrite video data when info json is empty. Using already_downloaded for channel videos. Awaiting refresh in Channel.refresh. Testing that videos already downloaded aren't downloaded again. Adding indexes for video/archive referenced paths. Changing grid width for mobile or computer. Adding button to refresh specific directories. Restoring Channel edit link. Fixing caption path in video_modeler. Adding ZIP files to mimetype colors. Using model instead of first file's model. Fetching File models in batches to avoid issuing a SELECT for every File with a model. Flushing generated poster file. Getting Channel in VideoPage, relying on Video.channl_id, falling back to URL's channel_id. Attaching Video to it's Channel during download. Using ExternalCardLink for ArchivePage singlefile link. Validating and indexing videos after download. Handling glob when file name contains matching characters. Reworking and testing split_path_stem_and_suffix and glob_shared_stem. Adding associated video files to DB during upsert. Adding pending downloads to dashboard. Hiding status when download is open. Handling when load is empty. Adding load statistics to Dashboard. Limiting concurrent after_refresh functions. Handling when no onSelect is declared. Adding ImageRow for FileTable. Moving image date into Card.Meta. Using ExternalLink for FileCard. Using ExternalLink for ImageCard. Handling when error occurs during refresh of a directory. Committing during refresh to avoid losing all files if an error occurs. Removing api health check to avoid spamming logs. Creating ImageCard to display image posters when they aren't too large. Removing archive.domain_id not null. Reworking file refresh to process all files in a single directory. Bumping version for major file rework. No longer checking for video corruption. Adding duration by default to speed up tests. Using register_indexer for all indexers. Adding CPU usage progress bar in Welcome. Modifying TextIndexer to ignore large text files. Counting all files during refresh. Adding refresh_logger to clarify logging. Checking for wrol_mode before refresh functions. Removing unused idempotency from video/channel tables. Using magic module to speed up mimetype identification. All tests pass. Archive tests pass. Changing video get endpoint to return 'file' to avoid ambiguity. Video tests pass. Simplifying regex filtering. Using regex to ignore case when filtering. Converting all channel classes to functions. Restoring channel refresh and download buttons. Matching format of Archive and Video pages. Adding sorting to Archives search. Reworking video orders for file search. Using SearchInput for videos search. Archives can be searched on the Archives page. Domains can be filtered. Reworking Archive search to use Files. Videos, Archives and Global search use the FilesView. Videos are claimed by their Channel based on the Channel's directory. Prev and Next video cards are displayed on the VideoPage. Creating useVideo for reworked VideoWrapper and VideoPage. Adding Paginator to FilesView. Moving file refresh button out of manage tab. Removing Files page tabs. Limiting refresh to one at a time. Adding class for card icon to match poster height. Video and Archives always re-claim their files on refresh. Archive files are indexed and claimed. FileView table is compact and striped. Videos page uses FileView. Updated navlinks to router v6. Removing ! character from file names. Disabling select button when cards are visible. Adding show/hide button for FileTable selection. The Archive Page is displayed. Adding mimetype dropdown. Upgrading to React Router v6 for useSearchParams. Validating video during modeling. Removing screenshot during corruption check because it was very slow. Updating app to use video files as search results. Updating video search to return Files. Video and WROLPi tests pass. Creating function to refresh files within a channel's directory. Removing troubleshooting print. Updating result order in test. Reworking video_factory to handle a specific video file location. Returning video model on file search. Getting session using video_file. Screenshotting when checking for video corruption. Reducing ffmpeg calls by gathering duration during poster generation. Moving video file matching into model. VideoIndexer is moved into video module. Video files are validated. Moving files module into wrolpi in preparation to use it in modules. Title and contents are indexed for text file. Files are indexed on refresh by using their indexer. Zip files are indexed using their contents. Testing multiple zip files. Adding unique to file.path. Checking that the correct indexer is used. Adding index to file.modification_datetime and file.size. Handling file info gathering in File.do_index. Only reading contents of file if file.indexed is False. Video model is found during file refresh. A Video record is created that has the associated files. Videos without associated files are found. Videos are found using registered video_modeler() rather than in refresh code. Removing old video columns. Handling video info json files. Trying to import each module individually. Will continue to import modules even if one fails. Reworking refresh so it performs less DB calls. Converting file.path to primary key. file.path is always a pathlib.Path. Updating video_factory to use new file system, testing it. Bump sanic from 21.9.1 to 21.12.2 in /docker/archive Bumps [sanic](https://github.com/sanic-org/sanic) from 21.9.1 to 21.12.2. - [Release notes](https://github.com/sanic-org/sanic/releases) - [Changelog](https://github.com/sanic-org/sanic/blob/main/CHANGELOG.rst) - [Commits](https://github.com/sanic-org/sanic/compare/v21.9.1...v21.12.2) --- updated-dependencies: - dependency-name: sanic dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
2022-08-06 05:22:07 +00:00
const localOnOpen = (name) => {
setDownloader(name);
if (onOpen) {
onOpen(name);
}
Fixing JS warnings. Handling all combinations of toggle in light and dark. Changing CPUUsageProgress colors to be more readable in dark mode. Moving theme handling into Theme.js. Importing element wrappers from Theme.js to handle element theming. Changing video menu to dark mode. Creating ThemeContext to share Dark Mode setting. Removing duplicate code for search functions. Displaying X when downloads are disabled. Using react context to share settings across components. Refusing to start workers when WROL Mode is enabled. Changing warning to debug. Queuing recurring downloads first. Adding Video.url index. Adding special handling for youtube shorts to videos. Reworking Downloads to have one worker per process. Testing creating downloads and getting downloaders. Testing video validation and poster generation. Testing video statistics. Increasing inventory test coverage. Adding pytest.ini to automatically detect async test. Adding more archive test coverage. Testing Archive CRUD. Catching and logging when encountering error during indexing. Using FileIcon when Archive has no screenshot. A Video will be assigned to a Channel by its info json first, then to its directory. Including suffix in File json. Adding index for File.suffix. Gathering suffix during indexing. Passing entire file down to FileIcon. Moving logging up. Returning File.associated in file json. Associated files are last in file search. Removing duplicate refreshing code by creating new _refresh_directory function. Changing File.idempotency to a datetime to avoid having to clear idempotency. Truncating indexes to avoid error with large files. Don't overwrite video data when info json is empty. Using already_downloaded for channel videos. Awaiting refresh in Channel.refresh. Testing that videos already downloaded aren't downloaded again. Adding indexes for video/archive referenced paths. Changing grid width for mobile or computer. Adding button to refresh specific directories. Restoring Channel edit link. Fixing caption path in video_modeler. Adding ZIP files to mimetype colors. Using model instead of first file's model. Fetching File models in batches to avoid issuing a SELECT for every File with a model. Flushing generated poster file. Getting Channel in VideoPage, relying on Video.channl_id, falling back to URL's channel_id. Attaching Video to it's Channel during download. Using ExternalCardLink for ArchivePage singlefile link. Validating and indexing videos after download. Handling glob when file name contains matching characters. Reworking and testing split_path_stem_and_suffix and glob_shared_stem. Adding associated video files to DB during upsert. Adding pending downloads to dashboard. Hiding status when download is open. Handling when load is empty. Adding load statistics to Dashboard. Limiting concurrent after_refresh functions. Handling when no onSelect is declared. Adding ImageRow for FileTable. Moving image date into Card.Meta. Using ExternalLink for FileCard. Using ExternalLink for ImageCard. Handling when error occurs during refresh of a directory. Committing during refresh to avoid losing all files if an error occurs. Removing api health check to avoid spamming logs. Creating ImageCard to display image posters when they aren't too large. Removing archive.domain_id not null. Reworking file refresh to process all files in a single directory. Bumping version for major file rework. No longer checking for video corruption. Adding duration by default to speed up tests. Using register_indexer for all indexers. Adding CPU usage progress bar in Welcome. Modifying TextIndexer to ignore large text files. Counting all files during refresh. Adding refresh_logger to clarify logging. Checking for wrol_mode before refresh functions. Removing unused idempotency from video/channel tables. Using magic module to speed up mimetype identification. All tests pass. Archive tests pass. Changing video get endpoint to return 'file' to avoid ambiguity. Video tests pass. Simplifying regex filtering. Using regex to ignore case when filtering. Converting all channel classes to functions. Restoring channel refresh and download buttons. Matching format of Archive and Video pages. Adding sorting to Archives search. Reworking video orders for file search. Using SearchInput for videos search. Archives can be searched on the Archives page. Domains can be filtered. Reworking Archive search to use Files. Videos, Archives and Global search use the FilesView. Videos are claimed by their Channel based on the Channel's directory. Prev and Next video cards are displayed on the VideoPage. Creating useVideo for reworked VideoWrapper and VideoPage. Adding Paginator to FilesView. Moving file refresh button out of manage tab. Removing Files page tabs. Limiting refresh to one at a time. Adding class for card icon to match poster height. Video and Archives always re-claim their files on refresh. Archive files are indexed and claimed. FileView table is compact and striped. Videos page uses FileView. Updated navlinks to router v6. Removing ! character from file names. Disabling select button when cards are visible. Adding show/hide button for FileTable selection. The Archive Page is displayed. Adding mimetype dropdown. Upgrading to React Router v6 for useSearchParams. Validating video during modeling. Removing screenshot during corruption check because it was very slow. Updating app to use video files as search results. Updating video search to return Files. Video and WROLPi tests pass. Creating function to refresh files within a channel's directory. Removing troubleshooting print. Updating result order in test. Reworking video_factory to handle a specific video file location. Returning video model on file search. Getting session using video_file. Screenshotting when checking for video corruption. Reducing ffmpeg calls by gathering duration during poster generation. Moving video file matching into model. VideoIndexer is moved into video module. Video files are validated. Moving files module into wrolpi in preparation to use it in modules. Title and contents are indexed for text file. Files are indexed on refresh by using their indexer. Zip files are indexed using their contents. Testing multiple zip files. Adding unique to file.path. Checking that the correct indexer is used. Adding index to file.modification_datetime and file.size. Handling file info gathering in File.do_index. Only reading contents of file if file.indexed is False. Video model is found during file refresh. A Video record is created that has the associated files. Videos without associated files are found. Videos are found using registered video_modeler() rather than in refresh code. Removing old video columns. Handling video info json files. Trying to import each module individually. Will continue to import modules even if one fails. Reworking refresh so it performs less DB calls. Converting file.path to primary key. file.path is always a pathlib.Path. Updating video_factory to use new file system, testing it. Bump sanic from 21.9.1 to 21.12.2 in /docker/archive Bumps [sanic](https://github.com/sanic-org/sanic) from 21.9.1 to 21.12.2. - [Release notes](https://github.com/sanic-org/sanic/releases) - [Changelog](https://github.com/sanic-org/sanic/blob/main/CHANGELOG.rst) - [Commits](https://github.com/sanic-org/sanic/compare/v21.9.1...v21.12.2) --- updated-dependencies: - dependency-name: sanic dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
2022-08-06 05:22:07 +00:00
}
let body = (<>
<Button
color='blue'
content='Videos'
disabled={disabled}
onClick={() => localOnOpen('video')}
style={{marginBottom: '1em'}}
/>
<Button
color='green'
content='Archives'
disabled={disabled}
onClick={() => localOnOpen('archive')}
style={{marginBottom: '1em'}}
/>
<Button
color='blue'
content='Channel/Playlist'
disabled={disabled}
onClick={() => localOnOpen('video_channel')}
style={{marginBottom: '1em'}}
/>
<Button
content='RSS Feed'
disabled={disabled}
onClick={() => localOnOpen('rss')}
style={{marginBottom: '1em'}}
/>
<Button
color='black'
2023-11-02 10:07:49 -06:00
content='Files'
disabled={disabled}
onClick={() => localOnOpen('file')}
style={{marginBottom: '1em'}}
/>
<Button
color='red'
content='Scrape'
disabled={disabled}
onClick={() => localOnOpen('scrape')}
style={{marginBottom: '1em'}}
/>
</>);
function clearSelected() {
Fixing JS warnings. Handling all combinations of toggle in light and dark. Changing CPUUsageProgress colors to be more readable in dark mode. Moving theme handling into Theme.js. Importing element wrappers from Theme.js to handle element theming. Changing video menu to dark mode. Creating ThemeContext to share Dark Mode setting. Removing duplicate code for search functions. Displaying X when downloads are disabled. Using react context to share settings across components. Refusing to start workers when WROL Mode is enabled. Changing warning to debug. Queuing recurring downloads first. Adding Video.url index. Adding special handling for youtube shorts to videos. Reworking Downloads to have one worker per process. Testing creating downloads and getting downloaders. Testing video validation and poster generation. Testing video statistics. Increasing inventory test coverage. Adding pytest.ini to automatically detect async test. Adding more archive test coverage. Testing Archive CRUD. Catching and logging when encountering error during indexing. Using FileIcon when Archive has no screenshot. A Video will be assigned to a Channel by its info json first, then to its directory. Including suffix in File json. Adding index for File.suffix. Gathering suffix during indexing. Passing entire file down to FileIcon. Moving logging up. Returning File.associated in file json. Associated files are last in file search. Removing duplicate refreshing code by creating new _refresh_directory function. Changing File.idempotency to a datetime to avoid having to clear idempotency. Truncating indexes to avoid error with large files. Don't overwrite video data when info json is empty. Using already_downloaded for channel videos. Awaiting refresh in Channel.refresh. Testing that videos already downloaded aren't downloaded again. Adding indexes for video/archive referenced paths. Changing grid width for mobile or computer. Adding button to refresh specific directories. Restoring Channel edit link. Fixing caption path in video_modeler. Adding ZIP files to mimetype colors. Using model instead of first file's model. Fetching File models in batches to avoid issuing a SELECT for every File with a model. Flushing generated poster file. Getting Channel in VideoPage, relying on Video.channl_id, falling back to URL's channel_id. Attaching Video to it's Channel during download. Using ExternalCardLink for ArchivePage singlefile link. Validating and indexing videos after download. Handling glob when file name contains matching characters. Reworking and testing split_path_stem_and_suffix and glob_shared_stem. Adding associated video files to DB during upsert. Adding pending downloads to dashboard. Hiding status when download is open. Handling when load is empty. Adding load statistics to Dashboard. Limiting concurrent after_refresh functions. Handling when no onSelect is declared. Adding ImageRow for FileTable. Moving image date into Card.Meta. Using ExternalLink for FileCard. Using ExternalLink for ImageCard. Handling when error occurs during refresh of a directory. Committing during refresh to avoid losing all files if an error occurs. Removing api health check to avoid spamming logs. Creating ImageCard to display image posters when they aren't too large. Removing archive.domain_id not null. Reworking file refresh to process all files in a single directory. Bumping version for major file rework. No longer checking for video corruption. Adding duration by default to speed up tests. Using register_indexer for all indexers. Adding CPU usage progress bar in Welcome. Modifying TextIndexer to ignore large text files. Counting all files during refresh. Adding refresh_logger to clarify logging. Checking for wrol_mode before refresh functions. Removing unused idempotency from video/channel tables. Using magic module to speed up mimetype identification. All tests pass. Archive tests pass. Changing video get endpoint to return 'file' to avoid ambiguity. Video tests pass. Simplifying regex filtering. Using regex to ignore case when filtering. Converting all channel classes to functions. Restoring channel refresh and download buttons. Matching format of Archive and Video pages. Adding sorting to Archives search. Reworking video orders for file search. Using SearchInput for videos search. Archives can be searched on the Archives page. Domains can be filtered. Reworking Archive search to use Files. Videos, Archives and Global search use the FilesView. Videos are claimed by their Channel based on the Channel's directory. Prev and Next video cards are displayed on the VideoPage. Creating useVideo for reworked VideoWrapper and VideoPage. Adding Paginator to FilesView. Moving file refresh button out of manage tab. Removing Files page tabs. Limiting refresh to one at a time. Adding class for card icon to match poster height. Video and Archives always re-claim their files on refresh. Archive files are indexed and claimed. FileView table is compact and striped. Videos page uses FileView. Updated navlinks to router v6. Removing ! character from file names. Disabling select button when cards are visible. Adding show/hide button for FileTable selection. The Archive Page is displayed. Adding mimetype dropdown. Upgrading to React Router v6 for useSearchParams. Validating video during modeling. Removing screenshot during corruption check because it was very slow. Updating app to use video files as search results. Updating video search to return Files. Video and WROLPi tests pass. Creating function to refresh files within a channel's directory. Removing troubleshooting print. Updating result order in test. Reworking video_factory to handle a specific video file location. Returning video model on file search. Getting session using video_file. Screenshotting when checking for video corruption. Reducing ffmpeg calls by gathering duration during poster generation. Moving video file matching into model. VideoIndexer is moved into video module. Video files are validated. Moving files module into wrolpi in preparation to use it in modules. Title and contents are indexed for text file. Files are indexed on refresh by using their indexer. Zip files are indexed using their contents. Testing multiple zip files. Adding unique to file.path. Checking that the correct indexer is used. Adding index to file.modification_datetime and file.size. Handling file info gathering in File.do_index. Only reading contents of file if file.indexed is False. Video model is found during file refresh. A Video record is created that has the associated files. Videos without associated files are found. Videos are found using registered video_modeler() rather than in refresh code. Removing old video columns. Handling video info json files. Trying to import each module individually. Will continue to import modules even if one fails. Reworking refresh so it performs less DB calls. Converting file.path to primary key. file.path is always a pathlib.Path. Updating video_factory to use new file system, testing it. Bump sanic from 21.9.1 to 21.12.2 in /docker/archive Bumps [sanic](https://github.com/sanic-org/sanic) from 21.9.1 to 21.12.2. - [Release notes](https://github.com/sanic-org/sanic/releases) - [Changelog](https://github.com/sanic-org/sanic/blob/main/CHANGELOG.rst) - [Commits](https://github.com/sanic-org/sanic/compare/v21.9.1...v21.12.2) --- updated-dependencies: - dependency-name: sanic dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
2022-08-06 05:22:07 +00:00
localOnOpen(null);
body = null;
}
const downloaders = {
archive: <ArchiveDownloadForm onCancel={clearSelected}/>,
video: <VideosDownloadForm singleDownload={false} onCancel={clearSelected}/>,
video_channel: <ChannelDownloadForm onCancel={clearSelected}/>,
rss: <RSSDownloadForm onCancel={clearSelected}/>,
file: <FilesDownloadForm onCancel={clearSelected}/>,
scrape: <ScrapeFilesDownloadForm onCancel={clearSelected}/>
};
if (downloader in downloaders) {
const downloaderForm = downloaders[downloader];
body = <>
{downloaderForm}
</>
}
Fixing JS warnings. Handling all combinations of toggle in light and dark. Changing CPUUsageProgress colors to be more readable in dark mode. Moving theme handling into Theme.js. Importing element wrappers from Theme.js to handle element theming. Changing video menu to dark mode. Creating ThemeContext to share Dark Mode setting. Removing duplicate code for search functions. Displaying X when downloads are disabled. Using react context to share settings across components. Refusing to start workers when WROL Mode is enabled. Changing warning to debug. Queuing recurring downloads first. Adding Video.url index. Adding special handling for youtube shorts to videos. Reworking Downloads to have one worker per process. Testing creating downloads and getting downloaders. Testing video validation and poster generation. Testing video statistics. Increasing inventory test coverage. Adding pytest.ini to automatically detect async test. Adding more archive test coverage. Testing Archive CRUD. Catching and logging when encountering error during indexing. Using FileIcon when Archive has no screenshot. A Video will be assigned to a Channel by its info json first, then to its directory. Including suffix in File json. Adding index for File.suffix. Gathering suffix during indexing. Passing entire file down to FileIcon. Moving logging up. Returning File.associated in file json. Associated files are last in file search. Removing duplicate refreshing code by creating new _refresh_directory function. Changing File.idempotency to a datetime to avoid having to clear idempotency. Truncating indexes to avoid error with large files. Don't overwrite video data when info json is empty. Using already_downloaded for channel videos. Awaiting refresh in Channel.refresh. Testing that videos already downloaded aren't downloaded again. Adding indexes for video/archive referenced paths. Changing grid width for mobile or computer. Adding button to refresh specific directories. Restoring Channel edit link. Fixing caption path in video_modeler. Adding ZIP files to mimetype colors. Using model instead of first file's model. Fetching File models in batches to avoid issuing a SELECT for every File with a model. Flushing generated poster file. Getting Channel in VideoPage, relying on Video.channl_id, falling back to URL's channel_id. Attaching Video to it's Channel during download. Using ExternalCardLink for ArchivePage singlefile link. Validating and indexing videos after download. Handling glob when file name contains matching characters. Reworking and testing split_path_stem_and_suffix and glob_shared_stem. Adding associated video files to DB during upsert. Adding pending downloads to dashboard. Hiding status when download is open. Handling when load is empty. Adding load statistics to Dashboard. Limiting concurrent after_refresh functions. Handling when no onSelect is declared. Adding ImageRow for FileTable. Moving image date into Card.Meta. Using ExternalLink for FileCard. Using ExternalLink for ImageCard. Handling when error occurs during refresh of a directory. Committing during refresh to avoid losing all files if an error occurs. Removing api health check to avoid spamming logs. Creating ImageCard to display image posters when they aren't too large. Removing archive.domain_id not null. Reworking file refresh to process all files in a single directory. Bumping version for major file rework. No longer checking for video corruption. Adding duration by default to speed up tests. Using register_indexer for all indexers. Adding CPU usage progress bar in Welcome. Modifying TextIndexer to ignore large text files. Counting all files during refresh. Adding refresh_logger to clarify logging. Checking for wrol_mode before refresh functions. Removing unused idempotency from video/channel tables. Using magic module to speed up mimetype identification. All tests pass. Archive tests pass. Changing video get endpoint to return 'file' to avoid ambiguity. Video tests pass. Simplifying regex filtering. Using regex to ignore case when filtering. Converting all channel classes to functions. Restoring channel refresh and download buttons. Matching format of Archive and Video pages. Adding sorting to Archives search. Reworking video orders for file search. Using SearchInput for videos search. Archives can be searched on the Archives page. Domains can be filtered. Reworking Archive search to use Files. Videos, Archives and Global search use the FilesView. Videos are claimed by their Channel based on the Channel's directory. Prev and Next video cards are displayed on the VideoPage. Creating useVideo for reworked VideoWrapper and VideoPage. Adding Paginator to FilesView. Moving file refresh button out of manage tab. Removing Files page tabs. Limiting refresh to one at a time. Adding class for card icon to match poster height. Video and Archives always re-claim their files on refresh. Archive files are indexed and claimed. FileView table is compact and striped. Videos page uses FileView. Updated navlinks to router v6. Removing ! character from file names. Disabling select button when cards are visible. Adding show/hide button for FileTable selection. The Archive Page is displayed. Adding mimetype dropdown. Upgrading to React Router v6 for useSearchParams. Validating video during modeling. Removing screenshot during corruption check because it was very slow. Updating app to use video files as search results. Updating video search to return Files. Video and WROLPi tests pass. Creating function to refresh files within a channel's directory. Removing troubleshooting print. Updating result order in test. Reworking video_factory to handle a specific video file location. Returning video model on file search. Getting session using video_file. Screenshotting when checking for video corruption. Reducing ffmpeg calls by gathering duration during poster generation. Moving video file matching into model. VideoIndexer is moved into video module. Video files are validated. Moving files module into wrolpi in preparation to use it in modules. Title and contents are indexed for text file. Files are indexed on refresh by using their indexer. Zip files are indexed using their contents. Testing multiple zip files. Adding unique to file.path. Checking that the correct indexer is used. Adding index to file.modification_datetime and file.size. Handling file info gathering in File.do_index. Only reading contents of file if file.indexed is False. Video model is found during file refresh. A Video record is created that has the associated files. Videos without associated files are found. Videos are found using registered video_modeler() rather than in refresh code. Removing old video columns. Handling video info json files. Trying to import each module individually. Will continue to import modules even if one fails. Reworking refresh so it performs less DB calls. Converting file.path to primary key. file.path is always a pathlib.Path. Updating video_factory to use new file system, testing it. Bump sanic from 21.9.1 to 21.12.2 in /docker/archive Bumps [sanic](https://github.com/sanic-org/sanic) from 21.9.1 to 21.12.2. - [Release notes](https://github.com/sanic-org/sanic/releases) - [Changelog](https://github.com/sanic-org/sanic/blob/main/CHANGELOG.rst) - [Commits](https://github.com/sanic-org/sanic/compare/v21.9.1...v21.12.2) --- updated-dependencies: - dependency-name: sanic dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
2022-08-06 05:22:07 +00:00
return <>
{body}
</>
}