Commit graph

10 commits

Author SHA1 Message Date
Roland
63c5f2b8d7
config is no longer mounted. 2022-03-12 10:17:04 -07:00
Roland
79e1c8c1df
Merge branch 'master' of ssh://github.com-lrnselfreliance/lrnselfreliance/wrolpi 2022-03-09 16:09:06 -07:00
Roland
d0de92ef7e
Skipping date tests on circleci. 2022-03-09 15:44:02 -07:00
Roland
888352c984
Squashed commit of the following:
commit 26bde0fb48d59a86c2fb9f53a7218d49f22d4860
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Mar 8 13:25:56 2022 -0700

    Using osm2pgsql append to preserve other imported maps.

commit c6c52623eaf02e691b1e75dcf39c3a8334ead55a
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Mar 8 08:20:38 2022 -0700

    Using placeholder instead of loader when table is loading.

commit 44fec6490e26603c88a98f6bc70fc18686e17f09
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 20:58:34 2022 -0700

    Simplifying tableRow function.

commit 11b8b57f1fa59556397620f3cad9a33a0237501d
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 20:53:51 2022 -0700

    Removing scientific humanFileSize.

commit 0a09d0a6963afebed7f388aad71bbdd3151d2e9e
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 20:46:22 2022 -0700

    Fixing active loader.

commit 07f4deeac861351d7483ef57278eb2e0941e31fb
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 20:40:44 2022 -0700

    Increasing recommended map file size.

commit af3ac492c81f4c8f84563a448bc272ae9334d32d
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 20:35:13 2022 -0700

    Reworking logic order for simplicity.  Fetching downloads on interval.

commit 3d9d31fcec87ed02ced7f67c34907faa5a3b54e5
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 20:24:39 2022 -0700

    Handling when pbf import status are loading.

commit 3798961a48e7c9eb165ca4e4051f239dbb088d0c
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 19:53:50 2022 -0700

    Fetching import status every 30 seconds.

commit f7a8193b0955d14ead8090aa74ce3c5a681b57ec
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 19:33:17 2022 -0700

    Stopping renderd during installation.

commit 5bf050b59e0a6e391184596f5e6f0737dea1221d
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 19:26:59 2022 -0700

    Removing duplciate serve.

commit 892e3315eeeb5d2516996b901a32a73a66dae078
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 19:22:14 2022 -0700

    Setting bash flags in sub-install script.

commit d5d2acd2cd85e089c78aa85a27c5d966222a1347
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 19:20:19 2022 -0700

    Moving bulk of installation script into separate script so the latest code will be used during initial installation.

commit ab030d8af834a4a8772edd06df360c51e72fbb02
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 18:40:34 2022 -0700

    Removing mod_tile/ajt directory instead of entire contents of mod_tile.  Showing import as relative to media directory.

commit c05199db4ecf56f4d096051b58689457882af12a
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 18:39:59 2022 -0700

    Remvoing marker from map.

commit ba5df4cf62d61b55281277a6254836e973c766ae
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 11:57:41 2022 -0700

    Creating import_map.sh script.

commit 1a16e46130e775efe375adb22917ba12e44855cd
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 11:11:01 2022 -0700

    Increasing ram usage during import to avoid hanging import.

commit 0f5ce1a9f806f1c0878dd804b04587b6340ffd16
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 19:19:47 2022 -0700

    Increasing map_file.size size.

commit eb6e01512af59fb4a18ab15643c7bc61f3d48e53
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 18:55:55 2022 -0700

    Copying leaflet js/css files into docker container and to Apache directory so they will be available offline.

commit 5b687324e119f021673f42e8d079b83034b70d53
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 18:36:06 2022 -0700

    Matching tab height on Map page.

commit 8031786d672e18366e9a2eb2d7bdfcd8611d27f3
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 18:34:20 2022 -0700

    Shrinking HelpPopup Icon.

commit 5ea94488e43bbc5c414e61c9173a3cfa908c7104
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 18:31:30 2022 -0700

    Testing if no PBFs are requested to be imported.

commit c4e86be1faed78bd3af1f3487471c01a1958a433
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 18:18:48 2022 -0700

    Using Icon instead of Button for HelpPopup.

commit 51d94f6d85885803813544fea16a56372babaa82
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 18:07:03 2022 -0700

    PBF files are sorted by the api.

commit 40f3a7ff04a83db0153f64a88a37f1d56ce043aa
Author: Roland <roland@learningselfreliance.com>
Date:   Fri Mar 4 18:35:51 2022 -0700

    Adding help popup.

commit 1da8d9fa403ec2a3b3c16c11a3937de37c895a35
Author: Roland <roland@learningselfreliance.com>
Date:   Fri Mar 4 09:41:01 2022 -0700

    Moving message up.  Clearing cached tiles after import.

commit 3192adf95192554c194c9e0a8aeb149822462682
Author: Roland <roland@learningselfreliance.com>
Date:   Thu Mar 3 21:37:13 2022 -0700

    Removing osm2pgsql from api docker because it seems to be unsupported.

commit 0e10522f3cdc7edd374c14f230bb2dbd68ab1a94
Author: Roland <roland@learningselfreliance.com>
Date:   Thu Mar 3 21:36:39 2022 -0700

    Disabling import button for docker.  Adding download instructions.

commit e1d8f55d159fb4d4ef2ece859ef7cc1059885791
Author: Roland <roland@learningselfreliance.com>
Date:   Thu Mar 3 21:14:55 2022 -0700

    Map files can be imported on RPi.

commit 3128530e09968430a71b27cd2e75a567f85a708b
Author: Roland <roland@learningselfreliance.com>
Date:   Thu Mar 3 15:31:54 2022 -0700

    Adding shm_size to increase size to render large map tiles.

commit f3cf80fae0ce46f44ea9ac5033f68a32d0197ae3
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Mar 2 10:39:18 2022 -0700

    Can get status of map imports.

commit 3b7bbce004cd55a5d89980c2c09124b72c450bc6
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Mar 1 20:51:13 2022 -0700

    Adding map link to 50x.html.  Swapping links using hostname to make it easier on the user.

commit a8b6b60fe182233f40c2f1ecd825636c3d37fd23
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Mar 1 12:01:50 2022 -0700

    Moving Inventory link to end.

commit c5346b8dc2ff83262d6fd153130912809dfd7574
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Mar 1 12:00:47 2022 -0700

    Squashed commit of the following:

    commit 39f0b8f3fc049fffd140d551cd91d7d9dba6d416
    Author: Roland <roland@learningselfreliance.com>
    Date:   Tue Mar 1 12:00:13 2022 -0700

        Starting map refresh function.

    commit 2fac9ddf07e378389ddd85ca972f33b3323d30ac
    Author: Roland <roland@learningselfreliance.com>
    Date:   Mon Feb 28 16:43:47 2022 -0700

        Updating readme with map requirements.

    commit 8325e1ea89537128329e313ab4b5588f8809d1c9
    Author: Roland <roland@learningselfreliance.com>
    Date:   Mon Feb 28 16:05:35 2022 -0700

        Fixing order of map db init scripts.  Creating index.html to display map.  Using :8084 index file to display the map.

    commit 8f81539dd9d25fc26607f472c156d41ddcfc11f5
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 19:17:14 2022 -0700

        Installing apache2.

    commit c7aaef07f8291070aeaf5e7df553e34aca0719ec
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 18:41:07 2022 -0700

        Always configure postgres extensions.  Initialize gis with dc.

    commit aa725ed4b8ec46d17accebd96dd41508ea2d9caf
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 15:48:20 2022 -0700

        Map is displayed as iframe.

    commit de9bb92dcaf44996e7727adce7774f01895ddf52
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 12:16:56 2022 -0700

        Running script in openstreetmap directory.

    commit f277388095d2221aa6407643ab655f26dc349eb7
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 12:06:41 2022 -0700

        Bumping version for map.

    commit afd4aaeffb2aced2dc5c1a8f94038885c8287265
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 11:59:31 2022 -0700

        Checking that the installation script is on an rpi.

    commit a799382ceac3c201fc01d41fd63e37607282831a
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 11:49:21 2022 -0700

        Only running mod_tile install once.

    commit 0b98bb19cd4d86c9630a863cab4eb46410d1c9d1
    Author: Roland <roland@learningselfreliance.com>
    Date:   Fri Feb 25 11:31:21 2022 -0700

        Building out map install script.

    commit 7c799775c11df573fa6cd65451adea28c0832cb0
    Author: Roland <roland@learningselfreliance.com>
    Date:   Thu Feb 24 12:16:53 2022 -0700

        Creating OSM installation script.

    commit d3b93a3ee97d1ae4a00b6a0423ed59ea8b897ccd
    Author: Roland <roland@learningselfreliance.com>
    Date:   Tue Feb 22 15:39:04 2022 -0700

        Removing unused map module.
2022-03-08 13:26:03 -07:00
Roland
7244f2c557
Mounting test directory to api's /media/wrolpi. Updating Archive's Sanic to avoid depreciation error with uvloop. 2022-01-22 19:54:46 -07:00
lrnselfreliance
46796f783a
Arbitrary videos can be downloaded. (#63)
* Squashed commit of the following:

commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Dec 14 22:16:36 2021 -0700

    Only saving a favorite video to the config if it has a source_id.

commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Dec 14 22:14:26 2021 -0700

    Ading description and changing Save button to Download.

commit 03a72be6dfdfa383915950a80e3f1ee9febf6705
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Dec 14 22:09:44 2021 -0700

    Removing uploader for now.

commit 5d0231284bf84e459687f6ab903b06bf4e3762bd
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Dec 14 22:07:17 2021 -0700

    Removing unique data from example info.

commit 9cc721a32d4715d9d6433e51db3b35e039c78d68
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Dec 14 22:02:33 2021 -0700

    Updating downloader tests.

commit 8e193b03349ae4613f105418cd3e8176c7101304
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Dec 14 21:53:31 2021 -0700

    Toasting error when WROL Mode is enabled.  Disabling new downloads during WROL Mode.

commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Dec 14 21:45:06 2021 -0700

    Raising error if download exits without a 0.  Clearing kill event when starting a new download.

commit 54ba3a890d59facc837410d34bf7afb40db3b685
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Dec 14 21:16:46 2021 -0700

    Downloads can be restarted and killed.

commit bd60751262247259c3e9ceb5116e017d6314c219
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Dec 13 16:28:18 2021 -0700

    Downloads can be started again.  Downloads are sorted on Downloads page.

commit ea7f39231b92982ae0b263b76c5bd1902cc8169d
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Dec 11 21:24:16 2021 -0700

    Building Downloads tab.

commit d3321872ac3743ff6df781b07ef0fb537f1d4c98
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Dec 11 20:41:46 2021 -0700

    Not scheduling channels withtout a URL/frequency.

commit 84f268a0d4f58c85f86b6377900492e0766f6619
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Dec 11 19:56:08 2021 -0700

    Channel downloads will be spread out when each channel is created.

commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Dec 11 13:08:11 2021 -0700

    Recurring downloads are renewed on startup.

commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6
Author: Roland <roland@learningselfreliance.com>
Date:   Fri Dec 10 17:03:57 2021 -0700

    Fetching download again after do_download.

commit fdadad327fc09c17a49eb63b11e632fb72cc3195
Author: Roland <roland@learningselfreliance.com>
Date:   Fri Dec 10 16:20:51 2021 -0700

    Fixing error with test.  Mock was not being reset.

commit 070dee238e53377acd22097c63cf3794e68ed423
Author: Roland <roland@learningselfreliance.com>
Date:   Fri Dec 10 15:36:39 2021 -0700

    Fixed bug in date handling (big surprise).

commit e53a8992268d68620c679455adaec3a590f20602
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Dec 1 16:48:36 2021 -0700

    Using chunks for regular channel refresh.

commit 3bd60c58d1bef0226325f321978c0434b36b3be5
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Dec 1 15:22:02 2021 -0700

    Reformatting.

commit 4fdc325f25a32c311ca6543fe321624b2bd564b2
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Dec 1 15:13:34 2021 -0700

    Creating function that refreshes NO CHANNEL videos.

commit d67feafb6c2994f5a4a1295920a70db369cc6816
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Dec 1 14:35:54 2021 -0700

    Removing unused functions.

commit 6b3918e4e78fbac64920f85c4de0df873ec366a1
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Dec 1 14:31:48 2021 -0700

    Removing reporter and events.  This will be replaced in the future with some sort of SSE.

commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1
Author: Roland <roland@learningselfreliance.com>
Date:   Sun Nov 28 17:55:43 2021 -0700

    Better error handling for archive button.

commit c31d124e511b27f626ed2df71486d47302b396f3
Author: Roland <roland@learningselfreliance.com>
Date:   Sun Nov 28 17:29:43 2021 -0700

    Displaying View Count in a human readable format.

commit 0ee2bd65588fd93b475cad6144e76f1b45191b39
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Nov 27 09:29:41 2021 -0700

    Fixing tests.

commit e64aff32bab95c311ea3747e1a21b58b99750d5e
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Nov 27 08:59:51 2021 -0700

    Handling when a Video has no Channel in UI.

commit b9549ecd5aac03c002adca7b7667f0a7189b662a
Author: Roland <roland@learningselfreliance.com>
Date:   Fri Nov 26 08:35:36 2021 -0700

    Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried.

commit da88798e2584d3e84900a70e34166c8c208c64fb
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Nov 24 20:40:29 2021 -0700

    Downloaders can refuse a download, which will then be deleted.

commit b865d856121db5ec8a3660c5d3c13b9846fde688
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Nov 24 19:35:43 2021 -0700

    Removing relative path handling.

commit 6d7757870b02ba030d78b3ef7e43f28b906eea61
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Nov 24 19:28:27 2021 -0700

    Using __json__ for Video and Channel models.

commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Nov 24 18:23:38 2021 -0700

    Fixing video links in UI.

commit 9d081af12742070b8519bd752812130cb5dbe393
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Nov 24 14:42:38 2021 -0700

    Fixing docs.

commit 1202cbe0f9ae744d7801b53a5071cded8ea39924
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Nov 24 14:40:36 2021 -0700

    Fixing wrolpi tests.

commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Nov 24 14:13:35 2021 -0700

    Fixing tests to use MediaPath.

commit faec04f2a5e4c267a502c03884940a9c5bcfe387
Author: Roland <roland@learningselfreliance.com>
Date:   Fri Nov 19 10:21:20 2021 -0700

    Testing when a video has no channel.

commit cbf22aad7a0b72e61685e0236e843554b57c4af1
Author: Roland <roland@learningselfreliance.com>
Date:   Thu Nov 18 16:24:03 2021 -0700

    Using MediaPath for Video and Channel models.  Testing VideoDownloader.do_download.

commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d
Author: Roland <roland@learningselfreliance.com>
Date:   Thu Nov 18 12:43:44 2021 -0700

    Channels/Videos downloaders can get gotten.

commit 808ef796cec01db6e7009065f9e26d25d9500ce2
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Nov 17 21:45:19 2021 -0700

    Upgrading to Sanic 21.9.1.

commit ea32813be28e78393ac456ee782a7245c098188f
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Nov 17 20:29:01 2021 -0700

    Creating Downloader and DownloaderManager classes which handle downloading URLs.

commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Nov 17 12:41:39 2021 -0700

    Creating UI segement which allows the download/upload of media.

* Displaying all recurring downloads.

* Not downloading channels with empty url.

* Showing positive row for recurring downloads.

* Removing default last_successful_download.

* Only displaying Last Successful Download when there is one.

* Refreshing videos before finding missing videos.

* Using short video path name for log.

* Removing download history feature in favor of new Downloads page.

* Only showing info date if it exists.

* Making get_or_create_channel more reliable.

* Allowing null for channel download frequency.

* Logging when searching for channel.

* Storing channel's source_id in config.

* Importing using link instead of name.

* Getting channel id when importing channels.

* Removing redunant parentheses from repr.

* Preserving database channel values if there are not values in the config.

* Only highlighting pending downloads.

* Deleting old downloads periodically.

* Sorting recurring by next_download/frequency.

* Creating fixtures for channels, media directory and db session.

* Removing unused function.  Wrapping config for tests.

* Deleting channel download when deleting a channel.

* Testing that a channel's download is deleted when the channel is deleted.

* Updating youtube-dl

* Switching from youtube-dl in favor of yt-dlp.

* Sorting once-downlaods by last_successful_download then id.

* Adding special handling to video_url_resolver for full urls.

* Consolidating downloader methods and functions.  Reversing once-download sort order so the most recently downloaded are at the top.

* Getting session if one is not provided.

* Adding test for create_downloads.

* Not passing format option when getting filename.

* Linting

Signed-off-by: Roland <roland@learningselfreliance.com>
2021-12-20 16:09:11 -07:00
Roland
d6769c4922
Compressing archive bytes before response. 2021-11-10 10:11:02 -07:00
Roland
7807c44fad
Installing curl into archive container so health check works. 2021-11-09 13:27:22 -07:00
Roland
7e0fcb2577
Restoring development nginx config. Reverting react-timezone-select version to avoid compilation error. 2021-11-02 10:26:07 -06:00
Roland
9b899f6fc0
A new start
Signed-off-by: Roland <roland@learningselfreliance.com>
2021-11-01 16:43:09 -06:00