Commit graph

62 commits

Author SHA1 Message Date
Roland
50d95dad19
Channels now have Channel.channel_downloads. A Channel can have many Downloads. 2024-10-16 18:55:46 -06:00
Roland
0fd545d2e8
Setting FileGroup.censored when refreshing channel files. 2024-06-17 14:48:41 -06:00
Roland
00fe799b8f
User can select from/to years when searching files. Indexing doc and docx files. 2024-03-07 12:12:48 -07:00
Roland
758ff8e810
Creating background process that fetches comments for Videos that do not yet have comments.
Displaying star for Uploader's comments, heart for favorited comments.

Adding wrolpi help files desktop shortcut to rpi image.

Using zim file names as FileGroup.title.

Removing accidental output commit.
2024-03-01 17:15:56 -07:00
Roland
0629c8b464
Converting VideoPlayer to use FileGroup.url. Fixing downgrade to use FileGroup.url. 2023-10-29 21:13:38 -06:00
Roland
15687e534a
Moving data from models to FileGroup.
* Ebook.creator -> FileGroup.author
* Video.viewed -> FileGroup.viewed
* Video.duration -> FileGroup.length
* Video.url -> FileGroup.url
* Archive.url -> FileGroup.url
2023-10-29 17:31:54 -06:00
Roland
ae9d188391
Adding FileGroup.published_date/author/etc. Creating script which converts a Singlefile to an Archive FileGroup files.
Parsing SingleFile datetime using dates.strpdate.

Adding timezone when no timezone info is available.

Installing python-dateutil to assist in handling diverse datetime strings.

Reworking refresh progress to use idempotency to display more accurate progress.  Only logging that files have been invalidated if they used to be indexed.

Clearing archives to force re-index.

Ignoring 'Unknown' author from PDF metadata. Using file_title when pdf metadata title is empty.
2023-10-23 18:09:08 -06:00
Roland
6e0f9a999c
Combining install scripts into a new repair script. Removing create_db script in favor of repair script. 2023-10-03 12:15:29 -06:00
Roland
2147bf90e3
Removing Video.validated because it was only used in the script. 2023-08-15 13:17:30 -06:00
Roland
e8e5b5727f
Creating Video.validated which is used by find_corrupt_videos.py to search. 2023-08-08 13:42:55 -06:00
Roland
36083200e6
Creating Video.ffprobe_json column which stores ffprobe information about the video file. Creating find_corrupt_videos.py script which automatically re-downloads videos which do not have both video and audio streams. 2023-08-03 21:40:25 -06:00
Roland
03a2df217d
Zim/Kiwix (Wikipedia) feature
* A new zim module has been added, this module allows several things to interact with Zim files.
  * Allow searching of Zim files using libzim.
  * Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
  * Creation of a new zim docker container which allows viewing of the Zim file contents.

* This commit also adds move/rename/mkdir features in the File Browser.
  * File tags are preserved when moving/renaming.
2023-07-10 21:30:40 -06:00
Roland
fedbbe31d7
Changing default Survival tag to Primitive. 2023-05-18 17:25:33 -06:00
Roland
7c1cc07365
Only joining FileGroup table when searching Archives when necessary. 2023-05-16 14:56:53 -06:00
Roland
0c0115dcd2
Adding Pests to default Tags. 2023-05-10 15:25:06 -06:00
Roland
387282b445
Adding my public gpg key.
Moving upload destination input out of drop Segment.

Disabling uploads when downloads are disabled.

Inverting labels for checkboxes in inverted modal.

Deleting file_groups which no longer contain the exclusive primary path.

Adding debug logging for all API errors.

Checking that upload output is not a directory.

Expanding nginx request size.

Adding docs to upload endpoint.

Using dot-circle icon for more image mimetypes.

Using Form.Field to make upload destination required.

Adding border to inverted Segments.

Many files can be uploaded in chunks.

Files can be uploaded, progress is displayed.

Showing white border around share QR code.

Displaying WROLPi colors for download icons.

Creating button to clear DirectorySearch.

Excluding channel/domain directories so other directories can be returned.

Ordering search results.

Ignoring when no channel is found only if we have a destination.

Limiting directory search results.

Displaying categories when searching destination directories.

Handling when live_chat.json file exists.

Allowing refresh of directory directly.

Logging which download was successful.

Removing timeout.

Creating index for directory names.

Sleeping to catch cancel.

Arbitrary files can be downloaded using the new FileDownloader.
2023-04-28 13:56:32 -06:00
Roland
d9a737d46b
Adding default to TagFile.created_at. 2023-04-10 08:41:35 -06:00
Roland
189f25399a
Adding WROL to default tags. 2023-04-09 17:14:22 -06:00
Roland
18273286dc
Removing FileGroup.full_stem in favor of primary_path. This is to handle when two unrelated files share a name ('some text.txt' and 'some text.jpg'). 2023-04-09 16:59:20 -06:00
Roland
92d0795000
Adding file_group_textsearch_idx index. 2023-04-04 16:01:31 -06:00
Roland
ae2b6daa33
FileBrowser files can be tagged. A Tag icon is displayed on each file row in the browser. 2023-04-01 11:52:41 -06:00
Roland
900f605d27
Replacing default tag colors. 2023-03-29 17:37:13 -06:00
Roland
a7c4fd1119
Displaying progress bar during refresh. 2023-03-27 20:46:31 -06:00
Roland
1dd19a8f5e
Reworking tag endpoints in a more restful way. 2023-03-27 16:17:33 -06:00
Roland
9b5c82219d
Adding indexes after file_groups. 2023-03-25 20:55:44 -06:00
Roland
5615ce542d
Making example label a tag.
Tags can be created and deleted in UI.

Adding tag_file using id to avoid session error.

Using UTC in test.

Matching tag_names style in test.

Tag icon is displayed on file rows.

Archives, Videos and Files can be filtered by their tags.

Tags can be added and removed from UI.

Tags are displayed in VideoPlayer.

The tags of a file are returned in it's JSON.

Video favorites are converted to Favorite tag on channel config import.

Updating tests to match frontend usage.

PDF creator and modification time are displayed.

Reworking file search to use Video.__json__ for complex video relations.

Archives can be viewed.

Removing ebook.title.

Reworking alembic migration to be an upgrade rather than a replace.

Skipping test on circleci.

Clearing out video caption index only if extract_subtitles if off.

Channel tests pass.

Some video tests pass.

Missing videos can be found.  A video can be downloaded.

Orphaned files can be found.

Videos can be tagged.

Videos can be searched.

Videos can be modeled.

Converting some tests to use FileGroup.primary_path.

Adding archive and domains to alembic migration.

Testing cascading Archive titles.

Archives now uses FileGroup.

archive.lib tests pass.

Archive is partially converted to FileGroup.

Calculating FileGroup.primary file on refresh, rather than on select.

wrolpi tests pass.

Using FilesType to convert between str/pathlib.Path for FileGroup.files.

Using list of files in FileGroup.files instead of dictionary.

Storing files in dedicated FileGroup.files to avoid overwriting them.

Adding comment explaining CASE.

Not overwriting FileGroup.data on refresh if no change has been detected.

Removing File model in favor of FileGroup.data.

Adding typing to Files methods.

Files are indexed using the primary mimetype of the FileGroup.

files test_lib tests pass.

Removing file_group_file in favor of simple one-to-many relationship.

PDF files are indexed during refresh.

Adding Tag and TagFile to tag files with specific tags.

Only iterating when full_stem_ids has contents.

Testing that FileGroups are related to their Files.

Files and file_groups are created and linked.

Bump webpack from 5.75.0 to 5.76.1 in /app (#160)

Bumps [webpack](https://github.com/webpack/webpack) from 5.75.0 to 5.76.1.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.75.0...v5.76.1)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-25 17:54:41 -06:00
Roland
93f0e7494f
Creating Event and Flag systems. Events are used to send and display a toast to the App. Flags are used to signal if some action has been performed, or if a package is installed. 2022-12-23 16:04:09 -07:00
Roland
d8c166379b
Removing unused File columns and indexes. 2022-12-16 10:00:46 -07:00
Roland
6fcb992540
Storing example ebooks for testing.
Special handling of json mimetype for json files.

Moving types up to avoid overwriting mime.types.

Adding custom types to nginx to server ebooks better.

Displaying EBookCard correctly when ebook was not indexed correctly.

Global search can search for all ebooks (mobi/epub).

Testing when mimetype suffix is searched.

Adding eBook to global search filter list.  Opening mimetype search to match mimetypes that may not end with expected suffix.

Adding log warning to explain refresh speed.

Generating ebook covers after refresh.

Testing that another nearby book removes the cover.

Calibre covers can be discovered and associated with an ebook.

Handling if ebooklib is not yet installed.

Adding ebook table and modeler to allow searching the contents of epub files.

Marking all PDFs as un-indexed, so they will be indexed.

Replacing null characters in PDF text.

Testing PDFIndexer against example PDF file.

Installing PyPDF2 to extract text for PDFIndexer.
2022-12-11 12:36:53 -07:00
Roland
c106fdaaf7
Invalidating video files for orphaned video meta files. 2022-12-07 18:27:45 -07:00
Roland
d0d69519cd
Bumping version for new feature.
Skipping video epoch test on circleci.

Created script which finds and cleans up (renames/deletes) orphaned video meta files.

Reworking orphaned video query, some video files were leaking through.

Modifying File table to allow finding of orphaned video files.

Extracting epoch from video info_json for more accurate upload_date.

Extracting epoch from video info_json for more accurate upload_date.
2022-12-07 18:15:28 -07:00
Roland
d689f3b5e7
Creating index for archive.url. 2022-11-02 15:21:34 -06:00
Roland
287edad9e6
Adding Download.settings column in alembic migration.
Reformatting.

Switching excluded_urls to a generic Download.settings json.

Download workers filter excluded urls.
2022-10-14 13:08:39 -06:00
Roland
90f59a1a7b
Adding on-delete-cascade to video and archive tables. 2022-09-28 15:25:10 -06:00
Roland
65322edf42
Creating video_channel_id_idx index. 2022-09-23 12:31:11 -06:00
dependabot[bot]
d4ed5fa5ce Fixing JS warnings.
Handling all combinations of toggle in light and dark.

Changing CPUUsageProgress colors to be more readable in dark mode.

Moving theme handling into Theme.js.  Importing element wrappers from Theme.js to handle element theming.

Changing video menu to dark mode.

Creating ThemeContext to share Dark Mode setting.

Removing duplicate code for search functions.

Displaying X when downloads are disabled.

Using react context to share settings across components.

Refusing to start workers when WROL Mode is enabled.

Changing warning to debug.

Queuing recurring downloads first.

Adding Video.url index.

Adding special handling for youtube shorts to videos.

Reworking Downloads to have one worker per process.

Testing creating downloads and getting downloaders.

Testing video validation and poster generation.  Testing video statistics.

Increasing inventory test coverage.

Adding pytest.ini to automatically detect async test.

Adding more archive test coverage.

Testing Archive CRUD.

Catching and logging when encountering error during indexing.

Using FileIcon when Archive has no screenshot.

A Video will be assigned to a Channel by its info json first, then to its directory.

Including suffix in File json.

Adding index for File.suffix.

Gathering suffix during indexing.

Passing entire file down to FileIcon.

Moving logging up.

Returning File.associated in file json.

Associated files are last in file search.

Removing duplicate refreshing code by creating new _refresh_directory function.

Changing File.idempotency to a datetime to avoid having to clear idempotency.

Truncating indexes to avoid error with large files.

Don't overwrite video data when info json is empty.

Using already_downloaded for channel videos.

Awaiting refresh in Channel.refresh.

Testing that videos already downloaded aren't downloaded again.

Adding indexes for video/archive referenced paths.

Changing grid width for mobile or computer.

Adding button to refresh specific directories.

Restoring Channel edit link.

Fixing caption path in video_modeler.

Adding ZIP files to mimetype colors.

Using model instead of first file's model.

Fetching File models in batches to avoid issuing a SELECT for every File with a model.

Flushing generated poster file.

Getting Channel in VideoPage, relying on Video.channl_id, falling back to URL's channel_id.

Attaching Video to it's Channel during download.

Using ExternalCardLink for ArchivePage singlefile link.

Validating and indexing videos after download.

Handling glob when file name contains matching characters.

Reworking and testing split_path_stem_and_suffix and glob_shared_stem.

Adding associated video files to DB during upsert.

Adding pending downloads to dashboard.

Hiding status when download is open.

Handling when load is empty.

Adding load statistics to Dashboard.

Limiting concurrent after_refresh functions.

Handling when no onSelect is declared.

Adding ImageRow for FileTable.

Moving image date into Card.Meta.

Using ExternalLink for FileCard.

Using ExternalLink for ImageCard.

Handling when error occurs during refresh of a directory.  Committing during refresh to avoid losing all files if an error occurs.

Removing api health check to avoid spamming logs.

Creating ImageCard to display image posters when they aren't too large.

Removing archive.domain_id not null.

Reworking file refresh to process all files in a single directory.

Bumping version for major file rework.

No longer checking for video corruption.

Adding duration by default to speed up tests.

Using register_indexer for all indexers.

Adding CPU usage progress bar in Welcome.

Modifying TextIndexer to ignore large text files.

Counting all files during refresh.

Adding refresh_logger to clarify logging.  Checking for wrol_mode before refresh functions.

Removing unused idempotency from video/channel tables.

Using magic module to speed up mimetype identification.

All tests pass.

Archive tests pass.

Changing video get endpoint to return 'file' to avoid ambiguity.

Video tests pass.

Simplifying regex filtering.

Using regex to ignore case when filtering.

Converting all channel classes to functions.

Restoring channel refresh and download buttons.

Matching format of Archive and Video pages.

Adding sorting to Archives search.

Reworking video orders for file search.  Using SearchInput for videos search.

Archives can be searched on the Archives page.  Domains can be filtered.

Reworking Archive search to use Files.  Videos, Archives and Global search use the FilesView.

Videos are claimed by their Channel based on the Channel's directory.  Prev and Next video cards are displayed on the VideoPage.

Creating useVideo for reworked VideoWrapper and VideoPage.

Adding Paginator to FilesView.

Moving file refresh button out of manage tab.  Removing Files page tabs.  Limiting refresh to one at a time.

Adding class for card icon to match poster height.

Video and Archives always re-claim their files on refresh.

Archive files are indexed and claimed.

FileView table is compact and striped.

Videos page uses FileView.

Updated navlinks to router v6.

Removing ! character from file names.

Disabling select button when cards are visible.

Adding show/hide button for FileTable selection.

The Archive Page is displayed.

Adding mimetype dropdown.

Upgrading to React Router v6 for useSearchParams.

Validating video during modeling.  Removing screenshot during corruption check because it was very slow.  Updating app to use video files as search results.

Updating video search to return Files.

Video and WROLPi tests pass.

Creating function to refresh files within a channel's directory.

Removing troubleshooting print.  Updating result order in test.

Reworking video_factory to handle a specific video file location.

Returning video model on file search.

Getting session using video_file.

Screenshotting when checking for video corruption.

Reducing ffmpeg calls by gathering duration during poster generation.

Moving video file matching into model.

VideoIndexer is moved into video module.  Video files are validated.

Moving files module into wrolpi in preparation to use it in modules.

Title and contents are indexed for text file.

Files are indexed on refresh by using their indexer.

Zip files are indexed using their contents.

Testing multiple zip files.

Adding unique to file.path.  Checking that the correct indexer is used.

Adding index to file.modification_datetime and file.size.

Handling file info gathering in File.do_index.  Only reading contents of file if file.indexed is False.

Video model is found during file refresh.  A Video record is created that has the associated files.

Videos without associated files are found.

Videos are found using registered video_modeler() rather than in refresh code.

Removing old video columns.

Handling video info json files.

Trying to import each module individually.  Will continue to import modules even if one fails.

Reworking refresh so it performs less DB calls.

Converting file.path to primary key.

file.path is always a pathlib.Path.

Updating video_factory to use new file system, testing it.

Bump sanic from 21.9.1 to 21.12.2 in /docker/archive

Bumps [sanic](https://github.com/sanic-org/sanic) from 21.9.1 to 21.12.2.
- [Release notes](https://github.com/sanic-org/sanic/releases)
- [Changelog](https://github.com/sanic-org/sanic/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/sanic-org/sanic/compare/v21.9.1...v21.12.2)

---
updated-dependencies:
- dependency-name: sanic
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-23 11:25:55 -06:00
Roland
d617c88343
Increasing file size column to bigint to allow for larger files. 2022-06-22 17:36:49 -06:00
Roland
f14a230bc6
RSSDownloader can schedule downloads for other Downloaders using sub_downloader.
RSS download can be submitted in UI.

Checking if entry has link.

A video playlist can be downloaded.

Passing sub_downloader when submitting RSS feed.

Adding Archive downloader button.

Playlists need more info.

Removing expunge, it causes errors elsewhere.  Fixing tests.

Showing errors on recurring downloads.  Ignoring CharacterEncodingOverride bozo error for rss feed.

Changing download button order.

Fixing bug where channel attributes were unavailable after session.  Simplifying video format selection.

Calling validate_video in Video.validate() to avoid duplicate code.

Adding more frequent downloads for RSS Feeds.

DownloadManager skips downloads in it's list.

Renaming get_next_download to more clear calculate_next_download.

Stopping download manager when server is shutting down.

Not shutting down downloads during tests, not all tests use downloads.

Making methods static.
2022-06-08 11:56:42 -06:00
Roland
785ad8274c
DownloadResult.try_again is not necessary.
Removing unused fixture in test.

Simplifying logic after download completes.  Removing unused download sorting method.

ChannelDownloader returns location.

Hiding View link when no location is provided.

Performing video upsert in try/except so any errors are attached to the Download.

Including stderr only.

Including logs when returning Downloader.process_runner.

Warning when extention is replaced.

Providing location link on Downloads page.  Showing error modal on errors that fail.

Handling empty URL strings on import.

Fixing channel download test.

Reworking how Channels schedule Video downloads.

Bumping version for DownloadResult.

Updating tests to use DownloadResult.

VideoDownloader returns a DownloadResult.
2022-05-10 21:20:36 -06:00
Roland
aa203cc102
Moving non-video functions to wrolpi.common. Handling Channel/Downlaod relationship in Channel.update.
Removing default download freqeuncy so channels will not be downloaded without a url and frequency.

Adding channel unique url constraint.

Handling when a Channel's directory may still be relative in DB.

Adding TODOs to remove old favorites.

Removing outdated favorites from config after favorites have been created.

Testing that old-style favorited videos in the config are imported.

Removing channels_by_link.

Using list to store channels in config.

Moving channel import function into lib.

Supporting old channel import with links.

Bumping version for channel.link removal.

Removing all reference to Channel.link in app and api.

Adding relative property to MediaPathType.

Testing that a channel can be downloaded after it has a frequency added.

All tests pass.

Video and Channel tests pass without channel.link.

Removing channel.link usage in channel api.

Changing video Statistics page to About.
2022-04-29 19:09:25 -06:00
Roland
888352c984
Squashed commit of the following:
commit 26bde0fb48d59a86c2fb9f53a7218d49f22d4860
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Mar 8 13:25:56 2022 -0700

    Using osm2pgsql append to preserve other imported maps.

commit c6c52623eaf02e691b1e75dcf39c3a8334ead55a
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Mar 8 08:20:38 2022 -0700

    Using placeholder instead of loader when table is loading.

commit 44fec6490e26603c88a98f6bc70fc18686e17f09
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 20:58:34 2022 -0700

    Simplifying tableRow function.

commit 11b8b57f1fa59556397620f3cad9a33a0237501d
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 20:53:51 2022 -0700

    Removing scientific humanFileSize.

commit 0a09d0a6963afebed7f388aad71bbdd3151d2e9e
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 20:46:22 2022 -0700

    Fixing active loader.

commit 07f4deeac861351d7483ef57278eb2e0941e31fb
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 20:40:44 2022 -0700

    Increasing recommended map file size.

commit af3ac492c81f4c8f84563a448bc272ae9334d32d
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 20:35:13 2022 -0700

    Reworking logic order for simplicity.  Fetching downloads on interval.

commit 3d9d31fcec87ed02ced7f67c34907faa5a3b54e5
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 20:24:39 2022 -0700

    Handling when pbf import status are loading.

commit 3798961a48e7c9eb165ca4e4051f239dbb088d0c
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 19:53:50 2022 -0700

    Fetching import status every 30 seconds.

commit f7a8193b0955d14ead8090aa74ce3c5a681b57ec
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 19:33:17 2022 -0700

    Stopping renderd during installation.

commit 5bf050b59e0a6e391184596f5e6f0737dea1221d
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 19:26:59 2022 -0700

    Removing duplciate serve.

commit 892e3315eeeb5d2516996b901a32a73a66dae078
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 19:22:14 2022 -0700

    Setting bash flags in sub-install script.

commit d5d2acd2cd85e089c78aa85a27c5d966222a1347
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 19:20:19 2022 -0700

    Moving bulk of installation script into separate script so the latest code will be used during initial installation.

commit ab030d8af834a4a8772edd06df360c51e72fbb02
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 18:40:34 2022 -0700

    Removing mod_tile/ajt directory instead of entire contents of mod_tile.  Showing import as relative to media directory.

commit c05199db4ecf56f4d096051b58689457882af12a
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 18:39:59 2022 -0700

    Remvoing marker from map.

commit ba5df4cf62d61b55281277a6254836e973c766ae
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 11:57:41 2022 -0700

    Creating import_map.sh script.

commit 1a16e46130e775efe375adb22917ba12e44855cd
Author: Roland <roland@learningselfreliance.com>
Date:   Mon Mar 7 11:11:01 2022 -0700

    Increasing ram usage during import to avoid hanging import.

commit 0f5ce1a9f806f1c0878dd804b04587b6340ffd16
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 19:19:47 2022 -0700

    Increasing map_file.size size.

commit eb6e01512af59fb4a18ab15643c7bc61f3d48e53
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 18:55:55 2022 -0700

    Copying leaflet js/css files into docker container and to Apache directory so they will be available offline.

commit 5b687324e119f021673f42e8d079b83034b70d53
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 18:36:06 2022 -0700

    Matching tab height on Map page.

commit 8031786d672e18366e9a2eb2d7bdfcd8611d27f3
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 18:34:20 2022 -0700

    Shrinking HelpPopup Icon.

commit 5ea94488e43bbc5c414e61c9173a3cfa908c7104
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 18:31:30 2022 -0700

    Testing if no PBFs are requested to be imported.

commit c4e86be1faed78bd3af1f3487471c01a1958a433
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 18:18:48 2022 -0700

    Using Icon instead of Button for HelpPopup.

commit 51d94f6d85885803813544fea16a56372babaa82
Author: Roland <roland@learningselfreliance.com>
Date:   Sat Mar 5 18:07:03 2022 -0700

    PBF files are sorted by the api.

commit 40f3a7ff04a83db0153f64a88a37f1d56ce043aa
Author: Roland <roland@learningselfreliance.com>
Date:   Fri Mar 4 18:35:51 2022 -0700

    Adding help popup.

commit 1da8d9fa403ec2a3b3c16c11a3937de37c895a35
Author: Roland <roland@learningselfreliance.com>
Date:   Fri Mar 4 09:41:01 2022 -0700

    Moving message up.  Clearing cached tiles after import.

commit 3192adf95192554c194c9e0a8aeb149822462682
Author: Roland <roland@learningselfreliance.com>
Date:   Thu Mar 3 21:37:13 2022 -0700

    Removing osm2pgsql from api docker because it seems to be unsupported.

commit 0e10522f3cdc7edd374c14f230bb2dbd68ab1a94
Author: Roland <roland@learningselfreliance.com>
Date:   Thu Mar 3 21:36:39 2022 -0700

    Disabling import button for docker.  Adding download instructions.

commit e1d8f55d159fb4d4ef2ece859ef7cc1059885791
Author: Roland <roland@learningselfreliance.com>
Date:   Thu Mar 3 21:14:55 2022 -0700

    Map files can be imported on RPi.

commit 3128530e09968430a71b27cd2e75a567f85a708b
Author: Roland <roland@learningselfreliance.com>
Date:   Thu Mar 3 15:31:54 2022 -0700

    Adding shm_size to increase size to render large map tiles.

commit f3cf80fae0ce46f44ea9ac5033f68a32d0197ae3
Author: Roland <roland@learningselfreliance.com>
Date:   Wed Mar 2 10:39:18 2022 -0700

    Can get status of map imports.

commit 3b7bbce004cd55a5d89980c2c09124b72c450bc6
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Mar 1 20:51:13 2022 -0700

    Adding map link to 50x.html.  Swapping links using hostname to make it easier on the user.

commit a8b6b60fe182233f40c2f1ecd825636c3d37fd23
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Mar 1 12:01:50 2022 -0700

    Moving Inventory link to end.

commit c5346b8dc2ff83262d6fd153130912809dfd7574
Author: Roland <roland@learningselfreliance.com>
Date:   Tue Mar 1 12:00:47 2022 -0700

    Squashed commit of the following:

    commit 39f0b8f3fc049fffd140d551cd91d7d9dba6d416
    Author: Roland <roland@learningselfreliance.com>
    Date:   Tue Mar 1 12:00:13 2022 -0700

        Starting map refresh function.

    commit 2fac9ddf07e378389ddd85ca972f33b3323d30ac
    Author: Roland <roland@learningselfreliance.com>
    Date:   Mon Feb 28 16:43:47 2022 -0700

        Updating readme with map requirements.

    commit 8325e1ea89537128329e313ab4b5588f8809d1c9
    Author: Roland <roland@learningselfreliance.com>
    Date:   Mon Feb 28 16:05:35 2022 -0700

        Fixing order of map db init scripts.  Creating index.html to display map.  Using :8084 index file to display the map.

    commit 8f81539dd9d25fc26607f472c156d41ddcfc11f5
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 19:17:14 2022 -0700

        Installing apache2.

    commit c7aaef07f8291070aeaf5e7df553e34aca0719ec
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 18:41:07 2022 -0700

        Always configure postgres extensions.  Initialize gis with dc.

    commit aa725ed4b8ec46d17accebd96dd41508ea2d9caf
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 15:48:20 2022 -0700

        Map is displayed as iframe.

    commit de9bb92dcaf44996e7727adce7774f01895ddf52
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 12:16:56 2022 -0700

        Running script in openstreetmap directory.

    commit f277388095d2221aa6407643ab655f26dc349eb7
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 12:06:41 2022 -0700

        Bumping version for map.

    commit afd4aaeffb2aced2dc5c1a8f94038885c8287265
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 11:59:31 2022 -0700

        Checking that the installation script is on an rpi.

    commit a799382ceac3c201fc01d41fd63e37607282831a
    Author: Roland <roland@learningselfreliance.com>
    Date:   Sun Feb 27 11:49:21 2022 -0700

        Only running mod_tile install once.

    commit 0b98bb19cd4d86c9630a863cab4eb46410d1c9d1
    Author: Roland <roland@learningselfreliance.com>
    Date:   Fri Feb 25 11:31:21 2022 -0700

        Building out map install script.

    commit 7c799775c11df573fa6cd65451adea28c0832cb0
    Author: Roland <roland@learningselfreliance.com>
    Date:   Thu Feb 24 12:16:53 2022 -0700

        Creating OSM installation script.

    commit d3b93a3ee97d1ae4a00b6a0423ed59ea8b897ccd
    Author: Roland <roland@learningselfreliance.com>
    Date:   Tue Feb 22 15:39:04 2022 -0700

        Removing unused map module.
2022-03-08 13:26:03 -07:00
Roland
812369fdb3
Refresh files removed files that were deleted. 2022-02-02 08:50:03 -07:00
Roland
643c86e9a4
Files can be searched. 2022-02-01 14:11:14 -07:00
Roland
a5e8ef16b6
Files are indexed by the words in their stem. Files can be searched via API. 2022-02-01 12:22:29 -07:00
Roland
0315def174
Videos can be sorted by their modification date. 2022-01-25 08:51:55 -07:00
lrnselfreliance
86893ef7f1
Feature/video validate (#74)
* Videos are marked as validated so they will not be tried again.  All previous bulk methods are combined into one.

* Passing json to fixture factory to avoid writing the files in every test.

* Adding log to explain if refresh is slow.

* Generating video poster if one cannot be found.

* Generating video poster if no poster can be found.

* Videos without a video file (yet to be downloaded) are not validated.

* Video title is extracted from the video file name as a last resort.

* Keeping existing view count because it may be from the channel, which will be more recent.

* Downloads should only be downloaded after they are due, not the same day.

* Setting manager on new downloads.

* Checking if downloading is disabled in loop.

* Adding log when refresh is complete.

* Not generating posters by default.  This is to avoid deleting files.

* Not converting posters unless channel has that enabled.

* Not logging on valid because all posters will be checked.

* Validating videos in upsert.

* Using video name parser in upsert.

* Using validate_video in upsert so that data is only read once from disk.

* Attaching manager in download_manager.get_download.

* Adding 22 to preferred video format for odd 720p resolution.

Signed-off-by: Roland <roland@learningselfreliance.com>
2022-01-14 15:54:51 -07:00
Roland
5133bc1c8c
Adding index to Video.source_id to speed up view count updates. 2022-01-10 08:30:10 -07:00
Roland
a96d6f97f3
A Channel must be refreshed before download. 2022-01-07 14:41:28 -07:00
Roland
ad1b8b805e
Applying textsearch coalesce fix to archive.textsearch. 2022-01-04 11:14:15 -07:00
Roland
6344e41db3
Fixing downgrade for latest patch. 2022-01-04 11:09:02 -07:00