Commit graph

77 commits

Author SHA1 Message Date
Roland
5b29144d70 Testing that nearly-empty configs are ignored on import. Adding setting to disable automatic upgrade checking. 2025-12-06 15:11:37 -07:00
Roland
79e0700609 Fixing local imports. 2025-12-06 15:11:37 -07:00
Roland
fa9fb9ab08 Not checking for updates when WROL Mode is enabled. 2025-12-06 15:11:37 -07:00
Roland
1d8738ace2 Running alembic check in cicd. 2025-12-06 15:11:37 -07:00
Roland
a1e0fb4e92 Creating a new Collection model for use in organizing FileGroups.
Replacing Channels and Domains with Collection.

Moving downloading relationships from Channel to Collection.

Created domains.yaml config to manage tagged domain collections.

Adding turn-key upgrade solution in the UI.
2025-12-06 15:11:37 -07:00
Roland
db956faedf
Not importing configs unless WROLPi config was imported, and WROL mode is disabled. 2025-08-11 13:05:51 -06:00
Roland
08a67eaf74
Fixing starting downloads on startup. 2024-12-26 14:45:10 -07:00
Roland
7d6b7114a7
Adding TRACE level for logging beyond debug. Reworking async_client for tests, not all tests need the App to handle signals, so any test that needs signals can use the new await_switches fixture which primes the app. 2024-12-11 11:52:33 -07:00
Roland
b7d163e7ff
Config files are initialized on startup, if they are missing. 2024-11-28 18:13:43 -07:00
Roland
5621d47309
Video downloads have many more options. Downloads are migrated.
Storing video settings in session storage to help user start downloads consistently.

Applying channel tag name to new channels when download videos or channels.

Initializing map tiles when importing and resetting map.
2024-11-28 09:48:57 -07:00
Roland
4905235083
Trimming file names more because yt-dlp uses huge temporary names while downloading. 2024-10-16 18:57:10 -06:00
Roland
584fdadcb7
Getting file name from yt-dlp error when video file name is too long, trimming that long file name. 2024-10-16 18:56:56 -06:00
Roland
d34301a085
Configs are imported manually in main. Configs now have dump/save/import methods. Configs will refuse to save if they have not been imported. 2024-10-16 18:56:44 -06:00
Roland
d54735c4bb
Configs can be saved in background using switches. 2024-10-16 18:56:38 -06:00
Roland
317f3f3dfb
Top processes are displayed on Status page. Nice of renderd is lowered to avoid clobbering system at startup. 2024-10-16 18:56:27 -06:00
Roland
47b4da1cfe
Adding warning to Dashboard when Internet is down. 2024-10-16 18:56:26 -06:00
Roland
45167e4f32
A Channel can be tagged with a single Tag. The Channel can be moved into a Tag Directory according to the config. 2024-10-16 18:56:19 -06:00
Roland
c839acb270
Converting Tag methods to use tag id or name. 2024-10-16 18:56:01 -06:00
Roland
a6647cc8f7
Adding FileGroup.censored when video can no longer be downloaded. 2024-10-16 18:55:43 -06:00
Roland
814b3c3662
Logging when periodic_check_db_is_up experiences error. 2024-06-12 16:39:04 -06:00
Roland
0fbbd36297
Creating perpetual funciton which checks if the internet is accessible. This is used to prevent downloads from starting when no internet exists. 2024-05-31 23:54:00 -06:00
Roland
0e3bbf8b08
Enabling comments downloading now that new release of yt-dlp supports comments again. 2024-05-28 19:16:38 -06:00
Roland
02d02c0836
Adding more info to log level change log. 2024-05-19 17:32:37 -06:00
Roland
2c5da4b12d
Warning when server is shutting down. 2024-05-19 15:46:54 -06:00
Roland
a527b0091d
Changing default log level back to INFO. 2024-05-08 18:05:09 -06:00
Roland
63381c1d77
Created download retry button. 2024-05-08 17:05:12 -06:00
Roland
8ad653fc40
Testing that api can start. 2024-05-07 10:39:00 -06:00
Roland
a65fa1c6fb
Removing start_initialize_flags because flags are initted in perpetual_check_db_is_up_worker. Logging when flags are initialized. 2024-05-07 09:42:50 -06:00
Roland
f0e4302de8
Restoring main.py as method to start WROLPi API because it is required to upgrade docker database. 2024-05-02 10:56:43 -06:00
Roland
2924867f55
Bumping single-file-cli version for all systems. 2024-05-02 09:09:21 -06:00
Roland
023dfc68e0
Verifying that root_api is imported before starting WROLPi API. 2024-05-01 18:59:25 -06:00
Roland
41f3969330
Modifying launch of API to use 'sanic' executable. This was done because startup is much faster. 2024-05-01 18:42:52 -06:00
Roland
58de8c7c5b
Upgrading Sanic and yt-dlp. Major changes occurred because Sanic changed how multiprocessing works.
Python 3.11 is now required, old versions were causing unpredictability in tests.  (Sanic does not yet
support 3.12)

Sanic has been upgraded to 23.6.0, which is the latest version that avoids this bug:
 https://github.com/sanic-org/sanic/issues/2921

New strategy for multiprocessing is to create all multiprocessing tools in one process, then fork
to other processes.  The previous strategy was to declare multiprocessing tools at the top of every
file, or wherever they were needed at import/creation.  Now all multiprocessing tools are attached to the
app.shared_ctx.  This means `api_app` is imported in many, many places.

This forced a change in how the DownloadManager works.  Previously, it would continually run
download workers which would pull downloads from a multiprocessing.Queue.  Now, a single worker checks
for new downloads and sends a Sanic signal.

Flags have been reworked to use the `api_app`.  I removed the `which` flag functionality because the
`which` are called at import and needed their own multiprocessing.Event.
2024-04-30 10:55:25 -06:00
Roland
b05b2e7a6d
Allowing maintainer to change special directories. 2024-03-09 16:51:26 -07:00
Roland
00fe799b8f
User can select from/to years when searching files. Indexing doc and docx files. 2024-03-07 12:12:48 -07:00
Roland
758ff8e810
Creating background process that fetches comments for Videos that do not yet have comments.
Displaying star for Uploader's comments, heart for favorited comments.

Adding wrolpi help files desktop shortcut to rpi image.

Using zim file names as FileGroup.title.

Removing accidental output commit.
2024-03-01 17:15:56 -07:00
Roland
7611cf8ea1
Converted ArchiveDownloader to use Downloader.process_runner so archiving can be canceled.
Moving Archive download functions to ArchiveDownloader.

Wrapping all do_download with cancel_wrapper which allows canceling of downloads.
2023-09-12 17:32:29 -06:00
Roland
95e870d659
Adding range input in settings so the user can change the API log level. 2023-09-08 17:10:15 -06:00
Roland
03a2df217d
Zim/Kiwix (Wikipedia) feature
* A new zim module has been added, this module allows several things to interact with Zim files.
  * Allow searching of Zim files using libzim.
  * Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
  * Creation of a new zim docker container which allows viewing of the Zim file contents.

* This commit also adds move/rename/mkdir features in the File Browser.
  * File tags are preserved when moving/renaming.
2023-07-10 21:30:40 -06:00
Roland
d82d5b67cb
Using db_up flag directly. 2023-05-10 22:44:51 -06:00
Roland
8e3c92b589
Handling when starting hotspot may keep WROLPi API from starting. 2023-05-08 11:06:12 -06:00
Roland
655e944960
A function can wait for a Flag to be set. 2023-03-29 19:55:10 -06:00
Roland
c01d62cd28
Tags config is imported on startup. 2023-03-29 19:30:33 -06:00
Roland
5615ce542d
Making example label a tag.
Tags can be created and deleted in UI.

Adding tag_file using id to avoid session error.

Using UTC in test.

Matching tag_names style in test.

Tag icon is displayed on file rows.

Archives, Videos and Files can be filtered by their tags.

Tags can be added and removed from UI.

Tags are displayed in VideoPlayer.

The tags of a file are returned in it's JSON.

Video favorites are converted to Favorite tag on channel config import.

Updating tests to match frontend usage.

PDF creator and modification time are displayed.

Reworking file search to use Video.__json__ for complex video relations.

Archives can be viewed.

Removing ebook.title.

Reworking alembic migration to be an upgrade rather than a replace.

Skipping test on circleci.

Clearing out video caption index only if extract_subtitles if off.

Channel tests pass.

Some video tests pass.

Missing videos can be found.  A video can be downloaded.

Orphaned files can be found.

Videos can be tagged.

Videos can be searched.

Videos can be modeled.

Converting some tests to use FileGroup.primary_path.

Adding archive and domains to alembic migration.

Testing cascading Archive titles.

Archives now uses FileGroup.

archive.lib tests pass.

Archive is partially converted to FileGroup.

Calculating FileGroup.primary file on refresh, rather than on select.

wrolpi tests pass.

Using FilesType to convert between str/pathlib.Path for FileGroup.files.

Using list of files in FileGroup.files instead of dictionary.

Storing files in dedicated FileGroup.files to avoid overwriting them.

Adding comment explaining CASE.

Not overwriting FileGroup.data on refresh if no change has been detected.

Removing File model in favor of FileGroup.data.

Adding typing to Files methods.

Files are indexed using the primary mimetype of the FileGroup.

files test_lib tests pass.

Removing file_group_file in favor of simple one-to-many relationship.

PDF files are indexed during refresh.

Adding Tag and TagFile to tag files with specific tags.

Only iterating when full_stem_ids has contents.

Testing that FileGroups are related to their Files.

Files and file_groups are created and linked.

Bump webpack from 5.75.0 to 5.76.1 in /app (#160)

Bumps [webpack](https://github.com/webpack/webpack) from 5.75.0 to 5.76.1.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.75.0...v5.76.1)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-25 17:54:41 -06:00
Roland
eb0e2e9d80
Restoring before startup function accidental commit. 2023-02-09 10:27:38 -07:00
Roland
163b14c294
API will start without DB. 2023-01-31 17:11:31 -07:00
Roland
636ce76777
Disabling download manager when refresh has not completed. 2023-01-04 11:51:11 -07:00
Roland
1161476c61
Handling if reader.metadata is None. Not sending ready event. 2022-12-23 16:36:08 -07:00
Roland
e0c6a2e40e
Using flag correctly. 2022-12-23 16:12:19 -07:00
Roland
93f0e7494f
Creating Event and Flag systems. Events are used to send and display a toast to the App. Flags are used to signal if some action has been performed, or if a package is installed. 2022-12-23 16:04:09 -07:00