Commit graph

73 commits

Author SHA1 Message Date
Roland
1ed0d63bac Creating Generate Screenshot button for Archives which are missing a screenshot. 2025-12-06 15:11:37 -07:00
Roland
88a0a15bef
Fixing syntax to avoid docker compose warning.
Some checks failed
CodeQL / Analyze (push) Has been cancelled
2025-08-20 09:45:52 -06:00
Roland
7bc261a8a3 Modifying https_proxy docker container so that upstream host and port can be configured. 2025-08-18 10:28:53 -06:00
Roland
f09e9ffb2f
Installing ping in api docker container to monitor db status.
Some checks are pending
CodeQL / Analyze (push) Waiting to run
2025-08-17 11:27:14 -06:00
Roland
e3ad88bfc3
Bumping api's python docker version because of debian 13 release. 2025-08-17 10:43:04 -06:00
Roland
5bd71f9664
Adding timeouts to archive docker container. 2025-05-14 17:09:53 -06:00
Roland
b3e91fcff7
Adding link to help for cookies feature. Updating help with latest cookies docs. 2025-05-05 17:02:32 -06:00
Roland
506a349276
Bumping WROLPi Help version. 2025-02-04 17:55:06 -07:00
Roland
531ec3d65e
Creating new /api/archive/upload which accepts a file from the SingleFile browser extension and converts it to an Archive. 2025-01-30 18:41:32 -07:00
Roland
2b811d1a51
Removing extra meta4 files after aria2c download. 2025-01-10 12:12:39 -07:00
Roland
d8b3db9d63
Using iostat to detect busy disk issues, reporting them in the UI. Detecting under-voltage/over-current issues using dmesg and reporting them in the UI. 2025-01-10 11:33:21 -07:00
Roland
26f4d2f55d
Bumping wrolpi-help commit. 2025-01-04 10:54:28 -07:00
Roland
8e4d173791
Converting map blob to use "custom" postgres dump format so restore can use multiple threads. Renaming map db blob from gis-map.dump.gz to map-db-gis.dump. 2025-01-02 12:04:03 -07:00
Roland
a068273beb
Updating wrolpi-help commit. Forcing re-install of mkdocs because of navbar bug. 2024-10-16 18:56:07 -06:00
Roland
8cf07c2ffb
Bumping Help version. 2024-10-16 18:55:40 -06:00
Roland
00c4e6ece0
nginx needs to serve downloads from the media directory. 2024-10-02 17:10:05 -06:00
Roland
e18b1b0552
Removing duplicate CORS header for Zim docker container. 2024-06-18 15:35:22 -06:00
Roland
bd431165e5
Displaying helpful error when iframes do not load. 2024-06-17 10:40:30 -06:00
Roland
38de2f9b40
Moving HTTP kiwix/help services to 9085/9086 ports. Nginx will serve HTTPS proxy for kiwix/help services at the old ports. Modifying apache2 to use HTTPS to serve map.
Using new https_proxy docker container to serve HTTPS for zim, map and help.
2024-06-12 21:38:51 -06:00
Roland
71cf292728
Creating cache directory in API Docker container for yt-dlp. 2024-05-18 19:13:41 -06:00
Roland
ad8433f9f4
Bumping WROLPi Help version. 2024-05-14 20:34:22 -06:00
Roland
86355b664d
Using older version of single-file-cli to avoid error. 2024-05-14 20:29:57 -06:00
Roland
517cabf105
Bumping wrolpi-help version. 2024-05-08 20:08:32 -06:00
Roland
a527b0091d
Changing default log level back to INFO. 2024-05-08 18:05:09 -06:00
Roland
f0e4302de8
Restoring main.py as method to start WROLPi API because it is required to upgrade docker database. 2024-05-02 10:56:43 -06:00
Roland
2924867f55
Bumping single-file-cli version for all systems. 2024-05-02 09:09:21 -06:00
Roland
41f3969330
Modifying launch of API to use 'sanic' executable. This was done because startup is much faster. 2024-05-01 18:42:52 -06:00
Roland
58de8c7c5b
Upgrading Sanic and yt-dlp. Major changes occurred because Sanic changed how multiprocessing works.
Python 3.11 is now required, old versions were causing unpredictability in tests.  (Sanic does not yet
support 3.12)

Sanic has been upgraded to 23.6.0, which is the latest version that avoids this bug:
 https://github.com/sanic-org/sanic/issues/2921

New strategy for multiprocessing is to create all multiprocessing tools in one process, then fork
to other processes.  The previous strategy was to declare multiprocessing tools at the top of every
file, or wherever they were needed at import/creation.  Now all multiprocessing tools are attached to the
app.shared_ctx.  This means `api_app` is imported in many, many places.

This forced a change in how the DownloadManager works.  Previously, it would continually run
download workers which would pull downloads from a multiprocessing.Queue.  Now, a single worker checks
for new downloads and sends a Sanic signal.

Flags have been reworked to use the `api_app`.  I removed the `which` flag functionality because the
`which` are called at import and needed their own multiprocessing.Event.
2024-04-30 10:55:25 -06:00
Roland
ba2dd6dd11
Many fixes split out from the feature/unified-query branch. Installing diagnostic tools in the built images. Displaying video comment statistics. Echoing error when install.sh fails. 2024-04-14 13:13:30 -06:00
Roland
1c5b277f24
Bumping help to latest. 2024-03-10 16:55:38 -06:00
Roland
00fe799b8f
User can select from/to years when searching files. Indexing doc and docx files. 2024-03-07 12:12:48 -07:00
Roland
a90c37a1df
Bumping wrolpi-help commit. 2024-02-19 11:00:23 -07:00
Roland
81419b8353
Bumping help version. 2023-11-30 10:17:06 -07:00
Roland
6dfed2e6c1
Require Python 3.11 for API. 2023-11-29 21:16:26 -07:00
Roland
73cb142ae1
Reverting to older singlefile container to keep archive screenshot ratio. Using FileGroup.url on ArchivePage. 2023-10-31 10:20:45 -06:00
Roland
36231de07f
Bumping help version. 2023-10-29 18:21:07 -06:00
Roland
e6d8c5ce99
Creating Help link in top navbar. Adding wrolpi-help as submodule and docker container which displays the WROLPi Help repo. 2023-10-15 17:18:33 -06:00
Roland
8fbf07d502
Monitor Zim library XML to automatically reload. 2023-08-26 19:37:29 -06:00
Roland
097c3143e1
Taking screenshot of singlefile during archiving, then URL if that fails. 2023-07-24 17:17:40 -06:00
Roland
03a2df217d
Zim/Kiwix (Wikipedia) feature
* A new zim module has been added, this module allows several things to interact with Zim files.
  * Allow searching of Zim files using libzim.
  * Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
  * Creation of a new zim docker container which allows viewing of the Zim file contents.

* This commit also adds move/rename/mkdir features in the File Browser.
  * File tags are preserved when moving/renaming.
2023-07-10 21:30:40 -06:00
Roland
ccb28db95c
Enabling Python optimization when running in Docker and systemd. 2023-05-23 16:31:10 -06:00
Roland
8d0f30566f Updating Sanic requirement for Archive container. 2023-04-28 13:56:21 -06:00
Roland
46cf28dee8
Restoring Archive docker container. Allowing readability to fail. 2023-04-01 15:50:11 -06:00
Roland
81d80238be
Fixing changes before merge.
Moving FilesRefreshButton and DirectoryRefreshButton out of browser file.

Moving file fetching tino FileBrowser.

Removing sub-folders from Files display when any parent folder is clicked.

Using closed captioning for vtt/srt file icon.

Using my own react-semantic-toasts-2 to use toasts in Node 18.

Fixing bug where more than the last page was displayed.

Upgrading yt-dlp.

Fixing sub-folder file selection.

Adding rowKey to SortableTable to give React a key for each row.

Renaming key to path.

Removing duplicate urls.

Unifying file modals.

Displaying modals for image and text files.

Handling refresh loading in FileBrowser.  Creating refreshing_directory flag.

Removing accidental commit.

Sorting by url_count.

Using SortableTable on Domains page.

Using SortableTable for Channels page.

Created new FileBrowser which allows browsing of files in the media directory.

Using Node 18 in pi-gen.

Creating new FileBrowser to replace outdated react-keyed-file-browser.

Returning mimetype with file list.

Do not display 'Do not download' on Channel download in Dashboard.

Allowing null channel download frequency so user can stop downloading a channel.

Bumping version for Node 18.

Using my own semantic-ui-offline to avoid pulling Google fonts.

Upgrading app docker container to use Node 18.  This requires the removal of several npm packages.  File browser will require re-write.  It now pulls fonts from Google, this will need to be fixed.

Testing DownloadsToggle click.

Testing trinary states of DownloadsToggle.

Installing @testing-library/user-event for testing App.
2023-02-08 21:18:51 -07:00
Roland
e81c029263
Not dumping comments. 2023-01-16 19:34:07 -07:00
Roland
838b4dd185
Using pi-gen to create wrolpi raspberry image. 2023-01-16 11:05:27 -07:00
Roland
12c3504291
Updating osm-map-dumper Readme with better instructions. 2023-01-12 14:52:29 -07:00
Roland
90e7d826c8
Dumping map in tar format. 2023-01-11 20:36:19 -07:00
Roland
69fab7eee8
Adding Readme to osm-map-dumper. 2023-01-11 19:25:36 -07:00
Roland
95e1d1d270
Converting map_dumper to use overv/openstreetmap-tile-server:2.2.0. 2023-01-11 18:53:03 -07:00