The archive service is PID 1 in its container and never reaps orphans.
check_output kills each command's process group, so every archive leaves
chrome, cat, and node grandchildren behind as zombies. A long-running
container accumulated hundreds of them, counting against its PID limit.
init: true runs tini as PID 1, matching the web container.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Caddy is PID 1 in the web container and reaps nothing. The healthcheck's
busybox `wget --spider https://...` forks an `ssl_client` helper that is
orphaned to caddy and stays a zombie, one per check. `init: true` runs
tini as PID 1 so they are reaped.
The zim healthcheck calls curl, which the debian:13 kiwix image does not
install, so every check exited 127. The help healthcheck probed port
8086, which is Caddy's HTTPS proxy on the web container; mkdocs serves
on port 80 inside its own container.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* ui: a preview map that waited for a container it could never see
Both map preview modals -- the per-region eye button and "View All Regions"
-- opened to an empty box. Each renders a div into a Modal and hands it to
MapLibre from an effect gated on `open`, reading the node out of a `useRef`:
if (!open || !mapContainer.current) return;
Mantine's Modal mounts its body one commit after `opened` flips true, so on
the render that opens the modal the ref is still null. The effect returns,
nothing it depends on changes again, and it never runs a second time. No
console error, no rejected promise, no canvas -- just an empty div.
Semantic UI's Portal mounted the body in the same commit that opened it, so
the ref was attached by the time effects ran. The effect bodies are
byte-identical across that migration; the modal library is the only variable,
confirmed against a device still running the older build where both previews
render.
A callback ref instead, so the effect re-runs when the node arrives and this
stops depending on any modal's mounting schedule. The async source lookup
also gets a cancelled flag: it could resolve after the cleanup ran, building
a map nothing would ever remove and leaking a WebGL context per open.
The tests drive the real page and assert a map was built against a node that
is in the document -- the outcome, rather than the mechanism that quietly
changed underneath it. Seven of the eight fail on the code before this.
* build: docker had no world underneath its map regions
Caddy serves /blobs/map-overview.pmtiles out of /opt/wrolpi-blobs, and that
blob is the world at zoom 0-6 -- what a regional extract sits on top of.
Without it a user who subscribes to a few regions gets those regions floating
on an empty background everywhere their data does not reach. Only a full
planet file, at 134GB, makes it unnecessary.
Raspberry Pi and Debian installs get the blob from pi-gen, the live-build
config, and upgrade.sh. Docker got nothing: the compose file bind-mounted a
git-ignored host directory over /opt/wrolpi-blobs, and a bind mount replaces
the image's directory rather than merging with it, so the path existed and was
empty on every Docker WROLPi. Verified against a running deployment, where
that URL returned 404 while the same URL on hardware returned 206.
So the web image fetches the blob at build time, as the other three builders
do, and the mount that would hide it is gone. `curl -f` so an HTTP error
fails the layer -- an image that builds without the blob is this same bug,
except silent at build time too.
The tests read the deployment files rather than a built container: building
to check costs a CDN download and several minutes, and both things that broke
-- nothing fetches it, a mount hides it -- are visible in the source.
* ui: cancel the load handler too, and make the cancel test falsifiable
Review follow-ups on the preview modals.
The `cancelled` flag was checked before constructing the map but not inside
the `load` handler it installs, so cleanup landing between the two left that
body drawing into a map `remove()` had already torn down. Guarded, mirroring
the `destroyed` pattern in MapViewer.js. The source promise also gets a
`.catch`: the helper cannot reject today, but nothing says it never will, and
an unhandled rejection here would be as quiet as the bug this all started as.
Two test corrections, both cases of an assertion claiming more than it read:
The region-drawing test spied on `addSource` AFTER construction and then
re-fired `load` by hand, so its count came entirely from that manual call and
said nothing about the production path. FakeMap now records the sources it is
given, and the assertion reads what the component's own `load` drew -- each
region once, in order, and none for the entry with no bbox.
The new cancel test passed with the guard deleted, which is worse than not
having it: Mantine had not mounted the modal body when it closed, so the
effect had never run and there was nothing to cancel. It now waits for the
effect to reach the source lookup before closing, and ships beside a control
that releases the same held promise WITHOUT closing and requires a map. The
control is what makes the zero meaningful; verified that removing the guard
fails the cancel test while the control still passes.
react-scripts lints in-process on every incremental rebuild. Each save
pays for it, and the result is buried under a few hundred warnings the
repo has carried for years -- `no-useless-computed-key` on the wire-gauge
tables, `react-hooks/exhaustive-deps` across most of Common.js. Finding
"Compiled successfully" in that takes a scroll.
This container only serves; it never builds. Linting still happens where
it is acted on: `npm run build` and CI both run it, and both fail on a
real error.
NOT a memory fix, though that is why I reached for it -- the dev server
was OOM-killed three times in an afternoon and I claimed this would be the
biggest saving. It is not. Measured both ways with the same procedure
(recreate, load a page, wait for the typecheck, read `docker stats`) the
container settles at 3.5GB with linting and 3.0GB then 3.7GB without it.
The spread between two runs of the SAME configuration is wider than the
spread between configurations, so the effect is below what this can
measure. Raising Docker Desktop's VM limit from 8GB to 16GB is what
actually stopped the kills.
Kept anyway, on the time and the noise, which are not in doubt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The database now lives at <media>/config/wrolpi.db so a library (files,
configs, AND database) travels with the drive between WROLPi systems.
Core:
- wrolpi/db.py: lazy SQLite engine factory (WAL, busy_timeout=30s,
foreign_keys=ON); get_db_curs yields a plain sqlite3.Cursor; native
sqlite3 paramstyle everywhere (no compatibility shims); global sqlite3
adapters keep raw-SQL datetime formats identical to the ORM's.
- wrolpi/fts.py: FTS5 external-content search replacing tsvector/GIN.
Injection-proof websearch->FTS5 translator, bm25(10,4,2,1) reproducing
A/B/C/D weighting, fast(abc)/deep column filtering, snippet headlines.
Stemming contract pinned by dedicated tests (watch/watches/watched/
watching are interchangeable).
- wrolpi/schema_ddl.py: SQLite triggers shared verbatim by the alembic
baseline and test DBs; fixes the channel video_count DELETE/UPDATE
bugs and a latent total_size bug (it used the video id as channel id).
- wrolpi/db_bootstrap.py: startup auto-create/migrate; refuses databases
written by a newer WROLPi (db_version_mismatch); SQLite 3.40 floor;
network-filesystem refusal; never creates/migrates/trusts a database
beneath an unmounted production media directory (shadow-DB guard).
- Alembic: 85 Postgres-era migrations squashed into one baseline;
revision ids are dates (2026_07_10_1350) so filenames self-sort and
alembic_version says when a schema was created; render_as_batch=True.
- All raw Postgres SQL rewritten (ANY->json_each IN, array_agg @> ->
HAVING COUNT(DISTINCT), unnest->executemany/UPDATE..FROM,
generate_series histograms->strftime GROUP BY, DISTINCT ON->
row_number(), writable CTEs->two statements, jsonb_*->json_*, explicit
NULLS FIRST/LAST, LIMIT-before-OFFSET). Simple statements converted
to SQLAlchemy instead (flag upsert, download retry/summary/tag checks,
channel-by-tag estimate, Directory delete).
- Config imports count skipped entries; background dumps are blocked
after a lossy import so partial imports can never overwrite YAML
configs (they are the sole source of truth for rebuilds now).
- Search tolerates explicit JSON nulls for limit/offset (Postgres
treated OFFSET NULL as 0; SQLite errors).
- file_group.id/doc_section.id render as INTEGER PRIMARY KEY on SQLite
(BIGINT is not a rowid alias; FTS5 content_rowid requires it).
Ops teardown: docker-compose db service, CI postgres sidecars,
initialize_api_db.sh, controller postgresql service, psycopg2-binary all
removed. upgrade.sh stops/disables postgresql (data is NOT deleted;
`systemctl enable --now postgresql` restores it); systemd units no
longer Want postgresql.service.
Existing-user upgrade: first boot creates+migrates the DB, configs
import (tags recreate their FileGroups from disk), a refresh rebuilds
the library index. QA'd on the 10.0.0.8 RPi: rebuild matched the
Postgres baseline exactly; survived an uncontrolled hard reboot with
clean integrity checks.
Accepted behavior changes: FTS5 has no stopwords; bm25 length
normalization shifts some rank orders; fast-path d_headline is
un-highlighted; all-negative queries match nothing.
Deferred: Debian/trixie package lists (incl. sqlite3 CLI), Cypress
re-baseline, VACUUM INTO snapshots, exFAT/NTFS warning, auto-kicked
rebuild refresh with progress UX.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Replace nginx with Caddy as the web server for both Docker and native
(RPi/Debian) deployments. Caddy handles HTTPS termination, reverse
proxying to all services (API, React app, Controller, Kiwix, Help, Map),
and serves the media/download file browsers.
- Generate a Root CA and signed leaf certificates for HTTPS. The Root CA
uses Name Constraints to restrict signing to local addresses only
(.local, localhost, and RFC 1918 private IP ranges), mitigating MITM
risk if the CA key is compromised.
- Add an HTTP landing page that guides users through one-time CA
certificate installation on iOS, Android, macOS, Windows, Linux, and
Firefox, with auto-redirect when the cert is already trusted.
- Add trailing-slash redirects for /controller, /docs, /media, and
/download so bare paths don't fall through to the React catch-all.
- Add a Root CA download link on the Settings page under Configs.
- Update help.sh diagnostics for IPv6 listener detection and HTTP/2
status codes.
- Remove all nginx config files and the https_proxy Docker container.
- Bump wrolpi-help submodule with certificate documentation.
* Added encrypted cookies feature for use with yt-dlp. Decrypted cookies never touch the disk.
* Adding sleep between all downloads to prevent rapid requests.
* Channels are fully refreshed before downloading.
* Removing superseded video browser profile.
Python 3.11 is now required, old versions were causing unpredictability in tests. (Sanic does not yet
support 3.12)
Sanic has been upgraded to 23.6.0, which is the latest version that avoids this bug:
https://github.com/sanic-org/sanic/issues/2921
New strategy for multiprocessing is to create all multiprocessing tools in one process, then fork
to other processes. The previous strategy was to declare multiprocessing tools at the top of every
file, or wherever they were needed at import/creation. Now all multiprocessing tools are attached to the
app.shared_ctx. This means `api_app` is imported in many, many places.
This forced a change in how the DownloadManager works. Previously, it would continually run
download workers which would pull downloads from a multiprocessing.Queue. Now, a single worker checks
for new downloads and sends a Sanic signal.
Flags have been reworked to use the `api_app`. I removed the `which` flag functionality because the
`which` are called at import and needed their own multiprocessing.Event.
* A new zim module has been added, this module allows several things to interact with Zim files.
* Allow searching of Zim files using libzim.
* Allow downloading of Kiwix projects (wikipedia, wikibooks, etc) on a schedule which replaces the outdated files.
* Creation of a new zim docker container which allows viewing of the Zim file contents.
* This commit also adds move/rename/mkdir features in the File Browser.
* File tags are preserved when moving/renaming.
Tags can be created and deleted in UI.
Adding tag_file using id to avoid session error.
Using UTC in test.
Matching tag_names style in test.
Tag icon is displayed on file rows.
Archives, Videos and Files can be filtered by their tags.
Tags can be added and removed from UI.
Tags are displayed in VideoPlayer.
The tags of a file are returned in it's JSON.
Video favorites are converted to Favorite tag on channel config import.
Updating tests to match frontend usage.
PDF creator and modification time are displayed.
Reworking file search to use Video.__json__ for complex video relations.
Archives can be viewed.
Removing ebook.title.
Reworking alembic migration to be an upgrade rather than a replace.
Skipping test on circleci.
Clearing out video caption index only if extract_subtitles if off.
Channel tests pass.
Some video tests pass.
Missing videos can be found. A video can be downloaded.
Orphaned files can be found.
Videos can be tagged.
Videos can be searched.
Videos can be modeled.
Converting some tests to use FileGroup.primary_path.
Adding archive and domains to alembic migration.
Testing cascading Archive titles.
Archives now uses FileGroup.
archive.lib tests pass.
Archive is partially converted to FileGroup.
Calculating FileGroup.primary file on refresh, rather than on select.
wrolpi tests pass.
Using FilesType to convert between str/pathlib.Path for FileGroup.files.
Using list of files in FileGroup.files instead of dictionary.
Storing files in dedicated FileGroup.files to avoid overwriting them.
Adding comment explaining CASE.
Not overwriting FileGroup.data on refresh if no change has been detected.
Removing File model in favor of FileGroup.data.
Adding typing to Files methods.
Files are indexed using the primary mimetype of the FileGroup.
files test_lib tests pass.
Removing file_group_file in favor of simple one-to-many relationship.
PDF files are indexed during refresh.
Adding Tag and TagFile to tag files with specific tags.
Only iterating when full_stem_ids has contents.
Testing that FileGroups are related to their Files.
Files and file_groups are created and linked.
Bump webpack from 5.75.0 to 5.76.1 in /app (#160)
Bumps [webpack](https://github.com/webpack/webpack) from 5.75.0 to 5.76.1.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.75.0...v5.76.1)
---
updated-dependencies:
- dependency-name: webpack
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Moving FilesRefreshButton and DirectoryRefreshButton out of browser file.
Moving file fetching tino FileBrowser.
Removing sub-folders from Files display when any parent folder is clicked.
Using closed captioning for vtt/srt file icon.
Using my own react-semantic-toasts-2 to use toasts in Node 18.
Fixing bug where more than the last page was displayed.
Upgrading yt-dlp.
Fixing sub-folder file selection.
Adding rowKey to SortableTable to give React a key for each row.
Renaming key to path.
Removing duplicate urls.
Unifying file modals.
Displaying modals for image and text files.
Handling refresh loading in FileBrowser. Creating refreshing_directory flag.
Removing accidental commit.
Sorting by url_count.
Using SortableTable on Domains page.
Using SortableTable for Channels page.
Created new FileBrowser which allows browsing of files in the media directory.
Using Node 18 in pi-gen.
Creating new FileBrowser to replace outdated react-keyed-file-browser.
Returning mimetype with file list.
Do not display 'Do not download' on Channel download in Dashboard.
Allowing null channel download frequency so user can stop downloading a channel.
Bumping version for Node 18.
Using my own semantic-ui-offline to avoid pulling Google fonts.
Upgrading app docker container to use Node 18. This requires the removal of several npm packages. File browser will require re-write. It now pulls fonts from Google, this will need to be fixed.
Testing DownloadsToggle click.
Testing trinary states of DownloadsToggle.
Installing @testing-library/user-event for testing App.
Handling all combinations of toggle in light and dark.
Changing CPUUsageProgress colors to be more readable in dark mode.
Moving theme handling into Theme.js. Importing element wrappers from Theme.js to handle element theming.
Changing video menu to dark mode.
Creating ThemeContext to share Dark Mode setting.
Removing duplicate code for search functions.
Displaying X when downloads are disabled.
Using react context to share settings across components.
Refusing to start workers when WROL Mode is enabled.
Changing warning to debug.
Queuing recurring downloads first.
Adding Video.url index.
Adding special handling for youtube shorts to videos.
Reworking Downloads to have one worker per process.
Testing creating downloads and getting downloaders.
Testing video validation and poster generation. Testing video statistics.
Increasing inventory test coverage.
Adding pytest.ini to automatically detect async test.
Adding more archive test coverage.
Testing Archive CRUD.
Catching and logging when encountering error during indexing.
Using FileIcon when Archive has no screenshot.
A Video will be assigned to a Channel by its info json first, then to its directory.
Including suffix in File json.
Adding index for File.suffix.
Gathering suffix during indexing.
Passing entire file down to FileIcon.
Moving logging up.
Returning File.associated in file json.
Associated files are last in file search.
Removing duplicate refreshing code by creating new _refresh_directory function.
Changing File.idempotency to a datetime to avoid having to clear idempotency.
Truncating indexes to avoid error with large files.
Don't overwrite video data when info json is empty.
Using already_downloaded for channel videos.
Awaiting refresh in Channel.refresh.
Testing that videos already downloaded aren't downloaded again.
Adding indexes for video/archive referenced paths.
Changing grid width for mobile or computer.
Adding button to refresh specific directories.
Restoring Channel edit link.
Fixing caption path in video_modeler.
Adding ZIP files to mimetype colors.
Using model instead of first file's model.
Fetching File models in batches to avoid issuing a SELECT for every File with a model.
Flushing generated poster file.
Getting Channel in VideoPage, relying on Video.channl_id, falling back to URL's channel_id.
Attaching Video to it's Channel during download.
Using ExternalCardLink for ArchivePage singlefile link.
Validating and indexing videos after download.
Handling glob when file name contains matching characters.
Reworking and testing split_path_stem_and_suffix and glob_shared_stem.
Adding associated video files to DB during upsert.
Adding pending downloads to dashboard.
Hiding status when download is open.
Handling when load is empty.
Adding load statistics to Dashboard.
Limiting concurrent after_refresh functions.
Handling when no onSelect is declared.
Adding ImageRow for FileTable.
Moving image date into Card.Meta.
Using ExternalLink for FileCard.
Using ExternalLink for ImageCard.
Handling when error occurs during refresh of a directory. Committing during refresh to avoid losing all files if an error occurs.
Removing api health check to avoid spamming logs.
Creating ImageCard to display image posters when they aren't too large.
Removing archive.domain_id not null.
Reworking file refresh to process all files in a single directory.
Bumping version for major file rework.
No longer checking for video corruption.
Adding duration by default to speed up tests.
Using register_indexer for all indexers.
Adding CPU usage progress bar in Welcome.
Modifying TextIndexer to ignore large text files.
Counting all files during refresh.
Adding refresh_logger to clarify logging. Checking for wrol_mode before refresh functions.
Removing unused idempotency from video/channel tables.
Using magic module to speed up mimetype identification.
All tests pass.
Archive tests pass.
Changing video get endpoint to return 'file' to avoid ambiguity.
Video tests pass.
Simplifying regex filtering.
Using regex to ignore case when filtering.
Converting all channel classes to functions.
Restoring channel refresh and download buttons.
Matching format of Archive and Video pages.
Adding sorting to Archives search.
Reworking video orders for file search. Using SearchInput for videos search.
Archives can be searched on the Archives page. Domains can be filtered.
Reworking Archive search to use Files. Videos, Archives and Global search use the FilesView.
Videos are claimed by their Channel based on the Channel's directory. Prev and Next video cards are displayed on the VideoPage.
Creating useVideo for reworked VideoWrapper and VideoPage.
Adding Paginator to FilesView.
Moving file refresh button out of manage tab. Removing Files page tabs. Limiting refresh to one at a time.
Adding class for card icon to match poster height.
Video and Archives always re-claim their files on refresh.
Archive files are indexed and claimed.
FileView table is compact and striped.
Videos page uses FileView.
Updated navlinks to router v6.
Removing ! character from file names.
Disabling select button when cards are visible.
Adding show/hide button for FileTable selection.
The Archive Page is displayed.
Adding mimetype dropdown.
Upgrading to React Router v6 for useSearchParams.
Validating video during modeling. Removing screenshot during corruption check because it was very slow. Updating app to use video files as search results.
Updating video search to return Files.
Video and WROLPi tests pass.
Creating function to refresh files within a channel's directory.
Removing troubleshooting print. Updating result order in test.
Reworking video_factory to handle a specific video file location.
Returning video model on file search.
Getting session using video_file.
Screenshotting when checking for video corruption.
Reducing ffmpeg calls by gathering duration during poster generation.
Moving video file matching into model.
VideoIndexer is moved into video module. Video files are validated.
Moving files module into wrolpi in preparation to use it in modules.
Title and contents are indexed for text file.
Files are indexed on refresh by using their indexer.
Zip files are indexed using their contents.
Testing multiple zip files.
Adding unique to file.path. Checking that the correct indexer is used.
Adding index to file.modification_datetime and file.size.
Handling file info gathering in File.do_index. Only reading contents of file if file.indexed is False.
Video model is found during file refresh. A Video record is created that has the associated files.
Videos without associated files are found.
Videos are found using registered video_modeler() rather than in refresh code.
Removing old video columns.
Handling video info json files.
Trying to import each module individually. Will continue to import modules even if one fails.
Reworking refresh so it performs less DB calls.
Converting file.path to primary key.
file.path is always a pathlib.Path.
Updating video_factory to use new file system, testing it.
Bump sanic from 21.9.1 to 21.12.2 in /docker/archive
Bumps [sanic](https://github.com/sanic-org/sanic) from 21.9.1 to 21.12.2.
- [Release notes](https://github.com/sanic-org/sanic/releases)
- [Changelog](https://github.com/sanic-org/sanic/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/sanic-org/sanic/compare/v21.9.1...v21.12.2)
---
updated-dependencies:
- dependency-name: sanic
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
commit 26bde0fb48d59a86c2fb9f53a7218d49f22d4860
Author: Roland <roland@learningselfreliance.com>
Date: Tue Mar 8 13:25:56 2022 -0700
Using osm2pgsql append to preserve other imported maps.
commit c6c52623eaf02e691b1e75dcf39c3a8334ead55a
Author: Roland <roland@learningselfreliance.com>
Date: Tue Mar 8 08:20:38 2022 -0700
Using placeholder instead of loader when table is loading.
commit 44fec6490e26603c88a98f6bc70fc18686e17f09
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 20:58:34 2022 -0700
Simplifying tableRow function.
commit 11b8b57f1fa59556397620f3cad9a33a0237501d
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 20:53:51 2022 -0700
Removing scientific humanFileSize.
commit 0a09d0a6963afebed7f388aad71bbdd3151d2e9e
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 20:46:22 2022 -0700
Fixing active loader.
commit 07f4deeac861351d7483ef57278eb2e0941e31fb
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 20:40:44 2022 -0700
Increasing recommended map file size.
commit af3ac492c81f4c8f84563a448bc272ae9334d32d
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 20:35:13 2022 -0700
Reworking logic order for simplicity. Fetching downloads on interval.
commit 3d9d31fcec87ed02ced7f67c34907faa5a3b54e5
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 20:24:39 2022 -0700
Handling when pbf import status are loading.
commit 3798961a48e7c9eb165ca4e4051f239dbb088d0c
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 19:53:50 2022 -0700
Fetching import status every 30 seconds.
commit f7a8193b0955d14ead8090aa74ce3c5a681b57ec
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 19:33:17 2022 -0700
Stopping renderd during installation.
commit 5bf050b59e0a6e391184596f5e6f0737dea1221d
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 19:26:59 2022 -0700
Removing duplciate serve.
commit 892e3315eeeb5d2516996b901a32a73a66dae078
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 19:22:14 2022 -0700
Setting bash flags in sub-install script.
commit d5d2acd2cd85e089c78aa85a27c5d966222a1347
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 19:20:19 2022 -0700
Moving bulk of installation script into separate script so the latest code will be used during initial installation.
commit ab030d8af834a4a8772edd06df360c51e72fbb02
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 18:40:34 2022 -0700
Removing mod_tile/ajt directory instead of entire contents of mod_tile. Showing import as relative to media directory.
commit c05199db4ecf56f4d096051b58689457882af12a
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 18:39:59 2022 -0700
Remvoing marker from map.
commit ba5df4cf62d61b55281277a6254836e973c766ae
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 11:57:41 2022 -0700
Creating import_map.sh script.
commit 1a16e46130e775efe375adb22917ba12e44855cd
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 11:11:01 2022 -0700
Increasing ram usage during import to avoid hanging import.
commit 0f5ce1a9f806f1c0878dd804b04587b6340ffd16
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 19:19:47 2022 -0700
Increasing map_file.size size.
commit eb6e01512af59fb4a18ab15643c7bc61f3d48e53
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 18:55:55 2022 -0700
Copying leaflet js/css files into docker container and to Apache directory so they will be available offline.
commit 5b687324e119f021673f42e8d079b83034b70d53
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 18:36:06 2022 -0700
Matching tab height on Map page.
commit 8031786d672e18366e9a2eb2d7bdfcd8611d27f3
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 18:34:20 2022 -0700
Shrinking HelpPopup Icon.
commit 5ea94488e43bbc5c414e61c9173a3cfa908c7104
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 18:31:30 2022 -0700
Testing if no PBFs are requested to be imported.
commit c4e86be1faed78bd3af1f3487471c01a1958a433
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 18:18:48 2022 -0700
Using Icon instead of Button for HelpPopup.
commit 51d94f6d85885803813544fea16a56372babaa82
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 18:07:03 2022 -0700
PBF files are sorted by the api.
commit 40f3a7ff04a83db0153f64a88a37f1d56ce043aa
Author: Roland <roland@learningselfreliance.com>
Date: Fri Mar 4 18:35:51 2022 -0700
Adding help popup.
commit 1da8d9fa403ec2a3b3c16c11a3937de37c895a35
Author: Roland <roland@learningselfreliance.com>
Date: Fri Mar 4 09:41:01 2022 -0700
Moving message up. Clearing cached tiles after import.
commit 3192adf95192554c194c9e0a8aeb149822462682
Author: Roland <roland@learningselfreliance.com>
Date: Thu Mar 3 21:37:13 2022 -0700
Removing osm2pgsql from api docker because it seems to be unsupported.
commit 0e10522f3cdc7edd374c14f230bb2dbd68ab1a94
Author: Roland <roland@learningselfreliance.com>
Date: Thu Mar 3 21:36:39 2022 -0700
Disabling import button for docker. Adding download instructions.
commit e1d8f55d159fb4d4ef2ece859ef7cc1059885791
Author: Roland <roland@learningselfreliance.com>
Date: Thu Mar 3 21:14:55 2022 -0700
Map files can be imported on RPi.
commit 3128530e09968430a71b27cd2e75a567f85a708b
Author: Roland <roland@learningselfreliance.com>
Date: Thu Mar 3 15:31:54 2022 -0700
Adding shm_size to increase size to render large map tiles.
commit f3cf80fae0ce46f44ea9ac5033f68a32d0197ae3
Author: Roland <roland@learningselfreliance.com>
Date: Wed Mar 2 10:39:18 2022 -0700
Can get status of map imports.
commit 3b7bbce004cd55a5d89980c2c09124b72c450bc6
Author: Roland <roland@learningselfreliance.com>
Date: Tue Mar 1 20:51:13 2022 -0700
Adding map link to 50x.html. Swapping links using hostname to make it easier on the user.
commit a8b6b60fe182233f40c2f1ecd825636c3d37fd23
Author: Roland <roland@learningselfreliance.com>
Date: Tue Mar 1 12:01:50 2022 -0700
Moving Inventory link to end.
commit c5346b8dc2ff83262d6fd153130912809dfd7574
Author: Roland <roland@learningselfreliance.com>
Date: Tue Mar 1 12:00:47 2022 -0700
Squashed commit of the following:
commit 39f0b8f3fc049fffd140d551cd91d7d9dba6d416
Author: Roland <roland@learningselfreliance.com>
Date: Tue Mar 1 12:00:13 2022 -0700
Starting map refresh function.
commit 2fac9ddf07e378389ddd85ca972f33b3323d30ac
Author: Roland <roland@learningselfreliance.com>
Date: Mon Feb 28 16:43:47 2022 -0700
Updating readme with map requirements.
commit 8325e1ea89537128329e313ab4b5588f8809d1c9
Author: Roland <roland@learningselfreliance.com>
Date: Mon Feb 28 16:05:35 2022 -0700
Fixing order of map db init scripts. Creating index.html to display map. Using :8084 index file to display the map.
commit 8f81539dd9d25fc26607f472c156d41ddcfc11f5
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 19:17:14 2022 -0700
Installing apache2.
commit c7aaef07f8291070aeaf5e7df553e34aca0719ec
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 18:41:07 2022 -0700
Always configure postgres extensions. Initialize gis with dc.
commit aa725ed4b8ec46d17accebd96dd41508ea2d9caf
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 15:48:20 2022 -0700
Map is displayed as iframe.
commit de9bb92dcaf44996e7727adce7774f01895ddf52
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 12:16:56 2022 -0700
Running script in openstreetmap directory.
commit f277388095d2221aa6407643ab655f26dc349eb7
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 12:06:41 2022 -0700
Bumping version for map.
commit afd4aaeffb2aced2dc5c1a8f94038885c8287265
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 11:59:31 2022 -0700
Checking that the installation script is on an rpi.
commit a799382ceac3c201fc01d41fd63e37607282831a
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 11:49:21 2022 -0700
Only running mod_tile install once.
commit 0b98bb19cd4d86c9630a863cab4eb46410d1c9d1
Author: Roland <roland@learningselfreliance.com>
Date: Fri Feb 25 11:31:21 2022 -0700
Building out map install script.
commit 7c799775c11df573fa6cd65451adea28c0832cb0
Author: Roland <roland@learningselfreliance.com>
Date: Thu Feb 24 12:16:53 2022 -0700
Creating OSM installation script.
commit d3b93a3ee97d1ae4a00b6a0423ed59ea8b897ccd
Author: Roland <roland@learningselfreliance.com>
Date: Tue Feb 22 15:39:04 2022 -0700
Removing unused map module.
* Squashed commit of the following:
commit c527475fcb4d4906457be5c10f0e9c6dee7fedc1
Author: Roland <roland@learningselfreliance.com>
Date: Tue Dec 14 22:16:36 2021 -0700
Only saving a favorite video to the config if it has a source_id.
commit ae8b712ebeaef467b8d5a5b87ca0f4f276373a0d
Author: Roland <roland@learningselfreliance.com>
Date: Tue Dec 14 22:14:26 2021 -0700
Ading description and changing Save button to Download.
commit 03a72be6dfdfa383915950a80e3f1ee9febf6705
Author: Roland <roland@learningselfreliance.com>
Date: Tue Dec 14 22:09:44 2021 -0700
Removing uploader for now.
commit 5d0231284bf84e459687f6ab903b06bf4e3762bd
Author: Roland <roland@learningselfreliance.com>
Date: Tue Dec 14 22:07:17 2021 -0700
Removing unique data from example info.
commit 9cc721a32d4715d9d6433e51db3b35e039c78d68
Author: Roland <roland@learningselfreliance.com>
Date: Tue Dec 14 22:02:33 2021 -0700
Updating downloader tests.
commit 8e193b03349ae4613f105418cd3e8176c7101304
Author: Roland <roland@learningselfreliance.com>
Date: Tue Dec 14 21:53:31 2021 -0700
Toasting error when WROL Mode is enabled. Disabling new downloads during WROL Mode.
commit 48c4311c31db8f2d2c8dd0a65f979a5f0a49a911
Author: Roland <roland@learningselfreliance.com>
Date: Tue Dec 14 21:45:06 2021 -0700
Raising error if download exits without a 0. Clearing kill event when starting a new download.
commit 54ba3a890d59facc837410d34bf7afb40db3b685
Author: Roland <roland@learningselfreliance.com>
Date: Tue Dec 14 21:16:46 2021 -0700
Downloads can be restarted and killed.
commit bd60751262247259c3e9ceb5116e017d6314c219
Author: Roland <roland@learningselfreliance.com>
Date: Mon Dec 13 16:28:18 2021 -0700
Downloads can be started again. Downloads are sorted on Downloads page.
commit ea7f39231b92982ae0b263b76c5bd1902cc8169d
Author: Roland <roland@learningselfreliance.com>
Date: Sat Dec 11 21:24:16 2021 -0700
Building Downloads tab.
commit d3321872ac3743ff6df781b07ef0fb537f1d4c98
Author: Roland <roland@learningselfreliance.com>
Date: Sat Dec 11 20:41:46 2021 -0700
Not scheduling channels withtout a URL/frequency.
commit 84f268a0d4f58c85f86b6377900492e0766f6619
Author: Roland <roland@learningselfreliance.com>
Date: Sat Dec 11 19:56:08 2021 -0700
Channel downloads will be spread out when each channel is created.
commit 26fa6beea38ef3f8d6b7bf218f18222a51a87a60
Author: Roland <roland@learningselfreliance.com>
Date: Sat Dec 11 13:08:11 2021 -0700
Recurring downloads are renewed on startup.
commit fc2ab850533f1c3ad073f4692c7dea2be43e85d6
Author: Roland <roland@learningselfreliance.com>
Date: Fri Dec 10 17:03:57 2021 -0700
Fetching download again after do_download.
commit fdadad327fc09c17a49eb63b11e632fb72cc3195
Author: Roland <roland@learningselfreliance.com>
Date: Fri Dec 10 16:20:51 2021 -0700
Fixing error with test. Mock was not being reset.
commit 070dee238e53377acd22097c63cf3794e68ed423
Author: Roland <roland@learningselfreliance.com>
Date: Fri Dec 10 15:36:39 2021 -0700
Fixed bug in date handling (big surprise).
commit e53a8992268d68620c679455adaec3a590f20602
Author: Roland <roland@learningselfreliance.com>
Date: Wed Dec 1 16:48:36 2021 -0700
Using chunks for regular channel refresh.
commit 3bd60c58d1bef0226325f321978c0434b36b3be5
Author: Roland <roland@learningselfreliance.com>
Date: Wed Dec 1 15:22:02 2021 -0700
Reformatting.
commit 4fdc325f25a32c311ca6543fe321624b2bd564b2
Author: Roland <roland@learningselfreliance.com>
Date: Wed Dec 1 15:13:34 2021 -0700
Creating function that refreshes NO CHANNEL videos.
commit d67feafb6c2994f5a4a1295920a70db369cc6816
Author: Roland <roland@learningselfreliance.com>
Date: Wed Dec 1 14:35:54 2021 -0700
Removing unused functions.
commit 6b3918e4e78fbac64920f85c4de0df873ec366a1
Author: Roland <roland@learningselfreliance.com>
Date: Wed Dec 1 14:31:48 2021 -0700
Removing reporter and events. This will be replaced in the future with some sort of SSE.
commit 54fa1b0f5e93ca25c80b780dcdc3876b700273b1
Author: Roland <roland@learningselfreliance.com>
Date: Sun Nov 28 17:55:43 2021 -0700
Better error handling for archive button.
commit c31d124e511b27f626ed2df71486d47302b396f3
Author: Roland <roland@learningselfreliance.com>
Date: Sun Nov 28 17:29:43 2021 -0700
Displaying View Count in a human readable format.
commit 0ee2bd65588fd93b475cad6144e76f1b45191b39
Author: Roland <roland@learningselfreliance.com>
Date: Sat Nov 27 09:29:41 2021 -0700
Fixing tests.
commit e64aff32bab95c311ea3747e1a21b58b99750d5e
Author: Roland <roland@learningselfreliance.com>
Date: Sat Nov 27 08:59:51 2021 -0700
Handling when a Video has no Channel in UI.
commit b9549ecd5aac03c002adca7b7667f0a7189b662a
Author: Roland <roland@learningselfreliance.com>
Date: Fri Nov 26 08:35:36 2021 -0700
Downloaders can raise UnrecoverableDownloadError to signal a downlaod should not be retried.
commit da88798e2584d3e84900a70e34166c8c208c64fb
Author: Roland <roland@learningselfreliance.com>
Date: Wed Nov 24 20:40:29 2021 -0700
Downloaders can refuse a download, which will then be deleted.
commit b865d856121db5ec8a3660c5d3c13b9846fde688
Author: Roland <roland@learningselfreliance.com>
Date: Wed Nov 24 19:35:43 2021 -0700
Removing relative path handling.
commit 6d7757870b02ba030d78b3ef7e43f28b906eea61
Author: Roland <roland@learningselfreliance.com>
Date: Wed Nov 24 19:28:27 2021 -0700
Using __json__ for Video and Channel models.
commit 649e4d6a61608272759d1e7ee02b490bb5c20b3e
Author: Roland <roland@learningselfreliance.com>
Date: Wed Nov 24 18:23:38 2021 -0700
Fixing video links in UI.
commit 9d081af12742070b8519bd752812130cb5dbe393
Author: Roland <roland@learningselfreliance.com>
Date: Wed Nov 24 14:42:38 2021 -0700
Fixing docs.
commit 1202cbe0f9ae744d7801b53a5071cded8ea39924
Author: Roland <roland@learningselfreliance.com>
Date: Wed Nov 24 14:40:36 2021 -0700
Fixing wrolpi tests.
commit 3f6af8cc1123f2f63a3fc0f21ec4d994c642cf13
Author: Roland <roland@learningselfreliance.com>
Date: Wed Nov 24 14:13:35 2021 -0700
Fixing tests to use MediaPath.
commit faec04f2a5e4c267a502c03884940a9c5bcfe387
Author: Roland <roland@learningselfreliance.com>
Date: Fri Nov 19 10:21:20 2021 -0700
Testing when a video has no channel.
commit cbf22aad7a0b72e61685e0236e843554b57c4af1
Author: Roland <roland@learningselfreliance.com>
Date: Thu Nov 18 16:24:03 2021 -0700
Using MediaPath for Video and Channel models. Testing VideoDownloader.do_download.
commit 7916ed61e43439e6cd60f23ce3fdf6c1a75bf02d
Author: Roland <roland@learningselfreliance.com>
Date: Thu Nov 18 12:43:44 2021 -0700
Channels/Videos downloaders can get gotten.
commit 808ef796cec01db6e7009065f9e26d25d9500ce2
Author: Roland <roland@learningselfreliance.com>
Date: Wed Nov 17 21:45:19 2021 -0700
Upgrading to Sanic 21.9.1.
commit ea32813be28e78393ac456ee782a7245c098188f
Author: Roland <roland@learningselfreliance.com>
Date: Wed Nov 17 20:29:01 2021 -0700
Creating Downloader and DownloaderManager classes which handle downloading URLs.
commit e859a77a77277ddf8b6e221ec9c9383c8ab91d66
Author: Roland <roland@learningselfreliance.com>
Date: Wed Nov 17 12:41:39 2021 -0700
Creating UI segement which allows the download/upload of media.
* Displaying all recurring downloads.
* Not downloading channels with empty url.
* Showing positive row for recurring downloads.
* Removing default last_successful_download.
* Only displaying Last Successful Download when there is one.
* Refreshing videos before finding missing videos.
* Using short video path name for log.
* Removing download history feature in favor of new Downloads page.
* Only showing info date if it exists.
* Making get_or_create_channel more reliable.
* Allowing null for channel download frequency.
* Logging when searching for channel.
* Storing channel's source_id in config.
* Importing using link instead of name.
* Getting channel id when importing channels.
* Removing redunant parentheses from repr.
* Preserving database channel values if there are not values in the config.
* Only highlighting pending downloads.
* Deleting old downloads periodically.
* Sorting recurring by next_download/frequency.
* Creating fixtures for channels, media directory and db session.
* Removing unused function. Wrapping config for tests.
* Deleting channel download when deleting a channel.
* Testing that a channel's download is deleted when the channel is deleted.
* Updating youtube-dl
* Switching from youtube-dl in favor of yt-dlp.
* Sorting once-downlaods by last_successful_download then id.
* Adding special handling to video_url_resolver for full urls.
* Consolidating downloader methods and functions. Reversing once-download sort order so the most recently downloaded are at the top.
* Getting session if one is not provided.
* Adding test for create_downloads.
* Not passing format option when getting filename.
* Linting
Signed-off-by: Roland <roland@learningselfreliance.com>
