* Add a Solar calculator with offline NASA POWER data
Estimates a solar array's typical-year production from a location and the
panels; everything else has defaults under an optional "Fine-tune" section.
The model works from 12 monthly averages: Klein's average day per month, the
Erbs monthly diffuse fraction, Collares-Pereira & Rabl / Liu & Jordan hourly
split, HDKR transposition for fixed, single-axis and dual-axis arrays, then
PVWatts' ASHRAE incidence loss, SAPM cell temperature, DC power, system
losses and optional inverter. "Best for the year" and "Best for winter" sweep
the tilt.
Sunlight, diffuse, albedo and temperature come from NASA POWER's 2001-2020
climatology on a 1 degree grid, packed into app/public/data (3.1 MB, ~1 MB
gzipped) by scripts/build_solar_climatology.py and fetched only when the
calculator opens. Users can replace it with their own monthly values.
Every setting lives in the URL, with defaults left out, so the Share button
shares the whole configuration.
Against PVWatts v8 for a 1 kW fixed array: Phoenix +0.6%, Seattle +8.8%,
Anchorage +8.0% annually; these are recorded as tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Open the Solar calculator from the map, pins and current location
The map's right-click (or long-press) menu gains "Calculate Solar
Performance", and each saved pin a sun button; both open the calculator at
that location. A button beside Latitude fills in the browser's current
location. Locations are rounded to 4 decimals, and longitudes from a map
panned around the world are wrapped back to -180..180.
The map removed lat/lon/z from the URL when it closed, even after navigating
to another page, which stripped the calculator's location from its URL (and
from links shared with the Share button). It now only cleans up while the
URL is still a map page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add off-grid sizing to the Solar calculator
A new "Off-grid sizing" tab sizes a battery system for the darkest month
from a list of devices (watts x hours per day):
- Battery bank for a number of days without sun, for LiFePO4 or lead-acid,
at 12, 24 or 48 V. Outdoor lead-acid is enlarged for the cold; outdoor
lithium warns against charging below 0 C.
- Panels needed, from the worst month's DC production per kW at the chosen
angle, with a button to apply the count. A PWM controller's loss (battery
voltage over the panel's Vmp) raises the count.
- Charge controller current, and the string voltage on the coldest morning
against the controller's limit, since cold panels exceed their rated Voc.
The coldest morning defaults to 25 C below the coldest monthly average.
All inputs live in the URL like the rest of the calculator.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Bump wrolpi-help for the Solar calculator help page
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Credit NASA POWER as it asks, and say how the data was changed
- The calculator gains a "Data attribution" link under the location with
NASA POWER's requested acknowledgment, the data reference (Climatology API
v2.10.0, downloaded 2026/10/04), the CC BY 4.0 license, and how WROLPi
modified the data (rounded to one byte per value, temperature averaged
into 1 degree cells, interpolated between cells), which CC BY 4.0 requires.
- The build script writes the same text to solar-climatology.LICENSE.txt
beside the data, with the API version taken from NASA's responses and the
date the tiles were downloaded. A test keeps the page and file in step.
- Bumps wrolpi-help for the updated attribution in the help page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Solar: fix PWM sizing, Voc sign, unit switching and loading on review
- PWM usable power now uses the string's voltage (Vmp x panels in series),
delivers nothing when the charging voltage is at or above the string's Voc,
and falls off between Vmp and Voc. The "too little voltage" warning covers
PWM as well as MPPT. Previously two panels in series counted double, and a
panel that could not charge a 48 V bank counted as fully usable.
- A Voc temperature coefficient entered without its minus sign is treated as
negative, in the math, the URL and the input, so the cold-morning check
cannot be made too generous.
- Switching Fahrenheit on or off converts the entered temperatures instead
of reinterpreting them.
- While the sunlight data downloads, the off-grid tab shows a loader rather
than asking for a location, and holds the cold-morning verdict until the
site's temperatures are known or one has been entered.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Solar: move the data attribution to the bottom of the page
The full NASA POWER acknowledgment, license and changes now sit in a small
"Data attribution" section at the end of the calculator, always visible,
instead of behind a link under the location. The source line under the
location points to it. Bumps wrolpi-help to match.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Solar: drop the data source line under the location
The attribution at the bottom of the page is enough. The line under the
location now only appears to report the data's state: loading, failed to
load, or replaced by the user's own values. Bumps wrolpi-help for the
refreshed screenshot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Fix API stalls: non-blocking switch worker, threaded searches, indexed Videos page
Three response-time fixes found by benchmarking the read-only API endpoints
hot and cold on real libraries:
* `switch_worker` runs on the Sanic worker's event loop every 0.1s and called
the blocking `multiprocessing.Event.wait(timeout=1)`. With no switch pending
(the normal state) this froze the loop for a second at a time; about a third
of all requests, including trivial ones, took ~1,000 ms. It now polls
`is_set()` and returns immediately.
* The search handlers (videos, files, archives, docs, zims) and `/api/statistics`
ran synchronous SQLite queries on the event loop, so one slow search stalled
every other request on that worker. They now run via `asyncio.to_thread`.
* The unfiltered Videos page (`POST /api/videos/search` with only an order)
used `mimetype LIKE 'video/%' OR ...` plus `COUNT(*) OVER()`, which visits
every video's index entry before LIMIT applies: seconds cold on a large
library on a spinning disk, and ~10 s hot for a deep page. It now seeks each
exact mimetype on the (mimetype, effective_datetime) index for `limit + offset`
rows and merges them, with the total from a separate count cached for 60 s.
Results are identical to the generic query.
Also adds `scripts/api_perf.py` (times every read-only endpoint against a host,
optionally restarting the API over SSH for cold numbers) and
`scripts/api_perf_compare.py` (merges per-host results). Host profiles live in a
gitignored JSON file; an example is included.
Measured on an 8-core docker host with a 10 GB library: median hot request
1,007 ms -> 22 ms; Videos page 1 hot 287 -> 108 ms, page 50 1,345 -> 120 ms.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Serve the video description from its own endpoint instead of Video.__json__
`Video.__json__` computed `description` as `c_text or get_video_description()`,
where the fallback opens and parses the video's info json from disk. The list
endpoints serialize every result with `__json__`, so any page containing videos
without an indexed description (common: many videos have none) read their info
json files on every request, cold, for a field the list never displays. The
description is not part of search either: search runs against the FTS copy of
`c_text`, so the fallback never fed the index.
Drop `description` from `__json__` and serve it from
`GET /api/videos/<id>/description`, like comments and captions. The endpoint
returns the indexed `c_text`, falling back to the info json only there, for a
row indexed before descriptions were stored. The player page fetches it through
`useVideoExtras` alongside the comments.
Tests assert the list and detail responses never touch the info json, and cover
the endpoint's indexed, fallback, and not-found cases.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address review: thread-local zim sessions, fast-path equivalence test, comments
* Zim search, single-zim search, and search estimates run libzim off the event
loop, each opening its own session inside the worker thread rather than
passing the request's SQLAlchemy Session across threads.
* The Videos fast-path test now runs the same requests through the generic
query and asserts identical rows and paging for both date orders. The one
intentional difference is documented: past the end, the fast path still
reports the true total.
* switch_worker returns before its try block so the empty path does not clear
the event; its comment states the invariant only. The copied "synchronous
SQLite" comments are reduced to one line in one place.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Drop .pgpass handling, Postgres ownership repairs, and the docker db
container viewability rules. Update comments and pi-gen map docs for
SQLite/PMTiles. Help submodule documents the new database layout.
Replacing Channels and Domains with Collection.
Moving downloading relationships from Channel to Collection.
Created domains.yaml config to manage tagged domain collections.
Adding turn-key upgrade solution in the UI.
Python 3.11 is now required, old versions were causing unpredictability in tests. (Sanic does not yet
support 3.12)
Sanic has been upgraded to 23.6.0, which is the latest version that avoids this bug:
https://github.com/sanic-org/sanic/issues/2921
New strategy for multiprocessing is to create all multiprocessing tools in one process, then fork
to other processes. The previous strategy was to declare multiprocessing tools at the top of every
file, or wherever they were needed at import/creation. Now all multiprocessing tools are attached to the
app.shared_ctx. This means `api_app` is imported in many, many places.
This forced a change in how the DownloadManager works. Previously, it would continually run
download workers which would pull downloads from a multiprocessing.Queue. Now, a single worker checks
for new downloads and sends a Sanic signal.
Flags have been reworked to use the `api_app`. I removed the `which` flag functionality because the
`which` are called at import and needed their own multiprocessing.Event.
commit 26bde0fb48d59a86c2fb9f53a7218d49f22d4860
Author: Roland <roland@learningselfreliance.com>
Date: Tue Mar 8 13:25:56 2022 -0700
Using osm2pgsql append to preserve other imported maps.
commit c6c52623eaf02e691b1e75dcf39c3a8334ead55a
Author: Roland <roland@learningselfreliance.com>
Date: Tue Mar 8 08:20:38 2022 -0700
Using placeholder instead of loader when table is loading.
commit 44fec6490e26603c88a98f6bc70fc18686e17f09
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 20:58:34 2022 -0700
Simplifying tableRow function.
commit 11b8b57f1fa59556397620f3cad9a33a0237501d
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 20:53:51 2022 -0700
Removing scientific humanFileSize.
commit 0a09d0a6963afebed7f388aad71bbdd3151d2e9e
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 20:46:22 2022 -0700
Fixing active loader.
commit 07f4deeac861351d7483ef57278eb2e0941e31fb
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 20:40:44 2022 -0700
Increasing recommended map file size.
commit af3ac492c81f4c8f84563a448bc272ae9334d32d
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 20:35:13 2022 -0700
Reworking logic order for simplicity. Fetching downloads on interval.
commit 3d9d31fcec87ed02ced7f67c34907faa5a3b54e5
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 20:24:39 2022 -0700
Handling when pbf import status are loading.
commit 3798961a48e7c9eb165ca4e4051f239dbb088d0c
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 19:53:50 2022 -0700
Fetching import status every 30 seconds.
commit f7a8193b0955d14ead8090aa74ce3c5a681b57ec
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 19:33:17 2022 -0700
Stopping renderd during installation.
commit 5bf050b59e0a6e391184596f5e6f0737dea1221d
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 19:26:59 2022 -0700
Removing duplciate serve.
commit 892e3315eeeb5d2516996b901a32a73a66dae078
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 19:22:14 2022 -0700
Setting bash flags in sub-install script.
commit d5d2acd2cd85e089c78aa85a27c5d966222a1347
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 19:20:19 2022 -0700
Moving bulk of installation script into separate script so the latest code will be used during initial installation.
commit ab030d8af834a4a8772edd06df360c51e72fbb02
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 18:40:34 2022 -0700
Removing mod_tile/ajt directory instead of entire contents of mod_tile. Showing import as relative to media directory.
commit c05199db4ecf56f4d096051b58689457882af12a
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 18:39:59 2022 -0700
Remvoing marker from map.
commit ba5df4cf62d61b55281277a6254836e973c766ae
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 11:57:41 2022 -0700
Creating import_map.sh script.
commit 1a16e46130e775efe375adb22917ba12e44855cd
Author: Roland <roland@learningselfreliance.com>
Date: Mon Mar 7 11:11:01 2022 -0700
Increasing ram usage during import to avoid hanging import.
commit 0f5ce1a9f806f1c0878dd804b04587b6340ffd16
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 19:19:47 2022 -0700
Increasing map_file.size size.
commit eb6e01512af59fb4a18ab15643c7bc61f3d48e53
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 18:55:55 2022 -0700
Copying leaflet js/css files into docker container and to Apache directory so they will be available offline.
commit 5b687324e119f021673f42e8d079b83034b70d53
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 18:36:06 2022 -0700
Matching tab height on Map page.
commit 8031786d672e18366e9a2eb2d7bdfcd8611d27f3
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 18:34:20 2022 -0700
Shrinking HelpPopup Icon.
commit 5ea94488e43bbc5c414e61c9173a3cfa908c7104
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 18:31:30 2022 -0700
Testing if no PBFs are requested to be imported.
commit c4e86be1faed78bd3af1f3487471c01a1958a433
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 18:18:48 2022 -0700
Using Icon instead of Button for HelpPopup.
commit 51d94f6d85885803813544fea16a56372babaa82
Author: Roland <roland@learningselfreliance.com>
Date: Sat Mar 5 18:07:03 2022 -0700
PBF files are sorted by the api.
commit 40f3a7ff04a83db0153f64a88a37f1d56ce043aa
Author: Roland <roland@learningselfreliance.com>
Date: Fri Mar 4 18:35:51 2022 -0700
Adding help popup.
commit 1da8d9fa403ec2a3b3c16c11a3937de37c895a35
Author: Roland <roland@learningselfreliance.com>
Date: Fri Mar 4 09:41:01 2022 -0700
Moving message up. Clearing cached tiles after import.
commit 3192adf95192554c194c9e0a8aeb149822462682
Author: Roland <roland@learningselfreliance.com>
Date: Thu Mar 3 21:37:13 2022 -0700
Removing osm2pgsql from api docker because it seems to be unsupported.
commit 0e10522f3cdc7edd374c14f230bb2dbd68ab1a94
Author: Roland <roland@learningselfreliance.com>
Date: Thu Mar 3 21:36:39 2022 -0700
Disabling import button for docker. Adding download instructions.
commit e1d8f55d159fb4d4ef2ece859ef7cc1059885791
Author: Roland <roland@learningselfreliance.com>
Date: Thu Mar 3 21:14:55 2022 -0700
Map files can be imported on RPi.
commit 3128530e09968430a71b27cd2e75a567f85a708b
Author: Roland <roland@learningselfreliance.com>
Date: Thu Mar 3 15:31:54 2022 -0700
Adding shm_size to increase size to render large map tiles.
commit f3cf80fae0ce46f44ea9ac5033f68a32d0197ae3
Author: Roland <roland@learningselfreliance.com>
Date: Wed Mar 2 10:39:18 2022 -0700
Can get status of map imports.
commit 3b7bbce004cd55a5d89980c2c09124b72c450bc6
Author: Roland <roland@learningselfreliance.com>
Date: Tue Mar 1 20:51:13 2022 -0700
Adding map link to 50x.html. Swapping links using hostname to make it easier on the user.
commit a8b6b60fe182233f40c2f1ecd825636c3d37fd23
Author: Roland <roland@learningselfreliance.com>
Date: Tue Mar 1 12:01:50 2022 -0700
Moving Inventory link to end.
commit c5346b8dc2ff83262d6fd153130912809dfd7574
Author: Roland <roland@learningselfreliance.com>
Date: Tue Mar 1 12:00:47 2022 -0700
Squashed commit of the following:
commit 39f0b8f3fc049fffd140d551cd91d7d9dba6d416
Author: Roland <roland@learningselfreliance.com>
Date: Tue Mar 1 12:00:13 2022 -0700
Starting map refresh function.
commit 2fac9ddf07e378389ddd85ca972f33b3323d30ac
Author: Roland <roland@learningselfreliance.com>
Date: Mon Feb 28 16:43:47 2022 -0700
Updating readme with map requirements.
commit 8325e1ea89537128329e313ab4b5588f8809d1c9
Author: Roland <roland@learningselfreliance.com>
Date: Mon Feb 28 16:05:35 2022 -0700
Fixing order of map db init scripts. Creating index.html to display map. Using :8084 index file to display the map.
commit 8f81539dd9d25fc26607f472c156d41ddcfc11f5
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 19:17:14 2022 -0700
Installing apache2.
commit c7aaef07f8291070aeaf5e7df553e34aca0719ec
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 18:41:07 2022 -0700
Always configure postgres extensions. Initialize gis with dc.
commit aa725ed4b8ec46d17accebd96dd41508ea2d9caf
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 15:48:20 2022 -0700
Map is displayed as iframe.
commit de9bb92dcaf44996e7727adce7774f01895ddf52
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 12:16:56 2022 -0700
Running script in openstreetmap directory.
commit f277388095d2221aa6407643ab655f26dc349eb7
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 12:06:41 2022 -0700
Bumping version for map.
commit afd4aaeffb2aced2dc5c1a8f94038885c8287265
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 11:59:31 2022 -0700
Checking that the installation script is on an rpi.
commit a799382ceac3c201fc01d41fd63e37607282831a
Author: Roland <roland@learningselfreliance.com>
Date: Sun Feb 27 11:49:21 2022 -0700
Only running mod_tile install once.
commit 0b98bb19cd4d86c9630a863cab4eb46410d1c9d1
Author: Roland <roland@learningselfreliance.com>
Date: Fri Feb 25 11:31:21 2022 -0700
Building out map install script.
commit 7c799775c11df573fa6cd65451adea28c0832cb0
Author: Roland <roland@learningselfreliance.com>
Date: Thu Feb 24 12:16:53 2022 -0700
Creating OSM installation script.
commit d3b93a3ee97d1ae4a00b6a0423ed59ea8b897ccd
Author: Roland <roland@learningselfreliance.com>
Date: Tue Feb 22 15:39:04 2022 -0700
Removing unused map module.
