A map is exported as a PDF: a 1:24,000 quadrangle, in layers, on a sheet, which says where it is (#1006)

* The map can choose a 1:24,000 quadrangle to export (#971)

The first part of exporting a map as a PDF: a grid of 7.5-minute quadrangles is drawn over the
map, and a click chooses one.  The grid is arithmetic, so it needs no data.  Exporting itself is
not here yet; the button is disabled.

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* A quadrangle of the map is exported as a PDF with a layer for each map layer (#972)

* A quadrangle of the map is exported as a PDF with a layer for each map layer

The browser draws the quadrangle off the screen at the size of a 1:24,000 sheet, once for each
layer which is on, and the API puts the images in a PDF, each in a layer of its own which a
reader can show and hide.  A layer in which nothing is drawn is left out.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* The exported map's canvas keeps what was drawn, and an export's memory is bounded

- The drawing buffer is asked for where this MapLibre reads it (canvasContextAttributes); as an
  option of the map it was ignored, so the canvas could be read after it was cleared.
- Each layer's pixels are compressed in the PDF as the layer is added.  PyMuPDF kept them
  uncompressed until the PDF was written: about 90 MB a layer of a quadrangle, all at once.
- The pixels of all the layers together are limited, before any image is decoded.
- pymupdf is imported when a PDF is made, not when the API starts.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* An exported map says where on the Earth it is (#973)

* An exported map says where on the Earth it is

The PDF's page has a geospatial viewport (ISO 32000-2): the page is the map in Web Mercator,
with the latitude and longitude of each corner.  The corners are the edges of the picture the
browser drew, read from the map which drew it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* A map whose edge is the antimeridian can be exported

The map's arithmetic puts the east edge of the last quadrangle before the antimeridian a rounding error past
180 degrees, and the export was refused.  An edge within 1e-8 of a degree of 180 is at 180 (it is not wrapped
to -180); a map which is really across the antimeridian is still refused.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* An exported map is on a sheet: a margin, a grid, scale bars and notes (#974)

* An exported map is on a sheet: a margin, a grid, scale bars and notes

The map is put on a larger page with what a topographic map has around it: a title, the latitude and longitude
of its corners and of ticks between them, scale bars (kilometers, miles, feet), and notes of the projection, the
grid and where the map's data is from.  Over the map is a 1,000 meter UTM grid, each line named in the margin.

The grid and the margin are PDF layers of their own.  The PDF says where the map is, not where the page is.  The
picker has a box to export the map alone.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* The scale of a narrow map is written under its name, not over it

A quadrangle near a pole is two inches wide, which is not room for its name and its scale on one line.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* An exported map says where a compass points (#975)

* An exported map says where a compass points

The sheet has a diagram of the three norths (true, grid and magnetic) beside its scale bars, and a note of how
far magnetic north is from true north on the day the map was made, and how fast it moves.

The declination is from the World Magnetic Model 2025 (NOAA NCEI and the British Geological Survey; public
domain), whose coefficient file is shipped: nothing is downloaded.  The model is good until 2030; outside its
years nothing is said of a compass.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* A corner of an exported map is not also a tick

The edges of the picture the browser draws are a hair outside the quadrangle's, so the quadrangle's own edges
were taken for ticks inside the map, and each was written over the coordinate of its corner.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* A compass which passes due south has moved a little, not most of the way around

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* A map can be exported at 400 DPI (#976)

The picker has a choice of 300 or 400 DPI.  A map is drawn less sharp than was asked for, rather than refused,
when it would be more pixels than the API makes a PDF of: a quadrangle near the equator, or one with many layers
on.  The picker says so while it draws.

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* An exported quadrangle goes by the name the USGS gives it (#977)

* An exported quadrangle goes by the name the USGS gives it

The picker says what the chosen quadrangle is called, and the sheet's title, the PDF's title and the file's
name have it.  Where the USGS has no name, which is most of the world, a quadrangle goes by where it is, as
before.

The names are an index of the 65,600 7.5 minute quadrangles of The National Map (U.S. Geological Survey; public
domain), which is shipped: nothing is downloaded.  In Alaska a USGS quadrangle is wider than 7.5 minutes, so a
quadrangle of the picker's grid there is "Part of" the one which has its middle.

The places the tests of the map export are of are changed for others.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* An area half of which is the last quadrangle of the land is a part of it

The middle of such an area is on the quadrangle's east or north edge, with no quadrangle beyond it, and it was
given no name.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* The PDF of a map is made by a Job, which the browser watches (#978)

* The PDF of a map is made by a Job, which the browser watches

A PDF of a quadrangle is a minute or two of work on a Raspberry Pi 4, and the request which waited for it was
cut off after a minute: only the smallest quadrangles could be exported there.

Now the browser describes the map, sends the image of each layer in a request of its own, and starts a Job; it
watches the Job, shows how far along it is, and fetches the PDF when the Job is done.  Cancel stops the Job.
The images are kept on disk until the PDF is made of them, and an export is removed when it is an hour old.

The credits on the sheet are no longer cut off: an attribution which names many sources is each of them.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* A cancelled map export stops, and is removed

- A cancelled Job is not over until the thread making the PDF has stopped, so the next Job does not start beside
  it; the work can also be stopped while the sheet is drawn and the PDF written.
- DELETE /api/map/export/<id> stops an export at any step and removes what was sent; the browser calls it when
  the person exporting cancels, while layers are sent as well as while the PDF is made.
- An export whose Job is waiting or running is not removed for being old; one whose Job was cancelled is removed
  at once.
- Cancel while the map is drawn stops the drawing between two layers.
- Exports are kept in a directory made for this user alone, which is refused when it is a link or is another
  user's; only what is an export there is removed.
- A request for the finished PDF which is lost is made again.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* An overlay's areas and lines go under the map's labels, and its own labels over them (#979)

An overlay was drawn whole over the basemap, so an area it filled covered the names of what is in it.  Labels are
now always on top: the areas, lines and pictures of every overlay are under the basemap's labels, and the text,
icons and points of every overlay are over them.  It is so in the map, the standalone viewer, and an exported PDF,
where an overlay with both is two layers ("Towers" and "Towers (labels)").

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* An overlay's ground is over the ground of every basemap file (#980)

* An overlay's ground is over the ground of every basemap file

On a map of several basemap files, the ground of an overlay was put before the first label of the first file,
which is under the land, water and roads of every file after it: an overlay's areas and lines were hidden, and
in an exported PDF its layer was just above the background.  It goes where the basemap's labels begin: the first
of the labels which are over all of the basemap's ground.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* The labels of every basemap file are over the ground of every file

A map of several basemap files was their layers one file after another, so only the labels of the last file were
on top: the labels of the others were under the land, water and roads of the files after them, and under the
ground of an overlay.  Both viewers now draw the ground of every file and then the labels of every file.

The contour lines are drawn after the basemap's labels and are not of the basemap: an overlay's ground goes under
the basemap's labels, not under the contours instead.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* Contours are drawn from as deep as the terrain file goes (#981)

The map read a terrain file to zoom 8, which is how deep the terrain WROLPi offers goes, however deep the file
was.  It now reads a file as deep as the file says it goes, in both viewers, and draws finer contour lines from a
deeper file: every 40 feet (or 20 meters) from zoom 11, as a 1:24,000 sheet has.  A file which goes to zoom 8 is
drawn as it was.

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* An exported map names its small roads (#982)

* An exported map names its small roads

A quadrangle is drawn at about zoom 13.  The basemap's style names small roads
from zoom 15, and its tiles have no names for them below zoom 14, so the streets
of a town were printed as lines with no names.

Before the map is drawn, the tiles of zoom 14 which the quadrangle is in are
read, and the roads in them which have a name are given to the export map as a
source of its own.  The layer which names small roads draws from that source, as
its style says to.

At the scale of a sheet most streets are shorter than their names, and a map
writes a name along a line only where the line is longer.  Such a road is named
over its middle, turned as the road goes; a road in pieces is named once.

A source whose tiles can not be read is exported as before.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* The names of small roads are placed by the export, at every latitude

A quadrangle nearer the equator is drawn at a little over zoom 14, where the
names were not read and the style still kept them for zoom 15: such a map had
no names of small roads.  They are now read wherever a layer would not name
roads.

Every name of a small road is now a point: over the middle of its road, turned
as the road goes under it, again every 400 pixels of a long road.  Before, a
road long enough was left for the map to name along its line, which the map
often does not (the line bends, or is in pieces), and a short piece beside it
was named too.  One place decides where a road is named, so a road in pieces is
named once, and a piece of another kind of road does not take the name.

Reading the names can be cancelled.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* A sheet has a bar of meters, its norths in mils, and how far apart its contour lines are (#983)

Under the map are bars of kilometers, meters, miles and feet.

How far the grid's north and a compass's north are from true north is written in
mils as well as in degrees, in the diagram and in the notes: a compass may be
marked in either.

A map which is exported with its contour lines says how far apart they are
(CONTOUR INTERVAL 40 FEET), under the bars.  The browser knows: it is the
interval of the tiles the lines are made for at the zoom of the sheet, in the
unit the map measures in.  It says so with the sheet, and what is not an
interval is not written.

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* A sheet says what the quadrangles around it are called (#984)

* A sheet says what the quadrangles around it are called

Under the scale bars is a diagram of nine squares, the map the middle one, and
beside it what the USGS calls each of the eight around it: the sheets to print
to carry on off the edge of this one.

The names are from the index of quadrangles which names the sheet itself.  A
quadrangle with no name (out to sea, or over a border) has no number; a map of
where the USGS names nothing, and a map too narrow for the diagram, go without.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* A sheet is never beside itself

Where a quadrangle of the USGS is wider than the map (Alaska), the area the
size of the map to its east or west is often more of the quadrangle the map is
in, and the diagram named the sheet as its own neighbor.  The quadrangle on a
side is now the next one which is not the map's own.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* A sheet has a legend of the lines of its map (#985)

* A sheet has a legend of the lines of its map

Under the scale bars is what each kind of line of the map is: highway, major
road, minor road, path, railway, boundary, contour and index contour, each a
piece of the line as the map draws it, on the color of the map's land.

The map's style says how a line is drawn at each zoom, and a sheet is drawn at
one.  The browser, which has the style, says what each line's color, width,
dashes and edges are at that zoom; the API draws them.  A line is in the legend
when its layer is exported and draws something at that zoom.

What the browser sends is made into lines before it is kept: what is not a line
is left out, and the map is exported.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* A legend has no line too thin to see, and no name longer than its room

A style brings a line in from nothing as the map is zoomed in: a path is a few
hundredths of a point wide on a quadrangle near the equator.  It was in the
legend, drawn several times wider than on the map.  A line under a tenth of a
point is now left out, by the browser and by the API.

What a line is called is cut to the room beside its line, as the name of an
adjoining quadrangle is.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* Fixes to the map PDF export: wrong neighbors, leftover exports, exports which outlive the map (#989)

* A sheet names what is around it, and its edges, by where they are and not a hair from there

The edges the browser gives of a map are a hair outside its quadrangle.  The quadrangles around a map whose
middle is on the edge two share (every other map of Alaska) were those around the map west of it, and one of
them was the map itself.  An edge a hair south of the equator was 0 degrees south, a map which ended where the
UTM zones do had no grid, and a sheet dated in the last year there is could not be made.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* An export is removed when it is over, and its work stops when the API does

The images and the PDF of an export were kept until the next export began, however long that was.  The browser
now has the export removed however it ends, and old exports are removed when a PDF is made too.  The work of a
Job which ended because the API was stopping went on to the end of the PDF, with the API waiting for it.  What
is said of an export, and each image, is written whole.  Text which is no text is taken out when an export
begins, not found by the Job.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* An export stops with its map, and waits for a page which can not be seen

Leaving the map while it was exported left the export going, to fail or to hand a PDF to another page; so did
closing the picker, which is now left open.  A PDF which arrives after the export was stopped is not saved.
The time a page is behind another tab is not counted against a map which is drawing.  The name of a road is
one line.  A photograph which ends a little inside the map is not made a JPEG with a black strip.  An accent
in the name of a quadrangle is not a gap in the name of its file.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* A PDF is not kept from who made it by a server slow to remove the export

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* An exported map has its water over its parks (#990)

* A sheet names what is around it, and its edges, by where they are and not a hair from there

The edges the browser gives of a map are a hair outside its quadrangle.  The quadrangles around a map whose
middle is on the edge two share (every other map of Alaska) were those around the map west of it, and one of
them was the map itself.  An edge a hair south of the equator was 0 degrees south, a map which ended where the
UTM zones do had no grid, and a sheet dated in the last year there is could not be made.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* An export is removed when it is over, and its work stops when the API does

The images and the PDF of an export were kept until the next export began, however long that was.  The browser
now has the export removed however it ends, and old exports are removed when a PDF is made too.  The work of a
Job which ended because the API was stopping went on to the end of the PDF, with the API waiting for it.  What
is said of an export, and each image, is written whole.  Text which is no text is taken out when an export
begins, not found by the Job.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* An export stops with its map, and waits for a page which can not be seen

Leaving the map while it was exported left the export going, to fail or to hand a PDF to another page; so did
closing the picker, which is now left open.  A PDF which arrives after the export was stopped is not saved.
The time a page is behind another tab is not counted against a map which is drawing.  The name of a road is
one line.  A photograph which ends a little inside the map is not made a JPEG with a black strip.  An accent
in the name of a quadrangle is not a gap in the name of its file.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* An exported map has its water over its parks

A layer of the PDF was one image, drawn where the topmost of its layers is.  The parks of Land Cover are under
the water on the map and its piers are over it, so all of Land Cover was drawn over the water: a lake in a
park, or among houses, was not on the PDF.  A second basemap file put its water over the first's buildings.

A layer is now as many images as it must be, each where it is on the map, in one layer of the PDF: a part
which fills an area stays where it is, and a part which is only lines is drawn with the next.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* A PDF is not kept from who made it by a server slow to remove the export

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* A map file which is of somewhere else is not drawn when a map is exported

Each basemap file is several images of an exported map, and most files of a library are of other places: each
was an image of nothing, counted against how many images an export may be and how sharp it may be.  A file
whose own edges are outside the quadrangle is now left out before the layers are counted.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* An exported map has its streams (#991)

A basemap draws streams from zoom 14, and service roads and taxiways from 13.  A quadrangle is drawn a little
short of that (zoom 13.8 at 40 degrees, 13.2 at 61), from tiles which have them: none were on the PDF.  What
is drawn from a zoom no deeper than a quadrangle's detail is drawn.  Names are left as they are.

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* An export which is cancelled stops at once, in the middle of a request (#993)

* An export which is cancelled stops at once, in the middle of a request

An export was stopped between two requests.  The image of a layer is tens of megabytes, which is a long request
on a slow connection, and a request to a server which has stopped answering never ends: the export said it was
stopping until then.  Cancel, and leaving the map, now end the request.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* The requests an export makes after its layers are sent are seen to be stopped too

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* A map is exported as it is printed: on white, in dark lines (#996)

* A map is exported as it is printed: on white, in dark lines

The light and dark maps are made for a screen.  On paper their roads are white on pale gray and their streams
half a pixel of pale blue.  The printed map is a third look, of the map's own: white ground, dark road edges
and names, blue streams wide enough to see, paths drawn, brown contours.

It is what a map is exported in unless Light or Dark is chosen where the quadrangle is, and the choice is
remembered in the browser.  While a quadrangle is chosen the map is shown in the look it will be exported in.
The light and dark maps are as they were.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* Changing the look of the map does not move it

What is done when a new style is in place was asked for after the style was set.  A style which only differs
from the one the map has is in place before that, so it was done at the next change of style instead, with
where the map had been at this one: the map went back there when a quadrangle was chosen, and again when it was
not.  It is asked for first, and the map is not moved at all.  An export no longer waits for the map to be
still, which it need not and which nothing ended.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* The colors of the printed map are chosen where its layers are (#997)

* The colors of the printed map are chosen where its layers are

While a quadrangle is chosen and the map is shown as it is printed, each layer of the layer control has its
color beside it, to be changed: one for most, three for the land (what grows, what was built, sand and ice)
and two for names (the name, and what is around it).  A basemap has several colors where one is chosen, which
are shades of it.  The edge of a road stays dark.  What is chosen is remembered in the browser, and can be put
back.  The map on the screen is as it was.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* A ramp on a bridge is the color of a major road, as it is off one

A basemap draws a ramp on a bridge in the color of a minor road on a bridge: on the printed map it was white
where the rest of the ramp is yellow, and followed the color chosen for minor roads.  What puts the colors
back is a button, which a keyboard reaches.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* A layer with nothing in it does not make an exported map less sharp (#1001)

* A layer with nothing in it does not make an exported map less sharp

How sharp a map may be depends on how many images it is, and every group of layers was counted: a basemap
file with no runway, highway or boundary in the quadrangle was images of nothing, drawn, sent, and counted
against the rest.  A map of 23 images asked for at 400 DPI was made at 330.  The groups are now looked at
first, on a small map, and only those with something in them are drawn, as sharp as that many may be.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* An image which arrives after its export was removed is not kept, and one mil is a mil

Stopping an export while an image was on its way left the image behind it, in a directory nothing would
remove for an hour.  An angle of one mil was "1 mils" on the sheet.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* The map which is exported from is made as sharp as it is to be (#1002)

* The map which is exported from is made as sharp as it is to be

The groups of a map are looked at on a small map, which was then made larger to draw them.  A browser lets
a canvas be larger when it is made than it lets one be made larger afterwards: a quadrangle asked for at
400 DPI was 33 million pixels, 288 DPI at 40 degrees, less than before the groups were looked at first.  The
small map is now removed, and the map which is exported from is made in its place.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* A canvas the browser stopped drawing on is not a layer with nothing in it

A canvas whose drawing was taken away by the browser reads as clear, and a group looked at then was left out
of the PDF without a word: the export fails instead.  An export stopped as the last group is looked at does
not make the map it would have been drawn on.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* An exported map says where it is in a projection Adobe Acrobat knows (#1003)

* An exported map says where it is in a projection Adobe Acrobat knows

The page was said to be in Web Mercator, which it is drawn in.  Acrobat does not know Web Mercator, in the
words of the standard or in Esri's: it took the page for a map, and put every place on it at 0, 0.  The page
is now said to be Mercator on the WGS 84 ellipsoid, which it reads, with the same corners.  Between the corners
the two differ by under three centimeters on the ground.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* How well a page is said to be where it is, is measured from what the PDF says

The test of it measured two formulas of its own.  It now reads the corners and the projection from the PDF and
does a reader's sum with them.  The few centimeters are of a quadrangle: a taller page is less well said, by
the square of how tall it is, which the comment now says and a test measures.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

* The help says how a map is exported as a PDF (#1004)

The help of the map is five pages, one of them the PDF export.

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
lrnselfreliance 2026-11-20 17:22:21 -07:00 • committed by Roland
parent d157057d81
commit faf8e735d1
No known key found for this signature in database
GPG key ID: 1BA620CE6C9A4680
42 changed files with 8744 additions and 102 deletions

View file

@ -0,0 +1,581 @@
import io
import json
import math
import re
import subprocess
import sys
from http import HTTPStatus
from unittest import mock
import pymupdf
import pytest
from PIL import Image
from modules.map import export
from modules.map.export import ExportLayer, InvalidExport, build_layered_pdf
from wrolpi.jobs import wait_for_job
WIDTH, HEIGHT = 60, 90
def make_image(color=(200, 80, 20, 255), size=(WIDTH, HEIGHT), image_format='PNG', box=None) -> bytes:
"""An image of a layer: transparent, with `color` in `box` (all of it by default)."""
image = Image.new('RGBA', size, (0, 0, 0, 0))
image.paste(color, box or (0, 0, *size))
if image_format == 'JPEG':
image = image.convert('RGB')
data = io.BytesIO()
image.save(data, image_format)
return data.getvalue()
BLANK = make_image(color=(0, 0, 0, 0))
def layer_names(pdf: bytes) -> list:
doc = pymupdf.open(stream=pdf, filetype='pdf')
return [i['name'] for _, i in sorted(doc.get_ocgs().items())]
def test_layers_are_pdf_layers():
"""Each image of the map is in a PDF layer of its own, named and in the order given, on one page the size
the map is at its DPI."""
pdf = build_layered_pdf([
ExportLayer('Background', make_image((240, 240, 230, 255))),
ExportLayer('Water', make_image((60, 120, 220, 255), box=(0, 0, 30, 90))),
ExportLayer('Satellite', make_image(image_format='JPEG')),
], WIDTH, HEIGHT, dpi=300, title='40°15′N 105°07′30″W')
doc = pymupdf.open(stream=pdf, filetype='pdf')
assert doc.page_count == 1
page = doc[0]
# 60x90 pixels at 300 DPI is 0.2x0.3 inches: 14.4x21.6 points.
assert (round(page.rect.width, 3), round(page.rect.height, 3)) == (14.4, 21.6)
assert layer_names(pdf) == ['Background', 'Water', 'Satellite']
assert all(i['on'] for i in doc.get_ocgs().values())
assert doc.metadata['title'] == '40°15′N 105°07′30″W'
# Every image fills the page, and is the picture it was given.
images = page.get_image_info(xrefs=True)
assert len(images) == 3
for image in images:
assert (image['width'], image['height']) == (WIDTH, HEIGHT)
assert pymupdf.Rect(image['bbox']) == page.rect
# The page looks like the layers stacked: water over the background on the left... but the satellite image
# is on top of both, and is all one color.
pixmap = page.get_pixmap(dpi=300)
assert pixmap.pixel(5, 5)[:3] == pixmap.pixel(55, 85)[:3]
def test_layers_are_stacked_in_order():
"""The first layer is at the bottom: what is above it covers it, and shows it where it is transparent."""
pdf = build_layered_pdf([
ExportLayer('Background', make_image((255, 255, 255, 255))),
ExportLayer('Water', make_image((0, 0, 255, 255), box=(0, 0, 30, 90))),
], WIDTH, HEIGHT, dpi=72)
pixmap = pymupdf.open(stream=pdf, filetype='pdf')[0].get_pixmap(dpi=72)
assert pixmap.pixel(10, 45) == (0, 0, 255)
assert pixmap.pixel(50, 45) == (255, 255, 255)
def test_hidden_layer_is_not_drawn():
"""A layer of the PDF can be turned off, which is the point of there being layers."""
pdf = build_layered_pdf([
ExportLayer('Background', make_image((255, 255, 255, 255))),
ExportLayer('Water', make_image((0, 0, 255, 255))),
], WIDTH, HEIGHT, dpi=72)
doc = pymupdf.open(stream=pdf, filetype='pdf')
assert doc[0].get_pixmap(dpi=72).pixel(10, 45) == (0, 0, 255)
# As a reader does it: clear the layer's box in the list of layers.
water, = [i['number'] for i in doc.layer_ui_configs() if i['text'] == 'Water']
doc.set_layer_ui_config(water, action=2)
assert doc[0].get_pixmap(dpi=72).pixel(10, 45) == (255, 255, 255)
def test_blank_layer_is_left_out():
"""A layer in which nothing was drawn (no tunnels in this quadrangle) is not a layer of the PDF."""
pdf = build_layered_pdf([
ExportLayer('Background', make_image()),
ExportLayer('Roads - Tunnels', BLANK),
ExportLayer('Labels', make_image(box=(1, 1, 2, 2))),
], WIDTH, HEIGHT, dpi=300)
assert layer_names(pdf) == ['Background', 'Labels']
with pytest.raises(InvalidExport, match='Nothing is drawn'):
build_layered_pdf([ExportLayer('Roads - Tunnels', BLANK)], WIDTH, HEIGHT, dpi=300)
@pytest.mark.parametrize('layers,width,height,dpi,error', [
([], WIDTH, HEIGHT, 300, 'no layers'),
([ExportLayer('A', make_image())] * (export.MAX_LAYERS + 1), WIDTH, HEIGHT, 300, 'more than'),
# Not the size the map is said to be.
([ExportLayer('A', make_image(size=(61, 90)))], WIDTH, HEIGHT, 300, '61x90'),
# Not an image, and not a kind of image the map is drawn as.
([ExportLayer('A', b'not an image')], WIDTH, HEIGHT, 300, 'not a readable image'),
([ExportLayer('A', make_image(image_format='GIF'))], WIDTH, HEIGHT, 300, 'not a PNG or JPEG'),
# Sizes no map is.
([ExportLayer('A', make_image())], 0, HEIGHT, 300, '0x90'),
([ExportLayer('A', make_image())], export.MAX_SIDE + 1, 10, 300, 'can not be exported'),
([ExportLayer('A', make_image())], 10000, 10000, 300, 'can not be exported'),
# Each layer is a size a map is, but all of them together are more than there is memory for.
([ExportLayer('A', b'never decoded')] * 30, 6000, 8000, 300, 'too large to export'),
([ExportLayer('A', make_image())], WIDTH, HEIGHT, 0, 'DPI'),
([ExportLayer('A', make_image())], WIDTH, HEIGHT, 5000, 'DPI'),
])
def test_invalid_export(layers, width, height, dpi, error):
with pytest.raises(InvalidExport, match=error):
build_layered_pdf(layers, width, height, dpi)
async def api_export(async_client, images, **meta):
"""Export a map through the API, as the browser does: describe it, send the image of each layer, have the PDF
made, and fetch it. Returns the response which ended it (the PDF's, or the first to refuse) and the Job's
record (None when it did not get as far as a Job)."""
meta = dict(dict(width=WIDTH, height=HEIGHT, dpi=300, layers=[name for name, _ in images]), **meta)
request, response = await async_client.post('/api/map/export', json=meta)
if response.status_code != HTTPStatus.CREATED:
return response, None
export_id = response.json['export_id']
for index, (_, image) in enumerate(images):
request, response = await async_client.post(f'/api/map/export/{export_id}/layer/{index}', content=image)
if response.status_code != HTTPStatus.NO_CONTENT:
return response, None
request, response = await async_client.post(f'/api/map/export/{export_id}/start')
if response.status_code != HTTPStatus.ACCEPTED:
return response, None
job = await wait_for_job(response.json['job_id'], timeout=60)
request, response = await async_client.get(f'/api/map/export/{export_id}/pdf')
return response, job
@pytest.mark.asyncio
async def test_api_export_pdf(async_client):
"""The images of a map's layers are sent, a Job makes the PDF of them, and the PDF is fetched."""
response, job = await api_export(async_client, [
('Background', make_image((240, 240, 230, 255))),
('Roads - Tunnels', BLANK),
('Water', make_image((60, 120, 220, 255), box=(0, 0, 30, 90))),
], title='A quadrangle')
assert job['status'] == 'complete', job
assert job['progress'] == 100
assert job['description'] == 'Map PDF: A quadrangle'
assert response.status_code == HTTPStatus.OK, response.text
assert response.headers['content-type'] == 'application/pdf'
assert response.content.startswith(b'%PDF-')
assert layer_names(response.content) == ['Background', 'Water']
assert pymupdf.open(stream=response.content, filetype='pdf').metadata['title'] == 'A quadrangle'
@pytest.mark.asyncio
async def test_api_export_pdf_invalid(async_client):
"""What is not a map is refused before any image is sent, and why is said."""
# No description of the map.
request, response = await async_client.post('/api/map/export', content=b'not json')
assert response.status_code == HTTPStatus.BAD_REQUEST
assert 'description of the map' in response.json['error']
request, response = await async_client.post('/api/map/export', json=['a', 'list'])
assert response.status_code == HTTPStatus.BAD_REQUEST
# A description which is not of a map.
response, _ = await api_export(async_client, [('A', make_image())], width='wide')
assert response.status_code == HTTPStatus.BAD_REQUEST
# More than a PDF is made of: said at once, with nothing sent.
response, _ = await api_export(async_client, [('A', make_image())], width=10000, height=10000)
assert response.status_code == HTTPStatus.BAD_REQUEST
assert '10000x10000' in response.json['error']
response, _ = await api_export(async_client, [], layers=[])
assert response.status_code == HTTPStatus.BAD_REQUEST
assert 'no layers' in response.json['error']
@pytest.mark.asyncio
async def test_api_export_layers(async_client):
"""Only the images of the map's own layers are taken, and the PDF is not made until all of them are sent."""
meta = dict(width=WIDTH, height=HEIGHT, dpi=300, layers=['A', 'B'])
request, response = await async_client.post('/api/map/export', json=meta)
export_id = response.json['export_id']
layer, start = f'/api/map/export/{export_id}/layer', f'/api/map/export/{export_id}/start'
for index, content, error in (
(2, make_image(), 'no layer 2'),
(0, b'', 'empty'),
(0, b'not an image', 'not a PNG or JPEG'),
(0, make_image(image_format='GIF'), 'not a PNG or JPEG'),
):
request, response = await async_client.post(f'{layer}/{index}', content=content)
assert response.status_code == HTTPStatus.BAD_REQUEST, error
assert error in response.json['error']
# One of two layers.
request, response = await async_client.post(f'{layer}/0', content=make_image())
assert response.status_code == HTTPStatus.NO_CONTENT
request, response = await async_client.post(start)
assert response.status_code == HTTPStatus.CONFLICT
assert '1 layers have not been sent: B' in response.json['error']
request, response = await async_client.get(f'/api/map/export/{export_id}/pdf')
assert response.status_code == HTTPStatus.CONFLICT
assert 'not started' in response.json['error']
request, response = await async_client.post(f'{layer}/1', content=make_image(image_format='JPEG'))
assert response.status_code == HTTPStatus.NO_CONTENT
request, response = await async_client.post(start)
assert response.status_code == HTTPStatus.ACCEPTED
job_id = response.json['job_id']
# Asked again, it is the same Job; and no more images are taken.
request, response = await async_client.post(start)
assert response.json['job_id'] == job_id
request, response = await async_client.post(f'{layer}/0', content=make_image())
assert response.status_code == HTTPStatus.BAD_REQUEST
assert 'already being made' in response.json['error']
# Until the Job is done there is no PDF.
request, response = await async_client.get(f'/api/map/export/{export_id}/pdf')
assert response.status_code == HTTPStatus.CONFLICT
assert (await wait_for_job(job_id, timeout=60))['status'] == 'complete'
request, response = await async_client.get(f'/api/map/export/{export_id}/pdf')
assert response.status_code == HTTPStatus.OK
assert layer_names(response.content) == ['A', 'B']
@pytest.mark.asyncio
async def test_api_export_unknown(async_client):
"""An export which there is not, and an id which is not one."""
for export_id in ('0' * 32, 'nothing', '..%2F..%2Fetc'):
for method, url in (('post', f'/api/map/export/{export_id}/layer/0'),
('post', f'/api/map/export/{export_id}/start'),
('get', f'/api/map/export/{export_id}/pdf')):
request, response = await getattr(async_client, method)(url)
assert response.status_code == HTTPStatus.NOT_FOUND, url
@pytest.mark.asyncio
async def test_api_export_job_fails(async_client):
"""An image which is not of the map fails the Job, which says why; there is no PDF."""
response, job = await api_export(async_client, [('A', make_image()), ('B', make_image(size=(10, 10)))])
assert job['status'] == 'failed'
assert '10x10' in job['error']
assert response.status_code == HTTPStatus.CONFLICT
assert 'failed' in response.json['error']
def test_total_pixels_are_limited_before_decoding():
"""The memory an export needs is the pixels of all its layers, which a small upload can be a great many of:
it is refused without any image being opened."""
with mock.patch('modules.map.export.Image.open') as image_open:
with pytest.raises(InvalidExport, match='too large to export'):
build_layered_pdf([ExportLayer(str(i), BLANK) for i in range(export.MAX_LAYERS)],
export.MAX_SIDE // 4, export.MAX_SIDE // 4, dpi=300)
image_open.assert_not_called()
# A quadrangle at 300 DPI near the equator, with every layer of a full layer control on, is within it.
assert 24 * 5300 * 6900 <= export.MAX_TOTAL_PIXELS
def test_pymupdf_is_not_imported_with_the_module():
"""pymupdf is slow to import, and the API imports this module when it starts."""
code = 'import sys, modules.map.api; print("pymupdf" in sys.modules)'
result = subprocess.run([sys.executable, '-c', code], capture_output=True, text=True, timeout=120)
assert result.returncode == 0, result.stderr
assert result.stdout.strip().splitlines()[-1] == 'False'
def test_images_are_compressed_as_they_are_added():
"""The pixels of a layer are compressed in the PDF when the layer is added, not kept until it is written:
kept, the layers of a quadrangle are more memory than a Raspberry Pi has to spare."""
doc = pymupdf.open()
page = doc.new_page(width=WIDTH, height=HEIGHT)
export._insert_layer(doc, page, page.rect, ExportLayer('Satellite', make_image(image_format='JPEG')), {})
export._insert_layer(doc, page, page.rect, ExportLayer('Water', make_image((0, 0, 255, 200), box=(0, 0, 30, 90))), {})
streams = [xref for xref in range(1, doc.xref_length()) if doc.xref_is_stream(xref)]
images = [xref for xref in streams if doc.xref_get_key(xref, 'Subtype')[1] == '/Image']
# The picture and its transparency, and the photograph.
assert len(images) == 3
assert sorted(doc.xref_get_key(xref, 'Filter')[1] for xref in images) \
== ['/DCTDecode', '/FlateDecode', '/FlateDecode']
# And it is still the picture: blue, mostly opaque, over the left of the photograph.
pixmap = page.get_pixmap(dpi=72, alpha=False)
assert pixmap.pixel(50, 45) != pixmap.pixel(10, 45)
# A quadrangle at Anchorage: west, south, east, north.
ANCHORAGE = [-150.0, 61.125, -149.875, 61.25]
def read_viewport(pdf: bytes) -> dict:
"""What the first page of a PDF says of where it is: its /VP, as the text of each part."""
doc = pymupdf.open(stream=pdf, filetype='pdf')
page = doc[0]
kind, viewports = doc.xref_get_key(page.xref, 'VP')
assert kind == 'array', f'The page has no /VP: {kind} {viewports}'
def numbers(key):
return [float(i) for i in re.search(rf'/{key}\s*\[([^]]*)]', viewports).group(1).split()]
return dict(
text=viewports,
bbox=numbers('BBox'), bounds=numbers('Bounds'), gpts=numbers('GPTS'), lpts=numbers('LPTS'),
page=(page.rect.width, page.rect.height),
)
def test_pdf_says_where_it_is():
"""With the edges of the map, the page has a geospatial viewport: all of the page, with the latitude and
longitude of each of its corners, in a projection every reader knows."""
pdf = build_layered_pdf([ExportLayer('Background', make_image())], WIDTH, HEIGHT, dpi=72,
title='1:24,000 61°07′30″N 150°00′W', bounds=ANCHORAGE)
viewport = read_viewport(pdf)
# All of the page.
assert viewport['bbox'] == [0, 0, WIDTH, HEIGHT]
# The corners, from the lower left of the page: upper left, lower left, lower right, upper right...
assert viewport['lpts'] == viewport['bounds'] == [0, 1, 0, 0, 1, 0, 1, 1]
# ...and where each is: latitude, then longitude.
assert viewport['gpts'] == [61.25, -150, 61.125, -150, 61.125, -149.875, 61.25, -149.875]
text = viewport['text']
assert '/Type/Viewport' in text.replace(' ', '') and '/Subtype/GEO' in text.replace(' ', '')
# Mercator on the ellipsoid, in Esri's words: Adobe Acrobat puts every place of a page said to be in Web
# Mercator at 0, 0.
assert '/Type/PROJCS' in text.replace(' ', '') and '/EPSG 3395' in text
assert 'PROJCS["WGS_1984_World_Mercator"' in text and 'PROJECTION["Mercator"]' in text
assert '3857' not in text and 'Auxiliary_Sphere' not in text and 'Mercator_1SP' not in text
# It is still the PDF it was.
assert layer_names(pdf) == ['Background']
assert pymupdf.open(stream=pdf, filetype='pdf')[0].get_pixmap(dpi=72).pixel(5, 5) == (200, 80, 20)
def test_corner_of_page_is_corner_of_map():
"""A reader finds where a point of the page is from the corners: here, as a reader does it, in the
projection the page says it is in."""
pdf = build_layered_pdf([ExportLayer('Background', make_image())], WIDTH, HEIGHT, dpi=72, bounds=ANCHORAGE)
viewport = read_viewport(pdf)
def mercator(latitude, longitude):
return (math.radians(longitude) * 6378137,
math.log(math.tan(math.pi / 4 + math.radians(latitude) / 2)) * 6378137)
def unmercator(x, y):
return math.degrees(2 * math.atan(math.exp(y / 6378137)) - math.pi / 2), math.degrees(x / 6378137)
lpts, gpts = viewport['lpts'], viewport['gpts']
corners = {(lpts[i], lpts[i + 1]): mercator(gpts[i], gpts[i + 1]) for i in range(0, 8, 2)}
def where(across, up):
"""The latitude and longitude of a point of the page (0 to 1 from its lower left)."""
(x0, y0), (x1, y1) = corners[(0, 0)], corners[(1, 1)]
return unmercator(x0 + across * (x1 - x0), y0 + up * (y1 - y0))
assert where(0, 0) == pytest.approx((61.125, -150))
assert where(1, 1) == pytest.approx((61.25, -149.875))
# The middle of the page is the middle of the map in longitude, and north of its middle in latitude: the
# further north, the more of the page a degree is.
latitude, longitude = where(0.5, 0.5)
assert longitude == pytest.approx(-149.9375)
assert 61.1875 < latitude < 61.1877
# The other two corners are the corners of the same rectangle.
assert corners[(0, 1)] == pytest.approx((corners[(0, 0)][0], corners[(1, 1)][1]))
assert corners[(1, 0)] == pytest.approx((corners[(1, 1)][0], corners[(0, 0)][1]))
def test_pdf_without_edges_says_nothing_of_where_it_is():
pdf = build_layered_pdf([ExportLayer('Background', make_image())], WIDTH, HEIGHT, dpi=72)
doc = pymupdf.open(stream=pdf, filetype='pdf')
assert doc.xref_get_key(doc[0].xref, 'VP')[0] == 'null'
def test_viewport_name():
"""The viewport is named as the map is; a name which is not ASCII is written so a PDF can hold it."""
viewport = export.geospatial_viewport(10, 10, ANCHORAGE, 'A quadrangle (of a map)')
assert '/Name (A quadrangle \\(of a map\\))' in viewport
assert '/Name' not in export.geospatial_viewport(10, 10, ANCHORAGE)
pdf = build_layered_pdf([ExportLayer('Background', make_image())], WIDTH, HEIGHT, dpi=72,
title='61°07′30″N', bounds=ANCHORAGE)
name = re.search(r'/Name\s*<([0-9A-Fa-f]+)>', read_viewport(pdf)['text']).group(1)
assert bytes.fromhex(name).decode('utf-16') == '61°07′30″N'
@pytest.mark.parametrize('bounds', [
[-150, 61.125, -149.875], # Three edges.
'here',
[-150, 61.125, -149.875, 'north'],
[-149.875, 61.125, -150, 61.25], # East of its east.
[-150, 61.25, -149.875, 61.125], # North of its north.
[-150, 61.125, -150, 61.25], # No width.
[-190, 61.125, -149.875, 61.25], # Off the world.
[179.875, 0, 180.00001, 0.125], # Across the antimeridian, if not by much.
[179.9, 0, 180.1, 0.125],
[-180.00001, 0, -179.875, 0.125],
[-150, 61.125, -149.875, 89], # Further north than the map goes.
[-150, float('nan'), -149.875, 61.25],
])
def test_invalid_bounds(bounds):
with pytest.raises(InvalidExport, match='edges'):
build_layered_pdf([ExportLayer('Background', make_image())], WIDTH, HEIGHT, dpi=72, bounds=bounds)
@pytest.mark.parametrize('bounds,expected', [
# The map's own arithmetic puts the edge of the world a hair past where it is.
([179.875, 0, 180.00000000000125, 0.125], [179.875, 0, 180, 0.125]),
([-180.00000000000125, 0, -179.875, 0.125], [-180, 0, -179.875, 0.125]),
([-180, 0, -179.875, 0.125], [-180, 0, -179.875, 0.125]),
])
def test_bounds_at_the_antimeridian(bounds, expected):
"""A map whose edge is the antimeridian is exported, with that edge at 180 degrees and not at -180."""
pdf = build_layered_pdf([ExportLayer('Background', make_image())], WIDTH, HEIGHT, dpi=72, bounds=bounds)
west, south, east, north = expected
assert read_viewport(pdf)['gpts'] == [north, west, south, west, south, east, north, east]
@pytest.mark.asyncio
async def test_api_export_pdf_georeferenced(async_client):
"""The edges of the map are sent with its description, and the PDF says where it is."""
images = [('Background', make_image())]
response, job = await api_export(async_client, images, bounds=ANCHORAGE)
assert response.status_code == HTTPStatus.OK, response.text
assert read_viewport(response.content)['gpts'] == [61.25, -150, 61.125, -150, 61.125, -149.875, 61.25, -149.875]
# Without them it is a PDF all the same.
response, job = await api_export(async_client, images)
assert response.status_code == HTTPStatus.OK
# Edges which are not those of a map are refused.
response, job = await api_export(async_client, images, bounds=[0, 0, 0, 0])
assert response.status_code == HTTPStatus.BAD_REQUEST
assert 'edges' in response.json['error']
@pytest.mark.asyncio
async def test_api_export_text_which_is_not_text(async_client):
"""What is sent as the title of a map, or the name of a layer, may have in it what is no character (half of
a pair, a NUL). It is taken out when the export begins: the Job, after every image is sent, is too late to
find that a PDF can not have it."""
body = json.dumps(dict(width=WIDTH, height=HEIGHT, dpi=300, title='A \ud800 map\x00', layers=['Ro\x00ads \ud800']))
request, response = await async_client.post('/api/map/export', content=body)
assert response.status_code == HTTPStatus.CREATED, response.json
export_id = response.json['export_id']
request, response = await async_client.post(f'/api/map/export/{export_id}/layer/0', content=make_image())
assert response.status_code == HTTPStatus.NO_CONTENT
request, response = await async_client.post(f'/api/map/export/{export_id}/start')
job = await wait_for_job(response.json['job_id'], timeout=60)
assert job['status'] == 'complete', job
request, response = await async_client.get(f'/api/map/export/{export_id}/pdf')
assert layer_names(response.content) == ['Roads ?']
assert pymupdf.open(stream=response.content).metadata['title'] == 'A ? map'
def test_images_of_one_name_are_one_pdf_layer():
"""A layer of the map may be several images, with other layers drawn between them (the parks of the land
are under the water, and its piers are over it). They are one layer of the PDF, and each is drawn where it
was given: turning the layer off hides them all."""
pdf = build_layered_pdf([
ExportLayer('Land Cover', make_image((0, 200, 0, 255))),
ExportLayer('Water', make_image((0, 0, 255, 255), box=(0, 0, 30, 90))),
ExportLayer('Land Cover', make_image((90, 90, 90, 255), box=(0, 0, 10, 90))),
], WIDTH, HEIGHT, dpi=300)
assert layer_names(pdf) == ['Land Cover', 'Water']
doc = pymupdf.open(stream=pdf, filetype='pdf')
page = doc[0]
assert len(page.get_images()) == 3
def color(x):
pixmap = page.get_pixmap(dpi=300)
return pixmap.pixel(x, 45)
# From the left: the pier over the water, the water over the park, the park.
assert [color(x) for x in (5, 20, 50)] == [(90, 90, 90), (0, 0, 255), (0, 200, 0)]
def _wkt_parameter(wkt: str, name: str) -> float:
return float(re.search(rf'PARAMETER\["{name}",([-0-9.]+)]', wkt).group(1))
def _read_as_a_reader_does(viewport: dict, wkt: str, latitude: float, longitude: float):
"""Where on the page (across and up, 0 to 1) a reader of the PDF puts a place: it projects the corners it
is given (/GPTS) in the projection it is told of (the WKT: Mercator on its spheroid), and the place, and puts
the place between the corners as they are on the page (/LPTS)."""
assert 'PROJECTION["Mercator"]' in wkt
a, inverse_flattening = [float(i) for i in re.search(r'SPHEROID\["[^"]*",([0-9.]+),([0-9.]+)]', wkt).groups()]
f = 1 / inverse_flattening
e = math.sqrt(f * (2 - f))
meridian = _wkt_parameter(wkt, 'Central_Meridian')
# Mercator of one standard parallel: the scale is that of the parallel.
parallel = math.radians(_wkt_parameter(wkt, 'Standard_Parallel_1'))
k = math.cos(parallel) / math.sqrt(1 - (e * math.sin(parallel)) ** 2)
def project(lat, lon):
phi = math.radians(lat)
y = math.log(math.tan(math.pi / 4 + phi / 2) * ((1 - e * math.sin(phi)) / (1 + e * math.sin(phi))) ** (e / 2))
return (a * k * math.radians(lon - meridian) + _wkt_parameter(wkt, 'False_Easting'),
a * k * y + _wkt_parameter(wkt, 'False_Northing'))
gpts, lpts = viewport['gpts'], viewport['lpts']
corners = [(project(gpts[i], gpts[i + 1]), (lpts[i], lpts[i + 1])) for i in range(0, 8, 2)]
# The page is a rectangle of the projection, so three corners say where the rest of it is.
(upper_left, page_ul), (lower_left, page_ll), (lower_right, page_lr), _ = corners
x, y = project(latitude, longitude)
across = (x - lower_left[0]) / (lower_right[0] - lower_left[0])
up = (y - lower_left[1]) / (upper_left[1] - lower_left[1])
return (page_ll[0] + across * (page_lr[0] - page_ll[0]), page_ll[1] + up * (page_ul[1] - page_ll[1]))
def _drawn(bounds, latitude: float, longitude: float):
"""Where on the page (across and up, 0 to 1) the map has a place: it is drawn in Web Mercator."""
west, south, east, north = bounds
def y(lat):
return math.log(math.tan(math.pi / 4 + math.radians(lat) / 2))
return (longitude - west) / (east - west), (y(latitude) - y(south)) / (y(north) - y(south))
@pytest.mark.parametrize('west,south', [(-78.5, 0), (-100, 20), (-105.125, 40.25), (-149.625, 61.25), (-156.75, 71.25),
(151.125, -33.875), (10, 84.875), (-0.0625, -0.0625)])
def test_page_is_where_the_projection_it_is_said_to_be_in_puts_it(west, south):
"""The page is drawn in Web Mercator (a sphere), and a reader is told it is Mercator on the ellipsoid, with
the latitude and longitude of its corners. A reader puts a place on the page by where that projection has
it between the corners: that is within 5 centimeters on the ground of where the map drew it, anywhere on a
quadrangle, at any latitude. What the reader is told is what the PDF says: its corners and its WKT."""
bounds = [west, south, west + 0.125, south + 0.125]
pdf = build_layered_pdf([ExportLayer('Background', make_image())], WIDTH, HEIGHT, dpi=72, bounds=bounds)
viewport = read_viewport(pdf)
wkt = re.search(r'/WKT\s*\((.*)\)\s*>>', viewport['text'], re.S).group(1)
# The page is this wide and this tall on the ground.
height = 6378137 * math.radians(0.125)
width = height * math.cos(math.radians(south + 0.0625))
worst = 0
for i in range(21):
for j in range(21):
lat, lon = south + 0.125 * i / 20, west + 0.125 * j / 20
read = _read_as_a_reader_does(viewport, wkt, lat, lon)
drawn = _drawn(bounds, lat, lon)
worst = max(worst, math.hypot((read[0] - drawn[0]) * width, (read[1] - drawn[1]) * height))
assert worst < 0.05
# The same sum finds a page whose corners are not where they are said to be: here, a second of arc south.
moved = dict(viewport, gpts=[v - 1 / 3600 if index % 2 == 0 else v for index, v in enumerate(viewport['gpts'])])
middle = (south + 0.0625, west + 0.0625)
off = _read_as_a_reader_does(moved, wkt, *middle)[1] - _drawn(bounds, *middle)[1]
assert abs(off) * height == pytest.approx(31, abs=1)
def test_page_taller_than_a_quadrangle_is_less_well_said():
"""What is left between the two projections grows with the square of how tall the page is: centimeters on a
quadrangle, more than a meter on a page a degree tall. (A page that tall is not exported yet.)"""
def worst(tall):
bounds = [-150, 61, -149, 61 + tall]
viewport = dict(gpts=[bounds[3], bounds[0], bounds[1], bounds[0], bounds[1], bounds[2], bounds[3], bounds[2]],
lpts=[0, 1, 0, 0, 1, 0, 1, 1])
height = 6378137 * math.radians(tall)
return max(abs(_read_as_a_reader_does(viewport, export.PAGE_PROJECTION_WKT, 61 + tall * i / 40, -149.5)[1]
- _drawn(bounds, 61 + tall * i / 40, -149.5)[1]) * height for i in range(41))
assert worst(0.125) < 0.05
assert 1 < worst(1) < 2

View file

@ -0,0 +1,422 @@
import asyncio
import json
import os
import stat
import threading
import time
from http import HTTPStatus
import pytest
from modules.map import export, export_job
from modules.map.test.test_export import make_image
from wrolpi import jobs
WIDTH, HEIGHT = 60, 90
def new_export(*names, **kwargs) -> str:
return export_job.create_export(list(names), WIDTH, HEIGHT, 300, **kwargs)
def test_create_export(map_export_root):
"""An export is a directory which holds what was said of the map."""
export_id = new_export('Background', 'Water', title='A quadrangle', bounds=[-150, 61.125, -149.875, 61.25],
sheet=export.Sheet(name='A quadrangle', credits=['Somebody'], date='2026-11-15'))
assert export_job.EXPORT_ID.match(export_id)
meta = json.loads((map_export_root / export_id / 'meta.json').read_text())
assert meta['layers'] == ['Background', 'Water']
assert meta['sheet'] == dict(name='A quadrangle', credits=['Somebody'], date='2026-11-15',
contour_interval=None, contour_unit=None, legend=None)
assert meta['job_id'] is None
# A map which can not be exported is not begun.
with pytest.raises(export.InvalidExport, match='no layers'):
new_export()
with pytest.raises(export.InvalidExport, match='edges of the map are needed'):
new_export('A', sheet=export.Sheet())
assert len(list(map_export_root.iterdir())) == 1
@pytest.mark.parametrize('export_id', ['0' * 32, '', 'nothing', '../' + '0' * 29, '0' * 31 + '/', None])
def test_unknown_export(export_id, map_export_root):
"""An id which was not given by an export is of no export, whatever it would be as a path."""
map_export_root.mkdir()
(map_export_root.parent / 'meta.json').write_text('{}')
with pytest.raises(export_job.ExportNotFound):
export_job.save_layer(export_id, 0, make_image())
with pytest.raises(export_job.ExportNotFound):
export_job.get_export_pdf(export_id)
def age(directory, seconds=export_job.EXPORT_KEEP_SECONDS + 60):
"""Make an export look as old as that."""
then = time.time() - seconds
os.utime(directory, (then, then))
def test_remove_old_exports(map_export_root, monkeypatch):
"""An export is removed when it is old; a new export is when that is looked for."""
old, new = new_export('A'), new_export('A')
export_job.save_layer(old, 0, make_image())
age(map_export_root / old)
(map_export_root / 'a file which is not an export').write_text('left alone')
newest = new_export('A')
assert sorted(i.name for i in map_export_root.iterdir() if i.is_dir()) == sorted([new, newest])
assert (map_export_root / 'a file which is not an export').is_file()
with pytest.raises(export_job.ExportNotFound):
export_job.start_export(old)
# Nothing to remove, and nowhere to look, are not errors; and looking does not make the directory.
export_job.remove_old_exports()
monkeypatch.setattr(export_job, 'EXPORT_ROOT', map_export_root / 'not made yet')
export_job.remove_old_exports()
assert not (map_export_root / 'not made yet').exists()
def test_only_exports_are_removed(map_export_root, tmp_path):
"""What is old in the directory of exports, and is not an export, is left: a directory by another name, and a
link by an export's name to a directory somewhere else."""
new_export('A')
other = map_export_root / 'someone-elses-directory'
other.mkdir()
(other / 'file').write_text('kept')
elsewhere = tmp_path / 'elsewhere'
elsewhere.mkdir()
(elsewhere / 'precious').write_text('kept')
link = map_export_root / ('f' * 32)
link.symlink_to(elsewhere, target_is_directory=True)
for path in (other, elsewhere):
age(path)
export_job.remove_old_exports()
assert (other / 'file').read_text() == 'kept'
assert (elsewhere / 'precious').read_text() == 'kept'
# And a link is not an export.
with pytest.raises(export_job.ExportNotFound):
export_job.save_layer('f' * 32, 0, make_image())
def test_export_directory_is_this_users(map_export_root, tmp_path, monkeypatch):
"""Exports are kept in a directory made for this user alone, and never through a link: anyone may make things
in /var/tmp, and a link there would have exports written, and old ones removed, wherever it leads."""
export_id = new_export('A')
assert stat.S_IMODE(map_export_root.stat().st_mode) == 0o700
assert stat.S_IMODE((map_export_root / export_id).stat().st_mode) == 0o700
# One which others could read is closed to them when it is next used.
os.chmod(map_export_root, 0o755)
new_export('A')
assert stat.S_IMODE(map_export_root.stat().st_mode) == 0o700
# A link where the directory should be.
elsewhere = tmp_path / 'elsewhere'
(elsewhere / ('a' * 32)).mkdir(parents=True)
age(elsewhere / ('a' * 32))
linked = tmp_path / 'linked-root'
linked.symlink_to(elsewhere, target_is_directory=True)
monkeypatch.setattr(export_job, 'EXPORT_ROOT', linked)
with pytest.raises(export_job.ExportStorageError):
new_export('A')
with pytest.raises(export_job.ExportStorageError):
export_job.remove_old_exports()
with pytest.raises(export_job.ExportStorageError):
export_job.save_layer('a' * 32, 0, make_image())
# Nothing was made, or removed, where the link leads.
assert [i.name for i in elsewhere.iterdir()] == ['a' * 32]
# A file where the directory should be.
not_a_directory = tmp_path / 'a-file'
not_a_directory.write_text('')
monkeypatch.setattr(export_job, 'EXPORT_ROOT', not_a_directory)
with pytest.raises(export_job.ExportStorageError):
new_export('A')
@pytest.mark.asyncio
async def test_exports_of_waiting_jobs_are_kept(async_client, map_export_root):
"""An export whose Job is waiting is kept however old it is: it may wait behind another Job for longer than an
export is kept, and its images are what the Job is to work from. One whose Job was cancelled is removed at
once."""
waiting, cancelled, finished = new_export('A'), new_export('A'), new_export('A')
for export_id in (waiting, cancelled, finished):
export_job.save_layer(export_id, 0, make_image())
await jobs.wait_for_job(export_job.start_export(finished), timeout=60)
# Queued, and not run: nothing takes Jobs from the queue in a test until one is waited for.
waiting_job = export_job.start_export(waiting)
jobs.cancel_job(export_job.start_export(cancelled))
for export_id in (waiting, finished):
age(map_export_root / export_id)
export_job.remove_old_exports()
assert sorted(i.name for i in map_export_root.iterdir()) == [waiting]
assert (map_export_root / waiting / 'layer-000').is_file()
# It is still there for its Job, which makes the PDF.
assert (await jobs.wait_for_job(waiting_job, timeout=60))['status'] == 'complete'
assert export_job.get_export_pdf(waiting).is_file()
@pytest.mark.asyncio
async def test_remove_export(async_client, map_export_root):
"""An export which is stopped is removed, with the images which were sent, whatever step it was at."""
# Before its PDF was asked for.
export_id = new_export('A', 'B')
export_job.save_layer(export_id, 0, make_image())
request, response = await async_client.delete(f'/api/map/export/{export_id}')
assert response.status_code == HTTPStatus.NO_CONTENT
assert not (map_export_root / export_id).exists()
request, response = await async_client.delete(f'/api/map/export/{export_id}')
assert response.status_code == HTTPStatus.NOT_FOUND
# While its Job waited: the Job is cancelled, and never runs.
export_id = new_export('A')
export_job.save_layer(export_id, 0, make_image())
job_id = export_job.start_export(export_id)
export_job.remove_export(export_id)
assert not (map_export_root / export_id).exists()
assert (await jobs.wait_for_job(job_id, timeout=60))['status'] == 'cancelled'
# After its PDF was made.
export_id = new_export('A')
export_job.save_layer(export_id, 0, make_image())
await jobs.wait_for_job(export_job.start_export(export_id), timeout=60)
export_job.remove_export(export_id)
assert not (map_export_root / export_id).exists()
with pytest.raises(export_job.ExportNotFound):
export_job.remove_export('0' * 32)
@pytest.mark.asyncio
async def test_job_makes_the_pdf(async_client, map_export_root):
"""The Job makes the PDF from the images, says how far along it is, and removes the images."""
export_id = new_export('Background', 'Water', title='A quadrangle')
export_job.save_layer(export_id, 0, make_image())
export_job.save_layer(export_id, 1, make_image((0, 0, 255, 255), image_format='JPEG'))
directory = map_export_root / export_id
assert sorted(i.name for i in directory.iterdir()) == ['layer-000', 'layer-001', 'meta.json']
job_id = export_job.start_export(export_id)
assert job_id.startswith('map_export_pdf-')
job = await jobs.wait_for_job(job_id, timeout=60)
assert job['status'] == 'complete', job
assert job['progress'] == 100
assert job['result']['export_id'] == export_id
pdf = export_job.get_export_pdf(export_id)
assert pdf.read_bytes().startswith(b'%PDF-')
assert job['result']['size'] == pdf.stat().st_size
# Only what was said of the map, and the PDF, are kept.
assert sorted(i.name for i in directory.iterdir()) == ['map.pdf', 'meta.json']
@pytest.mark.asyncio
async def test_job_progress(async_client, map_export_root, monkeypatch):
"""The Job says how far along it is as each layer is looked at and then put in the PDF."""
seen = []
real_set_progress = jobs.JobContext.set_progress
def set_progress(self, percent):
seen.append(percent)
real_set_progress(self, percent)
monkeypatch.setattr(jobs.JobContext, 'set_progress', set_progress)
export_id = new_export('A', 'B')
for index in range(2):
export_job.save_layer(export_id, index, make_image())
await jobs.wait_for_job(export_job.start_export(export_id), timeout=60)
# And, with every step done, before and after the PDF is written: when the work may still be stopped.
assert seen == [0, 25, 50, 75, 100, 100, 100]
@pytest.mark.asyncio
async def test_job_fails(async_client, map_export_root):
"""A map which can not be made into a PDF fails the Job; its images are removed all the same."""
export_id = new_export('A', 'B')
export_job.save_layer(export_id, 0, make_image())
export_job.save_layer(export_id, 1, make_image(size=(10, 10)))
job = await jobs.wait_for_job(export_job.start_export(export_id), timeout=60)
assert job['status'] == 'failed'
assert '10x10' in job['error']
assert sorted(i.name for i in (map_export_root / export_id).iterdir()) == ['meta.json']
with pytest.raises(export_job.ExportNotReady, match='failed'):
export_job.get_export_pdf(export_id)
@pytest.mark.asyncio
async def test_job_cancelled(async_client, map_export_root, monkeypatch):
"""A cancelled Job stops between two layers, makes no PDF, and removes the images. It is not over until the
work has stopped: the next Job is not to start beside it."""
export_id = new_export('A', 'B', 'C')
for index in range(3):
export_job.save_layer(export_id, index, make_image())
job_id = export_job.start_export(export_id)
# The work is held while its first layer is looked at.
real_check = export._check_layer
checked, looking, go_on = [], threading.Event(), threading.Event()
def check_layer(layer, width, height):
checked.append(layer.name)
looking.set()
assert go_on.wait(30)
return real_check(layer, width, height)
monkeypatch.setattr(export, '_check_layer', check_layer)
waiting = asyncio.ensure_future(jobs.wait_for_job(job_id, timeout=60))
for _ in range(200):
if looking.is_set():
break
await asyncio.sleep(0.02)
assert looking.is_set()
# Cancelled, as from another process. (Not `cancel_job` here: it logs while it holds the Jobs' lock, and
# this test's log is written to the running Job, which takes that lock.)
jobs._update_job(job_id, cancel_requested=True)
await asyncio.sleep(0.5)
# The work has not stopped, so neither has the Job: its images are still there.
assert jobs.get_job(job_id)['status'] == 'running'
assert not waiting.done()
assert (map_export_root / export_id / 'layer-000').is_file()
go_on.set()
job = await waiting
assert job['status'] == 'cancelled'
# When the Job is over, so is the work: one layer was looked at, and the images are gone.
assert checked == ['A']
assert sorted(i.name for i in (map_export_root / export_id).iterdir()) == ['meta.json']
with pytest.raises(export_job.ExportNotReady, match='cancelled'):
export_job.get_export_pdf(export_id)
@pytest.mark.asyncio
async def test_job_cancelled_as_the_pdf_is_written(async_client, map_export_root, monkeypatch):
"""A Job cancelled after its last layer, while the sheet is drawn or the PDF written, makes no PDF either."""
export_id = new_export('A')
export_job.save_layer(export_id, 0, make_image())
job_id = export_job.start_export(export_id)
real_insert = export._insert_layer
def insert_layer(*args):
real_insert(*args)
jobs._update_job(job_id, cancel_requested=True)
monkeypatch.setattr(export, '_insert_layer', insert_layer)
job = await jobs.wait_for_job(job_id, timeout=60)
assert job['status'] == 'cancelled'
assert sorted(i.name for i in (map_export_root / export_id).iterdir()) == ['meta.json']
def test_layers_are_read_one_at_a_time(tmp_path):
"""A layer in a file is read when it is needed, not when the map is described."""
path = tmp_path / 'layer'
layer = export.ExportLayer('A', path=path)
path.write_bytes(make_image())
assert layer.read() == make_image()
assert export.ExportLayer('A', image=b'given').read() == b'given'
pdf = export.build_layered_pdf([layer], WIDTH, HEIGHT, 300)
assert pdf.startswith(b'%PDF-')
def test_progress_may_stop_the_work():
"""What is told how far along the PDF is may stop it."""
calls = []
def progress(done, steps):
calls.append((done, steps))
if done == 3:
raise RuntimeError('enough')
layers = [export.ExportLayer(str(i), make_image()) for i in range(3)]
with pytest.raises(RuntimeError, match='enough'):
export.build_layered_pdf(layers, WIDTH, HEIGHT, 300, progress=progress)
assert calls == [(0, 6), (1, 6), (2, 6), (3, 6)]
class NeverCancelled:
"""A Job which was not cancelled, as one is which ends because the API is stopping."""
@staticmethod
def cancel_requested() -> bool:
return False
@staticmethod
def set_progress(_: float):
pass
def test_work_stops_when_it_is_told_to(map_export_root):
"""The work of a Job which ended without being cancelled (the API is stopping) stops between two layers when
it is told to: the API waits for it, and is not to wait for all of a PDF."""
export_id = new_export('A', 'B')
for index in range(2):
export_job.save_layer(export_id, index, make_image())
directory = map_export_root / export_id
stop = threading.Event()
stop.set()
with pytest.raises(export_job.ExportCancelled):
export_job._make_pdf(directory, NeverCancelled, stop)
assert sorted(i.name for i in directory.iterdir()) == ['meta.json']
# Not told to, it makes the PDF.
export_id = new_export('A')
export_job.save_layer(export_id, 0, make_image())
export_job._make_pdf(map_export_root / export_id, NeverCancelled, threading.Event())
assert sorted(i.name for i in (map_export_root / export_id).iterdir()) == ['map.pdf', 'meta.json']
def test_files_of_an_export_are_written_whole(map_export_root, monkeypatch):
"""What is said of an export, and each image, is all there or is what was there before: another request
may read it while it is written. A write which fails leaves nothing behind."""
export_id = new_export('A')
directory = map_export_root / export_id
before = (directory / 'meta.json').read_text()
def replace(*_):
raise OSError('The disk is full')
with monkeypatch.context() as patch:
patch.setattr(export_job.os, 'replace', replace)
with pytest.raises(OSError):
export_job.save_layer(export_id, 0, make_image())
with pytest.raises(OSError):
export_job._write_meta(directory, {'layers': []})
assert sorted(i.name for i in directory.iterdir()) == ['meta.json']
assert (directory / 'meta.json').read_text() == before
export_job.save_layer(export_id, 0, make_image())
assert sorted(i.name for i in directory.iterdir()) == ['layer-000', 'meta.json']
@pytest.mark.asyncio
async def test_job_removes_old_exports(async_client, map_export_root):
"""Old exports are removed when a PDF is made, as well as when an export begins: the last export of a day
is not to keep the one before it."""
old = new_export('A')
export_id = new_export('A')
age(map_export_root / old)
export_job.save_layer(export_id, 0, make_image())
job = await jobs.wait_for_job(export_job.start_export(export_id), timeout=60)
assert job['status'] == 'complete'
assert sorted(i.name for i in map_export_root.iterdir()) == [export_id]
def test_image_which_arrives_after_its_export_is_removed(map_export_root, monkeypatch):
"""An export may be removed while the image of a layer is on its way (the person exporting stopped it, and
the request was already here). The image is not kept: nothing would ever use it, or remove it soon."""
export_id = new_export('A', 'B')
directory = map_export_root / export_id
real_write = export_job._write_whole
def write_whole(path, contents):
# Removed, as by another request, between this image being let in and being written.
if path.name.startswith('layer-'):
export_job.remove_export(export_id)
path.parent.mkdir()
real_write(path, contents)
monkeypatch.setattr(export_job, '_write_whole', write_whole)
with pytest.raises(export_job.ExportNotFound):
export_job.save_layer(export_id, 0, make_image())
assert not directory.exists()
assert list(map_export_root.iterdir()) == []

View file

@ -0,0 +1,50 @@
import pytest
from modules.map import magnetic
# From the test values published with the model: year, altitude (km), latitude, longitude, declination, and the
# field north, east and down (nT).
TEST_VALUES = [
(2025.0, 28, 89, -121, -99.77, -255.388723, -1482.460628, 56194.288771),
(2025.0, 48, 80, -96, -29.91, 1875.982280, -1079.269389, 55623.044051),
(2025.0, 54, 82, 87, 54.89, 1324.336929, 1883.428620, 56740.772059),
(2025.0, 65, 43, 93, 0.50, 24299.852822, 210.517066, 50037.923998),
(2025.0, 51, -33, 109, -5.49, 21737.778822, -2090.274098, -52710.003920),
(2025.0, 39, -59, -8, -15.75, 14358.095523, -4049.107540, -24389.086374),
(2025.0, 3, -50, -103, 27.96, 19526.532799, 10362.990980, -31437.562789),
(2025.0, 94, -29, -110, 15.74, 23275.471080, 6559.046509, -19063.605287),
(2025.0, 66, 14, 143, -0.19, 35003.441364, -116.624818, 7966.315182),
(2025.0, 18, 0, 21, 1.29, 29274.811882, 659.800118, -14316.722540),
(2025.5, 6, -36, -137, 20.28, 23781.930678, 8786.698927, -32577.518648),
(2025.5, 63, 26, 81, 0.51, 34802.613433, 308.431881, 30332.056989),
]
def test_load_model():
model = magnetic.load_model()
assert (model.epoch, model.name, model.degree) == (2025.0, 'WMM-2025', 12)
# Every harmonic of degrees 1 to 12.
assert len(model.coefficients) == sum(n + 1 for n in range(1, 13))
assert model.coefficients[0] == (1, 0, -29351.8, 0.0, 12.0, 0.0)
assert model.coefficients[-1] == (12, 12, -0.7, 0.2, -0.1, -0.1)
@pytest.mark.parametrize('year,altitude,latitude,longitude,declination,north,east,down', TEST_VALUES)
def test_field(year, altitude, latitude, longitude, declination, north, east, down):
"""The field is what the model's own test values say it is."""
model = magnetic.load_model()
assert model.field(latitude, longitude, year, altitude) == pytest.approx((north, east, down), abs=0.01)
assert model.declination(latitude, longitude, year, altitude) == pytest.approx(declination, abs=0.006)
def test_declination():
"""A compass in Anchorage points well east of north; in Maine, west."""
assert 13 < magnetic.declination(61.19, -149.94, 2026.87) < 16
assert -17 < magnetic.declination(45.0, -69.0, 2026.87) < -13
def test_declination_is_not_guessed():
"""There is no declination in a year the model is not of."""
assert magnetic.declination(61.19, -149.94, 2024.99) is None
assert magnetic.declination(61.19, -149.94, 2030.0) is None
assert magnetic.declination(61.19, -149.94, 2029.99) is not None

View file

@ -0,0 +1,112 @@
from http import HTTPStatus
import pytest
from modules.map import quads
def test_load_index():
souths, rows = quads.load_index()
assert souths == sorted(souths) == sorted(rows)
count = 0
for south in souths:
wests, cells = quads._row(south)
assert wests == sorted(wests) == [i[0] for i in cells]
count += len(cells)
assert count > 65_000
@pytest.mark.parametrize('west,south,name,state', [
(-105.125, 40.25, 'Berthoud', 'Colorado'),
(-108.5, 37.75, 'South Mountain', 'Colorado'),
(-76.375, 36.375, 'South Mills', 'North Carolina'),
(-95.25, 37.375, 'South Mound', 'Kansas'),
])
def test_quad_name(west, south, name, state):
"""A quadrangle of the grid has the name the USGS gives it."""
assert quads.quad_name(west, south, west + 0.125, south + 0.125) == quads.QuadName(name, state, True)
# Asked with the edges the map has, which are not quite those.
assert quads.quad_name(west + 1e-7, south - 1e-7, west + 0.125, south + 0.125).exact
def test_quad_name_in_alaska():
"""A quadrangle in Alaska is wider than 7.5 minutes, so an area 7.5 minutes each way is a part of one."""
assert quads.quad_name(-150, 61.125, -149.875, 61.25) == quads.QuadName('Anchorage A-8 NW', 'Alaska', False)
# All of it, by its own edges.
assert quads.quad_name(-150, 61.125, -149.8125, 61.25) == quads.QuadName('Anchorage A-8 NW', 'Alaska', True)
# The area east of the first is the rest of that quadrangle, and the start of the next.
assert quads.quad_name(-149.875, 61.125, -149.75, 61.25).name == 'Anchorage A-8 NE'
@pytest.mark.parametrize('west,south,name,state', [
# A quadrangle 11.25 minutes wide, which ends at the middle of this area: there is none east of it.
(-153.25, 59.0, 'Iliamna A-1 SW', 'Alaska'),
(-151.375, 59.0, 'Seldovia A-4 SW', 'Alaska'),
# A quadrangle moved to fit an island, which ends at the middle of this area: there is none north of it.
(145.625, 18.75, 'Agrigan Island', 'Northern Mariana Islands'),
])
def test_quad_name_at_an_outer_edge(west, south, name, state):
"""An area half of which is a quadrangle, and the other half no quadrangle, is a part of that quadrangle:
its middle is on the quadrangle's edge, not outside it."""
found = quads.quad_name(west, south, west + 0.125, south + 0.125)
assert found is not None
assert (found.name, found.state, found.exact) == (name, state, False)
def test_quad_name_at_a_shared_edge():
"""A place on the edge two quadrangles share is in the one east, or north, of that edge."""
assert quads.quad_name(-105.25, 40.25, -105, 40.375).name == 'Berthoud'
assert quads.quad_name(-105.125, 40.125, -105, 40.375).name == 'Berthoud'
def test_quad_name_at_a_shared_edge_the_map_gives():
"""The edges the map gives are a hair from where they are. An area whose middle is that near the edge two
quadrangles share is in the one east, or north, of that edge, whichever side of it the hair is."""
assert quads.quad_name(-149.875, 61.125, -149.75, 61.25).name == 'Anchorage A-8 NE'
# What the map gave for those edges.
given = (-149.87500000000014, 61.12499878064878, -149.75000000000023, 61.25000121452442)
assert quads.quad_name(*given).name == 'Anchorage A-8 NE'
assert quads.quad_name(-105.25 - 1e-9, 40.25 - 1e-9, -105, 40.375).name == 'Berthoud'
@pytest.mark.parametrize('west,south', [
(0, 0),
(151.125, -33.875), # Sydney.
(-105.125, 30), # Mexico.
(-40, 40), # The Atlantic.
(-180, 84.875),
])
def test_quad_without_a_name(west, south):
assert quads.quad_name(west, south, west + 0.125, south + 0.125) is None
def test_every_quad_is_found():
"""The middle of each quadrangle of the index is in a quadrangle of that name, or (where two lie over each
other) of its neighbor's."""
souths, rows = quads.load_index()
missed = 0
for south in souths[::7]:
wests, cells = quads._row(south)
for west, east, north, name, state in cells[::5]:
found = quads.quad_name(west / 3600, south / 3600, east / 3600, north / 3600)
assert found is not None, (name, state)
missed += found.name != name
assert missed <= 2
@pytest.mark.asyncio
async def test_api_quad_name(async_client):
request, response = await async_client.get('/api/map/quad?west=-105.125&south=40.25&east=-105&north=40.375')
assert response.status_code == HTTPStatus.OK
assert response.json == {'quad': {'name': 'Berthoud', 'state': 'Colorado', 'exact': True}}
# Somewhere the USGS has no name for is not an error.
request, response = await async_client.get('/api/map/quad?west=0&south=0&east=0.125&north=0.125')
assert response.status_code == HTTPStatus.OK
assert response.json == {'quad': None}
for query in ('', '?west=-105.125&south=40.25&east=-105', '?west=a&south=b&east=c&north=d',
'?west=-105.125&south=40.25&east=-113&north=40.375', '?west=nan&south=0&east=1&north=1',
'?west=-500&south=0&east=1&north=1'):
request, response = await async_client.get(f'/api/map/quad{query}')
assert response.status_code == HTTPStatus.BAD_REQUEST, query

View file

@ -0,0 +1,737 @@
import json
import math
import re
from http import HTTPStatus
import pymupdf
import pytest
from modules.map import export, magnetic, quads, sheet
from modules.map.export import ExportLayer, InvalidExport, Sheet, build_layered_pdf
from modules.map.test.test_export import api_export, layer_names, make_image, read_viewport
# A quadrangle of Anchorage, and the size of its image: 11 by 22.9 inches of paper at 36 DPI.
ANCHORAGE = [-150, 61.125, -149.875, 61.25]
WIDTH, HEIGHT, DPI = 397, 824, 36
MAP_WIDTH, MAP_HEIGHT = WIDTH * 72 / DPI, HEIGHT * 72 / DPI
SHEET = Sheet(name='61°07′30″N 150°00′W', credits=['© OpenStreetMap contributors', 'Protomaps'],
date='2026-11-15')
def build(bounds=None, sheet_=SHEET, **kwargs) -> bytes:
return build_layered_pdf([ExportLayer('Background', make_image(size=(WIDTH, HEIGHT)))], WIDTH, HEIGHT, DPI,
bounds=bounds or ANCHORAGE, sheet=sheet_, **kwargs)
def page_text(pdf: bytes) -> str:
return pymupdf.open(stream=pdf, filetype='pdf')[0].get_text()
@pytest.mark.parametrize('latitude,longitude,zone,south,easting,northing', [
# As PROJ has them (gdaltransform, EPSG:326xx and 327xx).
(61.2181, -149.9003, 6, False, 344247.206470396, 6790536.87132859),
(61.125, -150, 6, False, 338419.895396155, 6780416.03492079),
(40.31, -105.0595, 13, False, 494944.139497354, 4462166.80845969),
(-33.87, 151.21, 56, True, 334435.70614183, 6250816.39780466),
])
def test_utm(latitude, longitude, zone, south, easting, northing):
"""A place has the UTM coordinates PROJ gives it, to a millimeter, and they are of that place."""
assert sheet.utm_zone(longitude) == zone
assert sheet.to_utm(latitude, longitude, zone, south) == pytest.approx((easting, northing), abs=0.001)
assert sheet.from_utm(easting, northing, zone, south) == pytest.approx((latitude, longitude), abs=1e-8)
def test_utm_zone():
assert [sheet.utm_zone(i) for i in (-180, -174.001, -174, 0, 179.999, 180)] == [1, 1, 2, 31, 60, 60]
def test_format_angle():
assert sheet.format_angle(61.125, 'N', 'S') == '61°07\'30"N'
assert sheet.format_angle(-150, 'E', 'W') == "150°00'W"
assert sheet.format_angle(-33.875, 'N', 'S') == '33°52\'30"S'
assert sheet.format_angle(61.125, short=True) == '07\'30"'
assert sheet.format_angle(-149.9166666667, short=True) == "55'"
# Seconds which round up to a minute.
assert sheet.format_angle(2.9999999) == "3°00'"
def test_graticule_interval():
"""A 7.5 minute map has a tick every 2.5 minutes; a larger map has them further apart."""
assert sheet.graticule_interval(0.125) == pytest.approx(2.5 / 60)
assert sheet.graticule_interval(0.25) == pytest.approx(5 / 60)
assert sheet.graticule_interval(1) == pytest.approx(15 / 60)
assert sheet.graticule_interval(1e-9) == pytest.approx(1 / 3600)
def test_map_scale():
"""The quadrangle the browser draws for a 1:24,000 sheet is 1:24,000."""
assert sheet.map_scale(MAP_WIDTH, ANCHORAGE) == pytest.approx(24000, rel=0.005)
assert sheet.map_scale(MAP_WIDTH * 2, ANCHORAGE) == pytest.approx(12000, rel=0.005)
def test_plan():
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, SHEET)
x0, y0, x1, y1 = plan.map_rect
assert (x1 - x0, y1 - y0) == pytest.approx((MAP_WIDTH, MAP_HEIGHT))
# There is a margin on every side, and the most of it is under the map.
assert x0 > 36 and y0 > 36 and plan.width - x1 > 36
assert plan.height - y1 > y0
assert plan.zone == 6 and not plan.south_hemisphere
# The corners of the map are the corners of its rectangle.
assert plan.point(61.25, -150) == pytest.approx((x0, y0))
assert plan.point(61.125, -149.875) == pytest.approx((x1, y1))
notes = ' '.join(plan.notes)
assert 'Scale 1:24,000' in notes
assert 'zone 6N' in notes and 'west (46 mils) of true north' in notes
assert 'Map data: © OpenStreetMap contributors; Protomaps' in notes
assert 'Made with WROLPi, 2026-11-15.' in notes
def test_plan_notes_fit_the_map():
"""Notes too long for the map are on more lines, and the page is longer for them."""
short = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, Sheet())
long = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, Sheet(credits=['Somebody who made a map'] * 20))
assert len(long.notes) > len(short.notes)
assert long.height > short.height
assert all(pymupdf.get_text_length(i, fontsize=sheet.NOTE_SIZE) <= MAP_WIDTH for i in long.notes)
class FakePen:
def __init__(self):
self.lines, self.texts = [], []
def line(self, points, **kwargs):
self.lines.append(list(points))
def text(self, x, y, text, **kwargs):
self.texts.append((x, y, text, kwargs.get('align', 'left')))
def box(self, rect, **kwargs):
pass
def place_at(plan, x, y):
"""The latitude and longitude of a point of the page."""
x0, y0, x1, y1 = plan.map_rect
west, south, east, north = plan.bounds
top, bottom = (math.log(math.tan(math.pi / 4 + math.radians(i) / 2)) for i in (north, south))
mercator = top - (y - y0) / (y1 - y0) * (top - bottom)
return math.degrees(2 * math.atan(math.exp(mercator)) - math.pi / 2), west + (x - x0) / (x1 - x0) * (east - west)
@pytest.mark.parametrize('bounds,zone,south', [
(ANCHORAGE, 6, False),
([151.125, -33.875, 151.25, -33.75], 56, True),
([0, 0, 0.125, 0.125], 31, False),
])
def test_grid_lines_are_where_they_say(bounds, zone, south):
"""Every line of the grid is on a whole kilometer of UTM along all of it, and is nowhere off the map."""
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, bounds, Sheet())
assert (plan.zone, plan.south_hemisphere) == (zone, south)
pen = FakePen()
sheet._draw_grid(pen, plan)
x0, y0, x1, y1 = plan.map_rect
eastings, northings = set(), set()
for line in pen.lines:
places = [sheet.to_utm(*place_at(plan, x, y), zone, south) for x, y in line]
for x, y in line:
assert x0 - 1e-6 <= x <= x1 + 1e-6 and y0 - 1e-6 <= y <= y1 + 1e-6
spread_e = max(i[0] for i in places) - min(i[0] for i in places)
value = places[0][0] if spread_e < 1 else places[0][1]
assert value == pytest.approx(round(value / 1000) * 1000, abs=0.5)
for easting, northing in places:
assert (easting if spread_e < 1 else northing) == pytest.approx(value, abs=0.5)
(eastings if spread_e < 1 else northings).add(round(value))
# A quadrangle is several kilometers each way, and every kilometer has its line.
for values in (eastings, northings):
assert len(values) >= 5
assert sorted(values) == list(range(min(values), max(values) + 1, 1000))
# Each line is named in the margin: the first in full, the others by their kilometers.
labels = [text for _, _, text, _ in pen.texts]
for letter in ('mE', 'mN'):
assert len([i for i in labels if i.endswith(letter)]) == 2, labels
assert len([i for i in labels if i.isdigit()]) >= 12, labels
# And where the line meets the edge of the map.
for x, y, text, align in pen.texts:
if align == 'center':
edge = y0 if y < y0 else y1
meters = int(text[:-2]) if text.endswith('mE') else int(text) * 1000
assert sheet.to_utm(*place_at(plan, x, edge), zone, south)[0] == pytest.approx(meters, abs=0.5)
def test_grid_labels_keep_clear_of_coordinates():
"""A line of the grid which meets the edge where a coordinate is written is not named there."""
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, Sheet())
pen = FakePen()
sheet._draw_grid(pen, plan)
across, up = sheet._graticule_labels(plan)
for x, y, text, align in pen.texts:
if align == 'center':
assert all(not low < x < high for low, high in across), text
else:
assert all(not low < y - sheet.LABEL_SIZE * 0.35 < high for low, high in up), text
def test_no_tick_at_a_corner():
"""The edges of the picture the browser draws are a hair outside the quadrangle's. The corner is still the
corner: it is not also a tick, written over the corner's own coordinate."""
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, [-149.87500012, 61.12499878, -149.74999988, 61.25000121],
Sheet())
pen = FakePen()
sheet._draw_graticule(pen, plan)
labels = sorted(text for _, _, text, _ in pen.texts)
assert labels == sorted(2 * ["10'", '12\'30"', "50'", '47\'30"', "149°52'30\"W", "149°45'W", "61°15'N",
'61°07\'30"N'])
# Two ticks on each edge, and a cross (two lines) where each tick of latitude meets one of longitude.
assert len(pen.lines) == 4 * 2 + 4 * 2
def test_clip():
rect = (0, 0, 10, 10)
assert sheet._clip([(-5, 5), (15, 5)], rect) == [[(0, 5), (10, 5)]]
assert sheet._clip([(2, 2), (4, 4), (6, 2)], rect) == [[(2, 2), (4, 4), (6, 2)]]
# Out of the rectangle and in again is two lines.
assert sheet._clip([(2, 5), (5, 15), (8, 5)], rect) == [[(2, 5), (3.5, 10)], [(6.5, 10), (8, 5)]]
assert sheet._clip([(-5, -5), (-1, 20)], rect) == []
assert sheet._clip([(5, 5)], rect) == []
def test_sheet_pdf():
"""A map on a sheet is in the middle of a larger page, with its grid and its margin in layers of their own."""
plain = build_layered_pdf([ExportLayer('Background', make_image(size=(WIDTH, HEIGHT)))], WIDTH, HEIGHT, DPI,
bounds=ANCHORAGE)
pdf = build()
assert layer_names(plain) == ['Background']
assert layer_names(pdf) == ['Background', 'UTM grid', 'Sheet']
doc = pymupdf.open(stream=pdf, filetype='pdf')
page = doc[0]
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, SHEET)
assert (page.rect.width, page.rect.height) == pytest.approx((plan.width, plan.height), abs=0.01)
# The map is as large as it is without the sheet.
image = page.get_image_rects(page.get_images()[0][0])[0]
assert tuple(image) == pytest.approx(plan.map_rect, abs=0.01)
text = page.get_text()
# The name and the scale, the corners, the ticks between them, the grid, the bars and the notes.
for expected in ('61°07\'30"N 150°00\'W', '1:24,000', "150°00'W", '149°52\'30"W', "61°15'N", '61°07\'30"N',
"55'", '57\'30"', "10'", '12\'30"', '6790', '342', 'mE', 'mN', 'SCALE 1:24,000', 'KILOMETERS',
'METERS', '2000', 'MILES', 'FEET', '7000', 'zone 6N', '© OpenStreetMap contributors; Protomaps',
'Made with WROLPi, 2026-11-15.'):
assert expected in text, expected
def test_sheet_layers_can_be_hidden():
"""The grid and the margin are drawn only in their own layers."""
doc = pymupdf.open(stream=build(), filetype='pdf')
full = doc[0].get_pixmap(dpi=36)
for config in doc.layer_ui_configs():
if config['text'] in ('UTM grid', 'Sheet'):
doc.set_layer_ui_config(config['number'], action=2)
hidden = doc[0].get_pixmap(dpi=36)
assert full.samples != hidden.samples
# Only the map is left: the margin is as white as paper.
assert hidden.pixel(5, 5) == (255, 255, 255)
assert hidden.color_count() == 2
def test_sheet_georeference_is_of_the_map():
"""The PDF says where the map is, not where the page is."""
pdf = build()
viewport = read_viewport(pdf)
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, SHEET)
x0, y0, x1, y1 = plan.map_rect
assert viewport['bbox'] == pytest.approx([x0, plan.height - y1, x1, plan.height - y0], abs=0.01)
assert viewport['lpts'] == [0, 1, 0, 0, 1, 0, 1, 1]
assert viewport['gpts'] == [61.25, -150, 61.125, -150, 61.125, -149.875, 61.25, -149.875]
def test_sheet_near_a_pole_has_no_grid():
"""UTM stops at 84 degrees north and 80 south."""
pdf = build(bounds=[0, 84.5, 0.125, 84.625])
assert layer_names(pdf) == ['Background', 'Sheet']
assert 'Universal Transverse Mercator' not in page_text(pdf)
assert 'UTM grid' in layer_names(build(bounds=[0, -80, 0.125, -79.875]))
def test_sheet_of_a_narrow_map():
"""A map too narrow for a scale bar goes without it."""
narrow = build_layered_pdf([ExportLayer('Background', make_image(size=(90, 400)))], 90, 400, DPI,
bounds=[0, 84.5, 0.125, 84.625], sheet=Sheet())
assert 'MILES' not in page_text(narrow)
@pytest.mark.parametrize('bounds,name', [
# The narrowest quadrangles there are, with the longest names.
([-179.875, 84.875, -179.75, 85], '84°52′30″N 179°52′30″W'),
([-179.875, -84.875, -179.75, -84.75], '84°52′30″S 179°52′30″W'),
([10, 84.875, 10.125, 85], '84°52′30″N 10°00′E'),
(ANCHORAGE, '61°07′30″N 150°00′W'),
])
def test_sheet_title_of_a_narrow_map(bounds, name):
"""The name of a map and its scale are not written over each other, however narrow the map is."""
# As wide as the paper of a 1:24,000 sheet is.
map_width = sheet.map_scale(1, bounds) / 24000
width, height = round(map_width * DPI / 72), 400
pdf = build_layered_pdf([ExportLayer('Background', make_image(size=(width, height)))], width, height, DPI,
bounds=bounds, sheet=Sheet(name=name))
page = pymupdf.open(stream=pdf, filetype='pdf')[0]
titles = [pymupdf.Rect(span['bbox']) for block in page.get_text('dict')['blocks'] for line in block.get('lines', [])
for span in line['spans'] if span['size'] == pytest.approx(sheet.TITLE_SIZE)]
assert len(titles) == 2, titles
assert not titles[0].intersects(titles[1]), titles
# Both are on the page, above the map and what is written along its top edge.
top = sheet.plan_sheet(map_width, height * 72 / DPI, bounds, Sheet(name=name)).map_rect[1]
for title in titles:
assert title in page.rect
assert title.y1 <= top - sheet.LABEL_SIZE
def test_plan_norths():
"""A sheet says how far the grid's north and a compass's north are from true north, at the middle of it."""
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, SHEET)
# West of the middle of its zone the grid leans west; a compass in Alaska points well east.
assert plan.grid_north == pytest.approx(-2.571, abs=0.001)
assert plan.magnetic_north == pytest.approx(magnetic.declination(61.1875, -149.9375, 2026 + 318 / 365), abs=0.01)
assert 13 < plan.magnetic_north < 15
notes = ' '.join(plan.notes)
assert "Grid north is 2°34' west (46 mils) of true north" in notes
assert re.search(r"Magnetic north is 13°5\d' east \(24\d mils\) of true north at the middle of the map on 2026-11-15,"
r" and moves about 0°1\d' west each year \(WMM-2025\)\.", notes), notes
def test_plan_compass_which_passes_south():
"""Near the south magnetic pole a compass points almost due south, and within a year passes from a little
west of south to a little east of it. It has moved a little, not most of the way around."""
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, [140.5, -74, 140.625, -73.875], Sheet(date='2026-11-15'))
assert plan.magnetic_north == pytest.approx(-179.94, abs=0.01)
assert magnetic.load_model().declination(-73.9375, 140.5625, sheet._decimal_year('2027-11-15')) > 179
notes = ' '.join(plan.notes)
assert "Magnetic north is 179°56' west (3199 mils) of true north" in notes
assert "moves about 0°21' west each year" in notes
@pytest.mark.parametrize('date', [None, 'yesterday', '', '2024-12-31', '2030-01-01', '2026-13-45'])
def test_plan_without_a_compass(date):
"""Where a compass points is not said when the day is not known, or is not one the model is of."""
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, Sheet(date=date))
assert plan.magnetic_north is None
assert 'Magnetic' not in ' '.join(plan.notes)
assert plan.grid_north is not None
def test_decimal_year():
assert sheet._decimal_year('2026-01-01') == 2026
assert sheet._decimal_year('2026-07-02') == pytest.approx(2026.4986, abs=0.0001)
assert sheet._decimal_year('2028-12-31T10:00:00') == pytest.approx(2028 + 365 / 366)
def test_norths_are_on_their_own_sides():
"""In the diagram a north is drawn on the side of true north it is on, and apart from it however near it
is."""
assert sheet._north_tip(0, 0, 14, 50)[0] > 0 > sheet._north_tip(0, 0, -2.5, 50)[0]
# A north a tenth of a degree off, and one eighty off, are both drawn where they can be seen.
near, far = sheet._north_tip(0, 0, 0.1, 50), sheet._north_tip(0, 0, -80, 50)
assert near[0] == pytest.approx(50 * math.sin(math.radians(7)))
assert far == pytest.approx((-25, -50 * math.cos(math.radians(30))))
def test_sheet_norths_diagram():
"""The diagram of the norths is beside the scale bars, with each angle written; a sheet too narrow for it,
or with nothing to put in it, goes without."""
text = page_text(build())
assert 'MN 13°' in text and 'E\n' in text
assert 'GN 2°34\' W 46 MILS' in text
assert re.search(r"MN 13°5\d' E 24\d MILS", text), text
# No compass: the grid's north is still drawn.
text = page_text(build(sheet_=Sheet()))
assert 'GN 2°34\' W' in text and 'MN' not in text
# No grid near a pole, but a compass points somewhere; the quadrangle is too narrow for the diagram.
narrow = build_layered_pdf([ExportLayer('Background', make_image(size=(90, 400)))], 90, 400, DPI,
bounds=[0, 84.5, 0.125, 84.625], sheet=Sheet(date='2026-11-15'))
text = page_text(narrow)
assert 'Magnetic north is' in text
assert 'MN ' not in text and 'GN ' not in text
def test_sheet_text_which_the_font_can_not_write():
pdf = build(sheet_=Sheet(name='Карта “map” – 地図', credits=['<b>© Somebody</b>\n\nelse']))
text = page_text(pdf)
assert '"map" -' in text
assert '<b>© Somebody</b> else' in text
def test_sheet_needs_bounds():
with pytest.raises(InvalidExport, match='edges of the map are needed'):
build_layered_pdf([ExportLayer('Background', make_image())], 60, 90, 72, sheet=Sheet())
@pytest.mark.asyncio
async def test_api_export_pdf_sheet(async_client):
images = [('Background', make_image(size=(WIDTH, HEIGHT)))]
size = dict(width=WIDTH, height=HEIGHT, dpi=DPI)
response, job = await api_export(
async_client, images, bounds=ANCHORAGE, **size,
sheet=dict(name='A quadrangle', credits=['Somebody', 'x' * 500] + ['More'] * 50, date='2026-11-15'))
assert response.status_code == HTTPStatus.OK, response.text
assert layer_names(response.content) == ['Background', 'UTM grid', 'Sheet']
text = page_text(response.content)
assert 'A quadrangle' in text and 'Somebody' in text
# What is written on a sheet is limited.
assert len(re.findall('More', text)) == 38
assert 'x' * 301 not in text.replace('\n', '')
# An empty sheet is a sheet.
response, job = await api_export(async_client, images, bounds=ANCHORAGE, sheet={}, **size)
assert response.status_code == HTTPStatus.OK, response.text
assert 'Sheet' in layer_names(response.content)
# A sheet which is not one, and a sheet of a map which is nowhere.
for meta in (dict(bounds=ANCHORAGE, sheet='yes'), dict(sheet={})):
response, job = await api_export(async_client, images, **size, **meta)
assert response.status_code == HTTPStatus.BAD_REQUEST, json.dumps(meta)
def test_mils():
"""A compass marked in mils has 6,400 of them around."""
assert sheet.mils(360) == 6400
assert sheet.mils(90) == 1600
assert sheet.mils(-2.57) == 46
assert sheet.mils(0) == 0
def test_sheet_scale_bars():
"""The bars are of kilometers, meters, miles and feet, from the top down."""
doc = pymupdf.open(stream=build(), filetype='pdf')
tops = {word[4]: word[1] for word in doc[0].get_text('words')
if word[4] in ('KILOMETERS', 'METERS', 'MILES', 'FEET')}
assert sorted(tops, key=tops.get) == ['KILOMETERS', 'METERS', 'MILES', 'FEET']
gaps = [tops[b] - tops[a] for a, b in (('KILOMETERS', 'METERS'), ('METERS', 'MILES'), ('MILES', 'FEET'))]
assert gaps == pytest.approx([sheet.BAR_SPACING] * 3, abs=0.01)
@pytest.mark.parametrize('interval,unit,expected', [
(40, 'feet', 'CONTOUR INTERVAL 40 FEET'),
(20.0, 'meters', 'CONTOUR INTERVAL 20 METERS'),
(1000, 'feet', 'CONTOUR INTERVAL 1,000 FEET'),
(2.5, 'meters', 'CONTOUR INTERVAL 2.5 METERS'),
])
def test_sheet_contour_interval(interval, unit, expected):
"""A map with contour lines says how far apart they are, under its scale bars and over its notes."""
plain = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, SHEET)
with_contours = Sheet(name=SHEET.name, credits=SHEET.credits, date=SHEET.date,
contour_interval=interval, contour_unit=unit)
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, with_contours)
assert plan.contours == expected
assert plain.contours is None
# The page is longer by the room it takes.
assert plan.height == pytest.approx(plain.height + sheet.CONTOUR_LEADING)
doc = pymupdf.open(stream=build(sheet_=with_contours), filetype='pdf')
lines = {line.strip(): y for _, y, _, _, text, *_ in doc[0].get_text('blocks') for line in text.splitlines()}
assert expected in lines
feet = min(y for line, y in lines.items() if line == 'FEET')
notes = min(y for line, y in lines.items() if line.startswith('Scale 1:24,000'))
assert feet < lines[expected] < notes
assert 'CONTOUR INTERVAL' not in page_text(build())
@pytest.mark.parametrize('interval,unit', [
(None, 'feet'), (40, None), (40, 'furlongs'), (0, 'feet'), (-40, 'feet'), (math.nan, 'feet'), (math.inf, 'meters'),
('40', 'feet'),
])
def test_sheet_contour_interval_which_is_not_one(interval, unit):
"""What is not an interval of contour lines is not written."""
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, Sheet(contour_interval=interval, contour_unit=unit))
assert plan.contours is None
@pytest.mark.asyncio
async def test_api_export_pdf_sheet_contours(async_client):
"""The browser says how far apart the contour lines of a map are, and the sheet says it."""
images = [('Background', make_image(size=(WIDTH, HEIGHT)))]
size = dict(width=WIDTH, height=HEIGHT, dpi=DPI, bounds=ANCHORAGE)
response, job = await api_export(async_client, images, **size,
sheet=dict(contours=dict(interval=40, unit='feet')))
assert response.status_code == HTTPStatus.OK, response.text
assert 'CONTOUR INTERVAL 40 FEET' in page_text(response.content)
# What is not an interval is not written, and the map is exported.
for contours in (None, 'yes', {}, dict(interval='40', unit='feet'), dict(interval=True, unit='feet'),
dict(interval=40), dict(interval=40, unit='furlongs'), dict(interval=-1, unit='meters'),
dict(interval=40, unit=['feet'])):
response, job = await api_export(async_client, images, **size, sheet=dict(contours=contours))
assert response.status_code == HTTPStatus.OK, json.dumps(contours)
assert 'CONTOUR INTERVAL' not in page_text(response.content), json.dumps(contours)
BERTHOUD = [-105.125, 40.25, -105.0, 40.375]
BERTHOUD_NEIGHBORS = ['Masonville', 'Loveland', 'Windsor', 'Carter Lake Reservoir', 'Johnstown', 'Hygiene',
'Longmont', 'Gowanda']
def test_adjoining_quads():
"""The eight quadrangles around a map, from the northwest across and down."""
assert sheet.adjoining_quads(BERTHOUD) == BERTHOUD_NEIGHBORS
# The edges of what the browser drew are a hair outside the quadrangle.
assert sheet.adjoining_quads([-105.1250004, 40.2499998, -104.9999996, 40.3750002]) == BERTHOUD_NEIGHBORS
# Where the USGS has no quadrangles there is nothing to say.
assert sheet.adjoining_quads([2.25, 48.75, 2.375, 48.875]) is None
# At the edge of the land only some of the eight have a name: those out to sea have none.
coast = sheet.adjoining_quads([-124.25, 40.75, -124.125, 40.875])
assert coast is not None and None in coast and any(coast)
def test_sheet_adjoining_quads():
"""A sheet says what the quadrangles around it are called, in a diagram under its bars."""
size = (630, 820)
pdf = build_layered_pdf([ExportLayer('Background', make_image(size=size))], *size, DPI, bounds=BERTHOUD,
sheet=SHEET)
plan = sheet.plan_sheet(size[0] * 72 / DPI, size[1] * 72 / DPI, BERTHOUD, SHEET)
assert plan.adjoining == BERTHOUD_NEIGHBORS
doc = pymupdf.open(stream=pdf, filetype='pdf')
assert doc[0].rect.height == pytest.approx(plan.height, abs=0.01)
lines = {line.strip(): y for _, y, _, _, text, *_ in doc[0].get_text('blocks') for line in text.splitlines()}
assert 'ADJOINING QUADRANGLES' in lines
for number, name in enumerate(BERTHOUD_NEIGHBORS, start=1):
assert f'{number} {name}' in lines, name
# Under the bars, and over the notes.
notes = min(y for line, y in lines.items() if line.startswith('Scale 1:'))
assert lines['FEET'] < lines['ADJOINING QUADRANGLES'] < lines['8 Gowanda'] < notes
# The page is longer by the room the diagram takes.
nowhere = [2.25, 48.75, 2.375, 48.875]
without = sheet.plan_sheet(size[0] * 72 / DPI, size[1] * 72 / DPI, nowhere, Sheet())
with_ = sheet.plan_sheet(size[0] * 72 / DPI, size[1] * 72 / DPI, BERTHOUD, Sheet())
assert without.adjoining is None
assert with_.height - with_.map_rect[3] == pytest.approx(
without.height - without.map_rect[3] + sheet.ADJOINING_HEIGHT, abs=0.01)
def test_sheet_adjoining_quads_where_there_are_none():
"""A map of where the USGS names nothing, and a map too narrow for the diagram, go without it."""
assert 'ADJOINING' not in page_text(build(bounds=[2.25, 48.75, 2.375, 48.875]))
narrow = build_layered_pdf([ExportLayer('Background', make_image(size=(90, 400)))], 90, 400, DPI,
bounds=BERTHOUD, sheet=Sheet())
assert 'ADJOINING' not in page_text(narrow)
def test_sheet_adjoining_quads_some_unnamed():
"""A quadrangle with no name has no number in the diagram, and the others keep theirs."""
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, BERTHOUD, Sheet())
plan.adjoining = ['Northwest', None, None, None, 'East', None, None, 'A name ' + 'far too long ' * 20]
pen = FakePen()
sheet._draw_adjoining(pen, plan, 54, 100)
written = [text for _, _, text, _ in pen.texts]
assert '1 Northwest' in written and '5 East' in written
assert '1' in written and '5' in written and '8' in written
assert not {'2', '3', '4', '6', '7'} & set(written)
# A name too long for its column is cut to fit it.
long = next(text for text in written if text.startswith('8 A name'))
assert pymupdf.get_text_length(long, fontsize=sheet.LABEL_SIZE) <= sheet.ADJOINING_COLUMN
@pytest.mark.parametrize('bounds', [
[-149.875, 61.125, -149.75, 61.25],
[-147.75, 64.75, -147.625, 64.875],
[-156.875, 71.25, -156.75, 71.375],
ANCHORAGE,
])
def test_adjoining_quads_are_never_the_map_itself(bounds):
"""Where a quadrangle is wider than the map (Alaska), the map beside this one may be more of the same
quadrangle. The quadrangle on that side is the next one."""
own = quads.quad_name(*bounds)
around = sheet.adjoining_quads(bounds)
assert own.name not in around
def test_adjoining_quads_of_the_edges_the_map_gives():
"""The edges of an exported map are a hair from those of its quadrangle. What is around it is the same."""
around = sheet.adjoining_quads([-149.875, 61.125, -149.75, 61.25])
given = [-149.87500000000014, 61.12499878064878, -149.75000000000023, 61.25000121452442]
assert sheet.adjoining_quads(given) == around
assert around[3:5] == ['Anchorage A-8 NW', 'Anchorage A-7 NW']
def test_adjoining_quads_step_past_the_map():
"""The east of a map in the west of a wide quadrangle is the quadrangle after it, not the rest of it."""
bounds = [-149.875, 61.125, -149.75, 61.25]
assert quads.quad_name(*bounds).name == 'Anchorage A-8 NE'
# One map east is still Anchorage A-8 NE.
assert quads.quad_name(-149.75, 61.125, -149.625, 61.25).name == 'Anchorage A-8 NE'
around = sheet.adjoining_quads(bounds)
assert around[3:5] == ['Anchorage A-8 NW', 'Anchorage A-7 NW']
LEGEND = dict(ground=[0.9, 0.88, 0.85], lines=[
dict(label='Highway', color=[1, 1, 1], width=1.2, opacity=1, casing_color=[0.8, 0.8, 0.8], casing_width=2.4),
dict(label='Railway', color=[0.6, 0.7, 0.7], width=0.5, opacity=0.5, dash=[0.15, 0.4]),
dict(label='Contour', color=[0.6, 0.4, 0.2], width=0.25, opacity=0.3),
])
def test_legend_lines():
"""The lines of a legend are what a browser sent of them, as far as they are lines."""
ground, lines = sheet.legend_lines(LEGEND)
assert ground == (0.9, 0.88, 0.85)
assert [i.label for i in lines] == ['Highway', 'Railway', 'Contour']
assert lines[0] == sheet.LegendLine(label='Highway', color=(1, 1, 1), width=1.2, opacity=1,
casing_color=(0.8, 0.8, 0.8), casing_width=2.4)
assert lines[1].dash == (0.15, 0.4) and lines[1].opacity == 0.5 and lines[1].casing_color is None
# As it is sent again (to the Job which makes the PDF), it is the same legend.
assert sheet.legend_lines(sheet.legend_as_sent(LEGEND)) == (ground, lines)
assert json.loads(json.dumps(sheet.legend_as_sent(LEGEND)))['lines'][0]['label'] == 'Highway'
@pytest.mark.parametrize('legend', [
None, 'yes', [], {}, dict(lines='roads'), dict(lines=[]), dict(lines=['Highway', None, 7]),
dict(lines=[dict(label='', color=[1, 1, 1], width=1)]),
dict(lines=[dict(label='No color', width=1)]),
dict(lines=[dict(label='No width', color=[1, 1, 1])]),
dict(lines=[dict(label='Two parts', color=[1, 1], width=1)]),
dict(lines=[dict(label='Not numbers', color=['red', 0, 0], width=1)]),
dict(lines=[dict(label='Not a width', color=[1, 0, 0], width='wide')]),
dict(lines=[dict(label='Not a width', color=[1, 0, 0], width=math.nan)]),
dict(lines=[dict(label='Not a width', color=[1, 0, 0], width=True)]),
# Too thin to be seen on the map.
dict(lines=[dict(label='Path or track', color=[1, 0, 0], width=0.02, dash=[0.06, 0.02])]),
dict(lines=[dict(label='Nothing', color=[1, 0, 0], width=0)]),
])
def test_legend_lines_which_are_not(legend):
"""What is not a legend has no lines, and is on white."""
assert sheet.legend_lines(legend) == (sheet.WHITE, [])
assert sheet.legend_as_sent(legend) is None
def test_legend_lines_are_limited():
"""A legend has only so many lines, each named in only so many letters and drawn only so wide."""
line = dict(label='x' * 500, color=[2, -1, 0.5], width=900, opacity=7, dash=[0, 900] * 20,
casing_color=[0, 0, 0], casing_width=900)
ground, lines = sheet.legend_lines(dict(ground=[5, 5, 5], lines=[line] * 100))
assert ground == (1, 1, 1)
assert len(lines) == sheet.LEGEND_LINES
assert lines[0].label == 'x' * sheet.LEGEND_LABEL
assert lines[0].color == (1, 0, 0.5) and lines[0].opacity == 1
assert lines[0].width == sheet.LEGEND_WIDTHS[1] and lines[0].casing_width == 8
assert lines[0].dash == (0.1, sheet.LEGEND_SAMPLE) * 3
# A casing with no color, or no width, is no casing; dashes which are not numbers are no dashes.
ground, lines = sheet.legend_lines(dict(lines=[
dict(label='A', color=[0, 0, 0], width=1, casing_width=2),
dict(label='B', color=[0, 0, 0], width=1, casing_color=[1, 1, 1]),
dict(label='C', color=[0, 0, 0], width=1, dash=[1, 'long']),
]))
assert [(i.casing_color, i.casing_width, i.dash) for i in lines] == [(None, 0, ())] * 3
def test_sheet_legend():
"""A sheet with a legend has what each line is called beside a piece of it, under the bars and over the
notes, and is longer for it."""
with_legend = Sheet(name=SHEET.name, credits=SHEET.credits, date=SHEET.date, legend=LEGEND)
plain = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, SHEET)
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, with_legend)
assert [i.label for i in plan.legend] == ['Highway', 'Railway', 'Contour'] and not plain.legend
assert plan.legend_ground == (0.9, 0.88, 0.85)
# One row of lines: the map is wide enough for the three side by side.
assert plan.height == pytest.approx(
plain.height + sheet.LABEL_SIZE + 8 + sheet.LEGEND_ITEM_HEIGHT + 8)
doc = pymupdf.open(stream=build(sheet_=with_legend), filetype='pdf')
words = {word[4]: (word[0], word[1]) for word in doc[0].get_text('words')}
assert words['LEGEND'][1] < words['Highway'][1] == words['Railway'][1] == words['Contour'][1]
assert words['FEET'][1] < words['LEGEND'][1] and words['Highway'][1] < words['Scale'][1]
assert words['Railway'][0] - words['Highway'][0] == pytest.approx(sheet.LEGEND_ITEM_WIDTH, abs=0.01)
assert 'LEGEND' not in page_text(build())
# The lines are drawn in their own colors: the page has the brown of a contour line.
colors = {tuple(round(c, 2) for c in d['color']) for d in doc[0].get_drawings() if d.get('color')}
assert (0.6, 0.4, 0.2) in colors and (0.6, 0.7, 0.7) in colors
def test_sheet_legend_of_many_lines_is_in_rows():
lines = [dict(label=f'Line {i}', color=[0, 0, 0], width=1) for i in range(9)]
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, Sheet(legend=dict(lines=lines)))
columns = int(MAP_WIDTH // sheet.LEGEND_ITEM_WIDTH)
assert 1 < columns < 9
pen = FakePen()
sheet._draw_legend(pen, plan, 54, 100)
places = {text: (x, y) for x, y, text, _ in pen.texts}
# The line after the last of a row is under the first of it.
assert places[f'Line {columns}'][0] == places['Line 0'][0]
assert places[f'Line {columns}'][1] == pytest.approx(places['Line 0'][1] + sheet.LEGEND_ITEM_HEIGHT)
assert places[f'Line {columns - 1}'][1] == places['Line 0'][1]
@pytest.mark.asyncio
async def test_api_export_pdf_sheet_legend(async_client):
"""The browser says how the lines of a map are drawn, and the sheet has a legend of them."""
images = [('Background', make_image(size=(WIDTH, HEIGHT)))]
size = dict(width=WIDTH, height=HEIGHT, dpi=DPI, bounds=ANCHORAGE)
response, job = await api_export(async_client, images, **size, sheet=dict(legend=LEGEND))
assert response.status_code == HTTPStatus.OK, response.text
text = page_text(response.content)
assert 'LEGEND' in text and 'Highway' in text and 'Railway' in text
# What is not a legend is not drawn, and the map is exported.
for legend in (None, 'yes', [], {}, dict(lines='roads'), dict(lines=[dict(label='No color', width=1)])):
response, job = await api_export(async_client, images, **size, sheet=dict(legend=legend))
assert response.status_code == HTTPStatus.OK, json.dumps(legend)
assert 'LEGEND' not in page_text(response.content), json.dumps(legend)
def test_sheet_legend_name_too_long_for_its_column():
"""What a line is called is cut to the room beside the line, so it is not under the next line."""
lines = [dict(label='A name which is far too long for it ' + str(i), color=[0, 0, 0], width=1) for i in range(3)]
plan = sheet.plan_sheet(MAP_WIDTH, MAP_HEIGHT, ANCHORAGE, Sheet(legend=dict(lines=lines)))
pen = FakePen()
sheet._draw_legend(pen, plan, 54, 100)
written = [(x, text) for x, _, text, _ in pen.texts if text.startswith('A name')]
assert len(written) == 3
for x, text in written:
assert len(text) < len(lines[0]['label'])
left = x - sheet.LEGEND_SAMPLE - 5
assert x + pymupdf.get_text_length(text, fontsize=sheet.LABEL_SIZE) <= left + sheet.LEGEND_ITEM_WIDTH
def test_format_angle_of_no_angle():
"""An edge on the equator, or on the prime meridian, is on neither side of it; nor is one a hair from it."""
assert sheet.format_angle(0, 'N', 'S') == "0°00'"
assert sheet.format_angle(-1e-7, 'E', 'W') == "0°00'"
assert sheet.format_angle(1e-7, 'N', 'S') == "0°00'"
assert sheet.format_angle(-0.125, 'N', 'S') == "0°07'30\"S"
@pytest.mark.parametrize('bounds,zone', [
# The edges the map gives are a hair outside the quadrangle.
([10, 83.8749999, 10.125, 84.0000001], 32),
([10, -80.0000001, 10.125, -79.875], 32),
# Past where the zones end.
([10, 84, 10.125, 84.125], None),
([10, -80.125, 10.125, -80], None),
])
def test_sheet_grid_where_the_zones_end(bounds, zone):
"""A map which ends where the UTM zones do (84 north, 80 south) has a grid; one past there has none."""
plan = sheet.plan_sheet(300, 600, bounds, sheet.Sheet())
assert plan.zone == zone
@pytest.mark.parametrize('date', ['9999-12-31', '0000-01-01', 'yesterday', '', None])
def test_sheet_of_no_such_day(date):
"""A sheet whose day is not one is a sheet which does not say where magnetic north is."""
plan = sheet.plan_sheet(300, 600, BERTHOUD, sheet.Sheet(date=date))
assert plan.magnetic_north is None
def test_one_mil_is_a_mil():
"""An angle of one mil is "1 mil", in the notes and under the norths; more are "mils"."""
# Grid north at Berthoud is 0°02' west of true north: one mil.
plan = sheet.plan_sheet(300, 600, BERTHOUD, sheet.Sheet(date='2026-11-19'))
assert sheet.mils(plan.grid_north) == 1
assert '(1 mil) of true north' in ' '.join(plan.notes)
text = page_text(build_layered_pdf(
[ExportLayer('Background', make_image(size=(WIDTH, HEIGHT)))], WIDTH, HEIGHT, DPI, bounds=BERTHOUD,
sheet=sheet.Sheet(date='2026-11-19')))
assert '1 MIL' in text and '1 MILS' not in text
# Magnetic north there is many.
assert re.search(r'MN .* \d{2,} MILS', text)