Choosing Float32 vs Float64 for Archived Geometry
Halving the width of every coordinate is the most tempting optimisation in a spatial archive and, for geographic coordinates, almost always the wrong one. This walkthrough works through the arithmetic that decides it: what a single-precision float can actually represent at the magnitudes real coordinates take, where the switch is safe, and what to do instead when the storage saving is genuinely needed.
The Arithmetic That Decides It
A single-precision float carries about 24 bits of mantissa, or roughly seven significant decimal digits. What that means in ground distance depends entirely on the magnitude of the number being stored — and projected coordinates are large numbers.
The northing row is the one that ends most float32 discussions. A UTM northing in the northern hemisphere is a seven-digit number, and single precision cannot resolve it better than half a metre — in the same file where the easting resolves to three centimetres, producing an archive whose accuracy is anisotropic.
Step-by-Step Procedure
Step 1 — Compute the actual resolution at your extents
import numpy as np
def float32_resolution(value: float) -> float:
v = np.float32(value)
return float(np.nextafter(v, np.float32(np.inf)) - v)
for label, v in [("lon 1.5", 1.5), ("lon 179", 179.0),
("easting 400000", 400_000.0), ("northing 5430000", 5_430_000.0)]:
r = float32_resolution(v)
unit = "deg" if "lon" in label else "m"
ground = r * 111_320 if unit == "deg" else r
print(f"{label:20s} step={r:.3e} {unit} ≈ {ground:.3f} m on the ground")Run this against the actual corners of the archive’s extent, not against a nominal value. An archive spanning several UTM zones will have worse cases than any single representative coordinate suggests.
Step 2 — Compare against the required accuracy
The decision is a single comparison: is the worst-case float32 resolution across the archive’s extent comfortably below the data’s positional accuracy? “Comfortably” means at least a factor of five, because representation error accumulates through any arithmetic a reader performs.
Step 3 — Where the answer is no, take the saving elsewhere
Float32 is not the only way to halve the width. Scaled integers are exactly as compact and exactly representable at a chosen resolution, which is why point clouds have used them since long before this question arose.
Validation & Verification
Where float32 has been used — in an inherited archive, say — measure the damage rather than assuming it.
import geopandas as gpd, numpy as np
from shapely import get_coordinates
f64 = get_coordinates(gpd.read_parquet("source_f64.parquet").geometry.values)
f32 = f64.astype(np.float32).astype(np.float64) # the round trip a float32 column performs
err = np.abs(f64 - f32)
print(f"max error x={err[:,0].max():.4f} m y={err[:,1].max():.4f} m")
print(f"p99 error x={np.percentile(err[:,0],99):.4f} m y={np.percentile(err[:,1],99):.4f} m")Expected output for a projected archive shows an x error in centimetres and a y error in tens of centimetres — the anisotropy that makes single precision unsuitable. If both are far below the data’s accuracy, float32 was in fact safe for that extent.
Troubleshooting
| Symptom | Root cause | Fix |
|---|---|---|
| Vertices collapse onto each other | float32 resolution coarser than the vertex spacing | Restore from the double-precision source; this is unrecoverable in place |
| Polygons self-intersect after conversion | Rounding to float32 moved vertices past each other | Use scaled integers; repair topology with a precision model |
| Error much worse in y than x | Northings are larger numbers than eastings | Expected; it is the core argument against float32 |
| No storage saving despite float32 | Column stored as WKB blobs, not typed floats | The type only matters in a typed column; see the delta-encoding guide |
| Accuracy fine in test, bad in production | Test extent near the projection origin | Test at the archive’s worst-case corner, not its centre |
Operational Execution Checklist
Inherited Float32 Data: Assess Before Acting
Archives regularly acquire collections already stored in single precision, usually from a system where it was reasonable. The question is not whether to prefer float32 — it is what to do with data that already is, and the answer depends on whether the loss has already happened.
Never convert single-precision coordinates to double as a remediation. It changes nothing about the positions and removes the one signal — the storage type — that told a careful reader to be sceptical. Record the measured error distribution alongside the data instead.
Frequently Asked Questions
Is float32 ever the right choice for archived geometry?
In two narrow cases. Local-grid coordinates with small magnitudes — a site survey in a project-local system where values are in the hundreds of metres — resolve finely enough for most uses. And derived visualisation copies, where a metre of error is invisible and the file is regenerable, can use it freely. Neither case is the archive of record for a national or regional holding.
What about float16 or other narrow types?
Not for coordinates in any form. Half precision carries about three significant digits, which cannot represent a coordinate at any realistic magnitude. It appears in raster pixel data occasionally and has no place in a geometry column.
Does the choice affect how a query engine performs?
Marginally, and in float32’s favour: narrower columns mean fewer bytes to scan and better cache behaviour. That benefit is real and small, and it is comprehensively outweighed by storing positions the archive cannot vouch for. Where scan performance genuinely matters, the integer route delivers it without the accuracy cost.
Does the same reasoning apply to raster pixel values?
The reasoning transfers, the conclusion often does not. Pixel values are measurements with their own precision, and single precision is frequently ample for reflectance or temperature — the magnitudes are small and the instruments are not that precise. What does not transfer is the coordinate case, because a raster’s georeferencing lives in the geotransform, which should always be double precision regardless of the pixel type.
How is the decision recorded so it is not revisited annually?
As a line in the collection profile with its reasoning: the extent’s worst-case magnitude, the resulting float32 step, the data’s stated accuracy, and the conclusion. That turns an annual argument into a lookup, and it means a genuine change — a collection extended into a new UTM zone, say — prompts a genuine re-evaluation rather than a repeat of the same discussion.
What about coordinate columns in an analytical copy rather than the archive?
There the trade is different and float32 is often fine: a derived extract used for visualisation or aggregate analysis does not carry the archive’s accuracy obligations and can be regenerated. Keeping the distinction explicit — archive of record in double precision, derived copies as convenient — avoids the slow drift where a convenience copy becomes the thing everyone uses.
How does the choice interact with the storage format?
Less than it appears. In a WKB geometry column the coordinates are inside an opaque blob, so the storage type is fixed by the format at double precision and the question does not arise. It arises only where coordinates are stored as typed columns — the vertex-table arrangement — which is precisely the case where the integer alternative is available and better.
Is there a reliable way to detect float32 data in an inherited archive?
Yes: read a sample of coordinates and check whether they round-trip through single precision unchanged. Values that survive that round trip exactly were almost certainly stored as float32 at some point, even if the current column is double. It is a two-line check and worth running over any collection whose provenance is unclear.
Does the decision differ for elevation values?
Yes, and usually in float32’s favour. Elevations are small numbers — tens to thousands of metres — so single precision resolves them to well under a millimetre, and the accuracy argument that rules it out for northings does not apply. Store the horizontal coordinates as doubles and the vertical as float32 where the format allows independent types.
Related
- Coordinate Precision & Encoding Strategies — the parent topic where representation and precision are decided together.
- Delta Encoding Vertex Coordinates in GeoParquet — the route that delivers the storage saving without the accuracy loss.
- Measuring Precision Loss Against Spatial Query Accuracy — quantifying what any precision decision costs a real query.
- Validating Point Cloud CRS and Scaling After Conversion — scaled integers in the format that has always used them.
Up one level: Coordinate Precision & Encoding Strategies.