Coordinate Precision & Encoding Strategies

Geometry columns resist compression because their low-order digits are noise: a longitude stored to fifteen decimal places describes a position finer than any instrument measured, and those digits never repeat, so no codec can model them. Precision and numeric encoding are the two levers that change that — and both are irreversible write-time decisions with the same permanence as a retention lock. This topic is for the data engineer or GIS archivist deciding how much precision an archive should keep, and in what representation, before the geometry column is written.

The Failure Mode: Archiving Measurement Noise #

A survey delivers coordinates at, say, five-centimetre accuracy. Those coordinates are then transformed, reprojected and reformatted through a pipeline whose intermediate representations are all double precision, and the archive stores fifteen significant digits of a five-centimetre measurement. The result is an archive whose geometry column is roughly a third larger than it needs to be, and whose extra bytes carry no information — they are an artefact of the arithmetic, not of the ground.

Worse, the stored precision is frequently mistaken for accuracy by later readers. A coordinate presented to eleven decimal places invites the conclusion that it is known to that resolution, and nothing in the file says otherwise unless the archive records the true accuracy explicitly.

Prerequisite Context #

Two facts must be established before any rounding: the source data’s actual positional accuracy, taken from the survey metadata rather than assumed, and the archive’s intended use, since a cadastral archive and a regional land-cover archive tolerate very different thresholds. The reference system must also be settled first — rounding in degrees and rounding in metres are different operations, and reprojecting after rounding reintroduces the digits you removed.

Concept & Design Decisions #

How much to keep. Round to the smallest resolution the source measurement supports, not to a convenient number. Six decimal places in geographic coordinates is about eleven centimetres at the equator and suits most survey-grade holdings; seven is about eleven millimetres and is beyond what most aerial or GNSS-derived data supports.

Which representation. Doubles are the safe default for geographic coordinates. Single-precision floats hold about seven significant digits, which is enough for a projected coordinate in a local grid and not enough for a longitude — a float32 longitude near the antimeridian resolves to about a metre. Scaled integers, as point clouds use, are exact at a chosen resolution and are the most compact of the three.

Where delta encoding applies. Consecutive vertices of a polygon are close together, so storing differences rather than absolute positions shrinks the numeric range dramatically and gives the codec far more redundancy to model. Parquet’s own encodings do some of this; explicit vertex delta encoding in the geometry representation does much more.

Implementation #

Rounding belongs upstream of the writer, applied once, with the resulting precision recorded in the dataset manifest.

import geopandas as gpd
from shapely import set_precision

gdf = gpd.read_parquet("staging/parcels_2026.parquet")
assert gdf.crs.to_epsg() == 27700, "round in the archive's storage CRS, not before reprojection"

# 1 cm grid in a projected CRS measured in metres; set_precision snaps and repairs topology
gdf["geometry"] = set_precision(gdf.geometry.values, grid_size=0.01)

gdf.to_parquet(
    "s3://spatial-archive/archive/parcels/2026/parcels.parquet",
    compression="zstd", compression_level=12,
    row_group_size=500_000, write_statistics=True,
    geometry_encoding="WKB",
)

set_precision snaps vertices to the grid and repairs the topological damage snapping can cause — collapsed slivers, self-intersections at the new resolution. A naive round-and-write leaves those defects in the archive.

Validation Gate #

Prove two things: that the intended precision was applied, and that no feature moved further than the tolerance allows.

import geopandas as gpd, numpy as np
from shapely import get_coordinates

before = gpd.read_parquet("staging/parcels_2026.parquet")
after  = gpd.read_parquet("s3://spatial-archive/archive/parcels/2026/parcels.parquet")

shift = np.abs(get_coordinates(before.geometry.values)
               - get_coordinates(after.geometry.values)).max()
invalid = (~after.geometry.is_valid).sum()
lost    = len(before) - len(after)
print(f"max vertex shift {shift:.4f} m · invalid geometries {invalid} · features lost {lost}")

Expected output is a maximum shift at or below half the grid size, zero invalid geometries and zero features lost. A non-zero lost count means snapping collapsed some features entirely, which happens to slivers narrower than the grid and needs a decision rather than a silent acceptance.

Cost & Performance Trade-offs #

Each step down the table removes roughly a fifth of the geometry column, and the sensible stopping point is the survey’s accuracy — the row below it discards real measurement, and the rows above it store arithmetic.

Failure Modes & Edge Cases #

Rounding before reprojection. Reprojecting rounded coordinates reintroduces full precision, because the transformation is arithmetic. Round last, in the archive’s storage reference system.

Topology destroyed by snapping. Shared boundaries between adjacent polygons can separate or overlap after independent snapping. Use a precision model that snaps consistently, and validate that shared edges remain shared.

Slivers collapsing to nothing. Features narrower than the grid disappear. That is often correct — a 3 mm sliver is a digitising artefact — but it must be counted and reported, never silently absorbed.

Precision recorded nowhere. An archive that rounds without recording the grid size leaves future readers unable to distinguish deliberate resolution from measurement uncertainty.

Operational Execution Checklist #

• Establish the source’s real positional accuracy from its metadata
• Choose a grid size at or just below that accuracy, never below it by orders of magnitude
• Reproject to the archive’s storage reference system before rounding
• Use a precision model that repairs topology, not a naive round
• Count and report features lost to sliver collapse
• Validate that no vertex moved more than half the grid size
• Record the grid size, the reference system and the tool version in the manifest
• Re-measure the geometry column’s compressed size to confirm the saving

Recording Precision So Readers Can Trust It #

Stored precision and measured accuracy are different quantities, and an archive that publishes only the first invites every reader to assume they are the same. Four fields, recorded per dataset, remove the ambiguity permanently and cost a few hundred bytes.

Surface the first two in the catalogue as well as the manifest, because they answer the question a user asks before deciding whether the dataset suits their analysis. A dataset advertising centimetre coordinates and eight-centimetre accuracy is being honest; one advertising only the former is not.

Precision Across a Conversion Chain #

Precision decisions interact with every other write-time choice, and the order in which they are applied determines whether they compose or cancel. Four steps recur in most archival pipelines, and only one ordering leaves the archive with the precision it intended.

Reprojection comes first, because it is arithmetic that regenerates full precision regardless of what preceded it. Rounding comes second, in the archive’s storage reference system and its units. Encoding — scaling to integers, if the archive uses that representation — comes third, at the same grid the rounding used, so the integers are exact rather than approximations of rounded floats. Compression comes last, working on data that is now genuinely low-entropy rather than merely rounded.

Getting the order wrong is not subtly wrong; it silently discards the benefit. Rounding before reprojection produces coordinates whose full precision is restored by the transformation, so the archive stores fifteen digits of a value that was briefly six. Encoding before rounding scales noise into the integers, which then delta-encode poorly because their low bits are still random. Compressing before rounding wastes the codec’s effort on digits about to be discarded and then requires a rewrite to remove them.

A second interaction worth naming is with the spatial ordering that partitioning imposes. Sorting features spatially before writing is what makes delta encoding effective, and it also improves the codec’s performance on the geometry column regardless of encoding — so the sort belongs after rounding and before the write, alongside the partition key computation. In a well-ordered pipeline these are four steps in one job rather than four jobs, which is also what keeps them from drifting apart over time.

Record the order as well as the parameters in the manifest. A future reader reproducing the archive from its source needs to know not only that coordinates were rounded to five centimetres in EPSG:27700, but that the rounding happened after reprojection — because performing the same steps in a different order produces different bytes and a different checksum, and a reproducibility claim that cannot survive that is not much of a claim.

Frequently Asked Questions

When Not to Reduce Precision #

Precision reduction is close to free in information terms when it is applied at or above the data’s measured accuracy, and genuinely destructive when it is not. Four situations argue for leaving coordinates alone, and recognising them is as much a part of the practice as the rounding itself.

The first is unknown source accuracy. An archive that cannot state what the coordinates were measured to cannot state that rounding discards nothing, and the honest response is to archive at full precision and record that the accuracy is undetermined. Rounding on an assumption converts an unknown into a false certainty, which is the one outcome preservation should never produce.

The second is data whose value lies in its exactness rather than its position — control points, calibration targets, survey monuments and the reference networks other datasets are adjusted against. These are small, so the storage saving is negligible, and they are the datasets against which everything else is checked, so any movement propagates. Exempt them explicitly rather than relying on their size to make the question moot.

The third is derived geometry whose vertices are computed rather than measured: intersections, buffers, generalisation outputs, and the boundaries of administrative units defined by a legal description rather than by survey. Rounding these can break the exact coincidences that make them consistent with one another — two boundaries that were computed to meet exactly may cease to after independent snapping — and the fix is either to round them together as one operation or not at all.

The fourth is any dataset already under an active legal hold or an unresolved dispute, where the archived form may be examined against a delivered original. Precision reduction is a modification, and even a well-documented one is an awkward fact to explain in that context. Archive as delivered, and apply the archive’s normal treatment once the matter closes.

Recording the exemption is as important as recording the rounding. A collection stored at full precision beside others that were rounded looks like an oversight unless the manifest says otherwise, and the next engineer to review storage costs will propose exactly the change the exemption exists to prevent.

How does precision policy interact with data supplied to third parties? #

Supply from the archive rather than re-deriving, and state the precision alongside the data. A recipient given coordinates to five decimal places without a stated accuracy will reasonably assume the two match, and the resulting misuse is the archive’s to prevent rather than theirs to detect. Where a recipient requires full source precision, that is an argument for retaining the source rather than for archiving unrounded.

Does rounding affect the ability to reproduce a published analysis? #

It can, which is why the precision decision is dated and recorded. An analysis run against the archive before a rounding change and repeated afterwards may differ in the boundary cases the measurement panel quantifies. Publishing archive generations, rather than mutating a single copy, is what allows a reproduction to name the version it used — and it is the same versioning discipline that boundary changes and schema changes need.

Is there a standard the archive should follow? #

No single standard governs stored precision, and the ones that touch it — metadata standards recording positional accuracy, format specifications defining coordinate types — assume rather than prescribe. That absence is why the manifest fields matter: in the absence of a convention every reader will assume a different one, and the only defence is stating what was actually done.

Does the precision decision belong to the archive or to the data producer? #

To the archive, informed by the producer’s stated accuracy. Producers deliver whatever their pipeline emits, which is usually full double precision regardless of what was measured, and expecting them to round to the archive’s grid adds a step outside the archive’s control. Taking the delivery as it comes and applying the archive’s own policy at write time keeps the decision in one place and makes it reviewable.

How is precision handled for data that arrives already rounded? #

Accept it, record the grid you observed, and do not round further. A delivery already at ten-centimetre precision has had its decision made upstream, and applying the archive’s own grid on top either does nothing or degrades it. What the archive should record is the observed precision and that no further rounding was applied, so the manifest describes the file rather than the policy.

Does precision reduction help formats other than the geometry column? #

It helps anywhere coordinates are written as text or as full-width floats, which includes GeoJSON exports, CSV extracts and the bounding-box columns derived from the geometry. Applying it once upstream of the write means every derived artefact inherits it, whereas rounding each output separately invites the copies to disagree with the archive about where a feature is.

Related #

• Reducing Coordinate Precision Before Archival — the step-by-step rounding procedure with topology repair.
• Delta Encoding Vertex Coordinates in GeoParquet — how to get the third column of the table above.
• Choosing Float32 vs Float64 for Archived Geometry — why the middle row is usually a trap.
• Measuring Precision Loss Against Spatial Query Accuracy — proving a rounding threshold is safe before committing to it.
• ZSTD Level Configuration for Spatial Files — the codec that works on whatever the coordinate representation gives it.
• CRS Synchronization in Pipelines — why rounding must come after the reference system is settled.

Up one level: Compression Tuning & Storage Optimization.