Spatial Data Archival · Cold Storage

Measuring Precision Loss Against Spatial Query Accuracy

A rounding threshold is only defensible if someone has measured what it does to the answers the archive gives. Coordinate error and query error are not the same quantity: a one-centimetre vertex shift can change a point-in-polygon result for a feature sitting on a boundary, leave an area calculation unchanged to five significant figures, and move a jurisdictional assignment for exactly the parcels that matter. This walkthrough is for the engineer proving a precision decision before it becomes permanent.

Which Queries Are Sensitive, and Which Are Not

The sensitivity depends on whether the query’s answer is continuous or categorical. Continuous measures absorb vertex noise; categorical assignments flip.

Query sensitivity to a one-centimetre rounding grid Five query types with the share of results that change under a one-centimetre grid: area and perimeter barely move, point-in-polygon and nearest-neighbour flip rarely, and adjacency between parcels is the most sensitive. Share of results that change · 1 cm grid · 2.4 M parcels polygon area 0.0002% — below reporting precision perimeter 0.004% — below reporting precision nearest neighbour rank 0.01% — ties between near-equal distances point in polygon 0.03% — points within 1 cm of a boundary parcel adjacency 0.6% of shared edges — the query the archive must actually be validated against

Step-by-Step Procedure

Step 1 — Build a query panel that reflects real use

Take the queries the archive actually serves — from access logs, from the analyses its users publish, from the reports it feeds — rather than a synthetic set. A panel of five to ten query types is enough if it includes at least one categorical assignment.

PANEL = [
    ("area",        lambda gdf: gdf.geometry.area),
    ("perimeter",   lambda gdf: gdf.geometry.length),
    ("centroid",    lambda gdf: gdf.geometry.centroid),
    ("pip",         lambda gdf, pts: gpd.sjoin(pts, gdf, predicate="within")["index_right"]),
    ("adjacency",   lambda gdf: adjacency_pairs(gdf)),
]

Step 2 — Run the panel against both versions 

import geopandas as gpd, numpy as np
from shapely import set_precision

src = gpd.read_parquet("staging/parcels_2026.parquet")
pts = gpd.read_parquet("staging/service_points.parquet")

results = {}
for grid in (0.001, 0.01, 0.05, 0.1, 1.0):
    rounded = src.copy()
    rounded["geometry"] = set_precision(src.geometry.values, grid_size=grid)

    area_rel   = np.abs(rounded.geometry.area - src.geometry.area) / src.geometry.area
    pip_src    = gpd.sjoin(pts, src,     predicate="within")["index_right"]
    pip_round  = gpd.sjoin(pts, rounded, predicate="within")["index_right"]
    pip_flips  = (pip_src.reindex(pts.index) != pip_round.reindex(pts.index)).sum()

    results[grid] = {
        "area_p99":  float(np.percentile(area_rel, 99)),
        "pip_flips": int(pip_flips),
        "pip_rate":  pip_flips / len(pts),
        "collapsed": int(rounded.geometry.is_empty.sum()),
    }

for g, r in results.items():
    print(f"grid={g:>6} m  area p99={r['area_p99']:.2e}  "
          f"pip flips={r['pip_flips']:>5} ({r['pip_rate']:.4%})  collapsed={r['collapsed']}")

Step 3 — Read the curve and pick the knee

Result changes grow slowly with grid size and then accelerate once the grid approaches the scale of real features. The threshold to pick is below that acceleration, not at the point where the first result changes — some result changes are inevitable and immaterial.

Result change rate against rounding grid size A curve of point-in-polygon flip rate rising slowly to five centimetres and accelerating beyond ten, with the chosen threshold marked at five centimetres where it matches the survey accuracy. 4%2%0.5%0 chosen: 5 cm 1 mm1 cm5 cm 10 cm50 cm1 m rounding grid size At 1 m the curve is not the only problem: 118 features collapse entirely, which no rate captures.

Validation & Verification

The measurement is itself validated by checking that the panel would detect a threshold that is obviously too coarse.

python -m archive.precision_panel --grid 5.0 --expect-failure
# grid=5.0 m  area p99=1.4e-02  pip flips=98,412 (4.10%)  collapsed=3,847
# PANEL VERDICT: reject — flip rate and collapse count exceed tolerance

Expected output is a rejection. A panel that accepts a five-metre grid on cadastral data is not measuring what it claims to, usually because the point set used for the point-in-polygon test contains no points near boundaries — which is where all the sensitivity lives.

Troubleshooting

Symptom Root cause Fix
Flip rate is zero at every grid Test points not near boundaries Sample points within a few centimetres of edges deliberately
Area error larger than expected Features collapsing, not vertices shifting Report collapse count separately from relative error
Adjacency changes everywhere Polygons snapped independently Snap the layer in one operation so shared nodes stay shared
Results differ between runs Non-deterministic ordering in the join Sort both sides before comparing; use stable indices
Panel passes but production disagrees Panel queries not representative Rebuild the panel from access logs rather than from intuition

Operational Execution Checklist

Presenting the Result to a Non-Technical Owner

The precision decision is usually signed off by someone who owns the data rather than the pipeline, and a flip-rate table will not get a decision. Framing the same measurement in terms of the organisation’s own outputs will.

The same measurement, framed for the data owner Three technical findings restated in terms of the organisation’s outputs so a data owner can approve the precision decision. technical finding what it means for outputs owner decision 0.03% point-in-polygon flips 12 of 41,000 points change parcel all within 5 cm of a boundary 0.0002% area change no published area changes below reporting precision 18 features collapsed 18 slivers under 5 cm wide removed all digitising artefacts 37% smaller geometry column £1,400 a year saved the reason for doing it The last row is why the conversation is happening; the first three are why it is safe.

Include the list of the twelve affected points as an appendix. An owner who can look at the specific records that change is in a position to approve; one shown only a percentage is being asked to trust an abstraction.

Frequently Asked Questions

What flip rate is acceptable?

There is no universal figure, and the useful framing is comparative rather than absolute: the flip rate should be well below the rate at which the same queries already disagree because of the data’s own positional uncertainty. A cadastral layer accurate to eight centimetres already gives ambiguous point-in-polygon answers for points within eight centimetres of a boundary; a rounding grid that adds materially less ambiguity than that is not the limiting factor.

Should the panel run on every archive generation?

On the first, and thereafter whenever the precision decision or the data’s accuracy changes. It is a decision-support measurement rather than a continuous gate — the continuous check is the much cheaper assertion that no vertex moved further than half the grid.

Does this apply to raster data?

The analogous question for rasters is resampling and bit depth rather than coordinate precision, and the method transfers: run the analyses the archive actually serves against the original and the reduced version, and look at where categorical outputs — classifications, thresholded masks — start to disagree. Continuous outputs absorb the change; categorical ones flip, exactly as here.

How large should the point set for the point-in-polygon test be?

Large enough to include several hundred points near boundaries, which for a naturally distributed set usually means tens of thousands overall. The efficient alternative is to generate the test points deliberately: sample along boundaries at a range of offsets, from a millimetre to a metre, which measures the sensitivity directly rather than waiting for a random sample to find it.

Should the panel include queries that combine spatial and attribute filters?

Yes, because they are the common shape in practice and because the combination can behave unexpectedly — a filter that selects few features makes each individual flip a larger share of the result. A query returning three parcels where one flips is a 33% error in that answer, however small the archive-wide rate.

What if two grids give indistinguishable results?

Choose the coarser one, and record that the finer was measured and rejected. That is the outcome the measurement exists to produce: the coarser grid saves storage permanently, and the evidence that it changes nothing measurable is what makes the choice defensible later.

How does this measurement interact with the archive’s stated accuracy?

They should agree, and where they do not the measurement is usually right. An archive advertising eight-centimetre accuracy whose query panel shows meaningful result changes at a two-centimetre grid has a data problem rather than a rounding problem — most often a collection merged in at lower accuracy than the headline figure suggests. The panel is a good detector for that, because it reads the data rather than the metadata.

Should the panel be re-run when the query patterns change?

When they change materially, yes, and the trigger is worth defining in advance: a new dominant query type, a new consumer with different tolerances, or a change in what the archive is used to decide. Re-running is cheap once the harness exists, and the result is usually that the previous threshold still holds — which is itself worth recording, because it turns the original decision from a one-off judgement into a standing, tested position.

What if the measurement says the current precision is already excessive?

Then you have found a saving that costs nothing, and the constraint on acting is whether the affected objects can still be rewritten. For material still in a mutable class the rewrite is straightforward; for locked or deeply tiered objects it is not worth the restore. Apply the finding to new writes, record it against the collection, and let the existing objects age out under whatever retention governs them.

Should the panel be version-controlled with the pipeline?

Yes — it is a test, and it decays like one. Keeping the queries, the fixtures and the expected results alongside the conversion code means a change to either is reviewed against the other, and it makes the measurement reproducible by whoever inherits the archive.

Up one level: Coordinate Precision & Encoding Strategies.

Related pages

Choosing Float32 vs Float64 for Archived Geometry Delta Encoding Vertex Coordinates in GeoParquet Reducing Coordinate Precision Before Archival