Spatial Data Archival · Cold Storage

Validating Point Cloud CRS and Scaling After Conversion

Point-cloud conversions fail quietly. A file with the wrong offset, a dropped vertical datum or a downgraded record format opens without complaint, reports a plausible bounding box, and renders correctly on its own — the error only surfaces when it is compared with something else, often years later. This walkthrough is the gate that catches those defects at the moment of conversion, when fixing them costs a re-run rather than a restore.

The Four Header Facts That Must Survive

Everything this gate checks lives in the LAS header and its variable-length records. All four are cheap to read and all four are silently mutable by a converter.

Four header facts and the silent failure each one hides Scale and offset, compound reference system, point data record format, and classification scheme, each with the silent failure produced when a converter changes it. header factwhat it controlssilent failure if changed scale + offset how int32 values decode to coordinates every point shifts, consistently compound CRS horizontal and vertical placement elevations ambiguous by tens of metres point record format which attributes each point carries time, colour or returns disappear classification scheme what class codes mean ground becomes vegetation, silently

Step-by-Step Procedure

Step 1 — Compare headers directly, source against output 

compare_headers() {
  for f in "$1" "$2"; do
    pdal info --metadata "$f" | jq -c '.metadata | {
      count, dataformat_id, minor_version,
      scale: [.scale_x, .scale_y, .scale_z],
      offset: [.offset_x, .offset_y, .offset_z],
      srs: (.srs.compoundwkt // .srs.wkt // "NONE")[0:80]}'
  done
}
compare_headers datasets/lidar/2023/raw/region_north_merged.las \
                /vsis3/spatial-archive/archive/lidar/2023/region_north.copc.laz

Every field except minor_version should be identical. A differing scale or offset is a blocking failure even when the decoded bounding box looks right, because the bounding box is computed from the same wrong decode.

Step 2 — Verify coordinates decode identically, not merely plausibly

The stronger check compares actual point positions on a deterministic sample. Bounding boxes agree in cases where individual points do not.

import pdal, json, numpy as np

def sample_points(path, n=5000):
    pipe = pdal.Pipeline(json.dumps({"pipeline": [
        {"type": "readers.las", "filename": path},
        {"type": "filters.decimation", "step": 997}]}))   # prime step: deterministic, spread out
    pipe.execute()
    arr = pipe.arrays[0][:n]
    return np.stack([arr["X"], arr["Y"], arr["Z"]], axis=1)

src = sample_points("datasets/lidar/2023/raw/region_north_merged.las")
out = sample_points("/vsis3/spatial-archive/archive/lidar/2023/region_north.copc.laz")
delta = np.abs(src - out).max()
print(f"max coordinate difference: {delta:.6f} m")
assert delta < 1e-6, "coordinates changed during conversion"

Expected output is a maximum difference below one micrometre — that is, exact within the storage scale. Anything at millimetre level or above means the frame changed.

Step 3 — Compare the classification histogram 

for f in source.las output.copc.laz; do
  pdal info --all "$f" \
    | jq -c '[.stats.statistic[] | select(.name=="Classification")
              | .counts[] | {v: (.value|tonumber), n: .count}]'
done
Classification histogram before and after conversion A matching histogram between source and output, contrasted with a failed conversion where all class codes shifted by one while every aggregate statistic stayed the same. Classification counts · source vs output class 2 · ground 412 M — matches class 3 · low veg 88 M — matches class 5 · high veg 210 M — matches class 6 · building 61 M — matches The failure this catches and nothing else does A converter that remaps class codes by one produces 412 M points of “low vegetation” where the ground was. Point count, extent, scale, offset and file size are all unchanged. Only the histogram shows it.

Validation & Verification

Wrap the three checks into one gate that blocks promotion, and run it on fixtures so it is known to fire.

python -m archive.pc_gate validate \
  --source datasets/lidar/2023/raw/region_north_merged.las \
  --output /vsis3/spatial-archive/archive/lidar/2023/region_north.copc.laz \
  --require-vertical-datum --max-coord-delta 1e-6
# frame:          PASS  (scale and offset identical)
# crs:            PASS  (EPSG:27700+5701, vertical datum present)
# coordinates:    PASS  (max delta 0.000000 m over 5,000 sampled points)
# record format:  PASS  (6 -> 6)
# classification: PASS  (4 classes, counts identical)

Troubleshooting

Symptom Root cause Fix
Coordinates differ by exactly the scale Rounding on write with a coarser scale Pin the source scale; never let the writer choose
Vertical datum absent from the output Writer given a horizontal-only SRS string Use a compound code such as EPSG:27700+5701
Record format 6 written as 3 Writer default, or a filter that downgrades Pin dataformat_id on the writer
Classification counts shift by one A remapping filter left in the pipeline Remove it; re-run — the archive must hold source semantics
Point count matches, sample delta large Points reordered and the sample is positional Sample by decimation step, as shown, not by index range

Operational Execution Checklist

Fixtures the Gate Should Be Tested Against

A validation gate is only as good as the failures it has been shown to catch. Point-cloud conversions fail in a small number of characteristic ways, and keeping one deliberately broken file per failure mode is what proves the gate still fires after a library upgrade.

One fixture per failure mode Five deliberately broken fixtures and the gate check each one exercises. fixture what is wrong with it check it exercises derived_offset.laz offset recomputed per tile frame comparison no_vertical.laz horizontal-only SRS compound CRS requirement format3_downgrade.laz record format 6 written as 3 record format pin shifted_classes.laz class codes moved by one classification histogram coarse_scale.laz scale changed 0.001 → 0.01 coordinate sample comparison Run all five on every change to the gate; a fixture that stops being blocked is the only warning you will get that a check has silently stopped working.

Generate the fixtures from one small, known-good source so they differ from it in exactly one respect. A fixture that is broken in several ways proves less: it will be blocked by whichever check runs first, which tells you nothing about the others.

Frequently Asked Questions

Why is a matching bounding box not enough?

Because the bounding box is derived from the same header values the check is testing. A file whose offset changed reports a bounding box computed with the new offset, which will look entirely reasonable — it is internally consistent and externally wrong. Only comparing decoded coordinates against the source, or comparing the header values themselves, detects it.

What if the source has no declared reference system?

Then the conversion cannot supply one honestly, and the correct outcome is quarantine rather than an assumption. Point-cloud deliveries frequently carry the reference system in an accompanying document; recover it from there, record where it came from, and only then convert. Guessing produces an archive whose elevations mean nothing.

How large should the coordinate sample be?

Five thousand points spread deterministically across the file is ample — the defects this catches are systematic, so they appear in the first handful of points and the rest is confirmation. What matters more than size is that the sample is spread rather than clustered, since a positional slice can miss a defect confined to one part of the octree.

Should the gate compare point counts as well as headers?

Yes, and it is the cheapest check in the set — both sides report it in their headers. A count mismatch is unambiguous and points at a specific failure: an input skipped during a merge, a filter left in the pipeline, or a duplicate removal nobody intended. Include it even though it is not strictly about the frame.

How is the vertical datum verified when the source declares it informally?

By recovering it from the delivery documentation and recording where it came from, then asserting the recovered value on the output. Point-cloud deliveries frequently state the vertical datum in a report rather than in the file, and treating that as sufficient is reasonable provided the provenance is recorded — treating it as absent and writing a horizontal-only system is not.

Can the gate run against objects already in the archive?

Yes, for the header checks, which read only a few kilobytes. The coordinate comparison needs the source, which for an archived object usually means the retained delivery — one more reason to keep it. Where no source remains, the header checks alone still detect a frame that disagrees with the collection’s declared frame.

Should the gate check the octree as well as the header?

For COPC output, yes — a header can be perfect while the hierarchy is truncated, which produces a file that opens, reports the right point count, and fails on any spatial query. The check is cheap: read the root hierarchy page and confirm its node count and byte ranges fall inside the file. Adding it closes the one structural failure mode a header comparison cannot see.

How is the gate’s result recorded for later use?

Alongside the object, in the same manifest entry that holds the frame and the checksum. That record is the baseline for every later integrity check on the object and the only evidence that the frame was correct at the moment of writing, which is a fact no later check can establish on its own.

How is the gate applied to deliveries that arrive already as COPC?

Identically, with the delivered file as both source and candidate output — the checks that matter are whether its frame matches the archive’s, whether its reference system is compound, and whether its record format carries what the archive expects. A delivered COPC is not exempt from the frame decision simply because it is already indexed, and a delivery in a different frame is the case the gate exists to catch.

What should be recorded when a delivery cannot be validated at all?

The reason, the quarantine location and what would be needed to resolve it. A delivery with no recoverable reference system is not a failure of the gate; it is a finding about the delivery, and recording it as such is what allows someone with access to the producing organisation to close it later.

Up one level: Point Cloud & COPC Conversion.

Related pages

Compressing LiDAR with LASzip for Deep Archive Converting LAS to COPC for Cold Archives Streaming COPC Octree Nodes Over HTTP Range Requests