Preserving Field Names Across Shapefile to GeoPackage Conversion

Preserving field names across a shapefile-to-GeoPackage conversion means defeating the DBF ten-character column-name limit that silently truncates and collides descriptive attribute names before they ever reach the long-name-capable GeoPackage schema. A shapefile’s DBF header stores every field name in ten bytes; population_density and population_total both arrive as populatio, and the second write clobbers the first. GeoPackage and GeoParquet impose no such ceiling, so the conversion is an opportunity to restore intent — but only if you carry an explicit field-name mapping instead of trusting the driver to reverse a lossy truncation it cannot undo. This guide is for the data engineers and GIS archivists who need round-trip-safe attribute names when lifting legacy shapefile archives into GeoPackage under the Schema Mapping & Attribute Validation framework.

Where the Ten-Character Ceiling Bites #

The dBASE III format that backs the shapefile .dbf reserves exactly eleven bytes for a field name, one of which is a null terminator, leaving ten usable characters. GDAL enforces this on write by truncating, and — critically — by resolving collisions with numeric suffixes only within a single write session. Convert parcel_owner_name and parcel_owner_id into a shapefile and you get parcel_own and parcel_o_1; the suffix is positional, not semantic, so a second export in a different field order produces parcel_o_1 for a different column. Once names have degraded this way, no downstream tool can recover the originals, because the mapping from long name to truncated stub is many-to-one and unlabeled. GeoPackage stores column names as SQLite identifiers with no practical length limit, which is why it is the right target — but the truncation happens on the shapefile side, so the fix must live in the conversion step, not after it. This is the specific mechanism behind much of the loss catalogued in handling attribute loss during spatial format conversion.

The Alias Table Model #

The durable fix is a field-name mapping table — an explicit, version-controlled record of truncated_stub → canonical_name — captured either from the original data dictionary or from a metadata sidecar carried alongside the shapefile. The conversion applies the table to rename columns to their full form as it writes GeoPackage, and the same table drives round-trip verification:

The highlighted band marks the collision: both population fields share the populatio stub, so a naive shapefile-to-GeoPackage copy cannot know which restored name belongs to which column. The alias table breaks the ambiguity by keying restoration on something other than the truncated name — position, or better, a stable sidecar written when the shapefile was first created.

Building and Applying the Mapping #

1. Capture the canonical names before truncation happens #

The only reliable source of full names is upstream of the shapefile. If the archive still holds the originating GeoPackage, PostGIS table, or a documented data dictionary, extract the canonical list there. When only the shapefile survives, write a mapping sidecar the first time a human confirms the intended names, and version it with the data:

{
  "layer": "parcels_region_north",
  "field_map": {
    "populatio":  "population_density",
    "populat_1":  "population_total",
    "land_use_c": "land_use_code",
    "parcel_own": "parcel_owner_name",
    "parcel_o_1": "parcel_owner_id"
  }
}

The keys are the exact truncated stubs GDAL produced, including its _1 collision suffixes; the values are the canonical names. This file is the contract, checked into the pipeline repository next to the dataset manifest.

2. Apply the map during conversion #

Rename as you write GeoPackage so the long names land in the SQLite schema directly. ogr2ogr with an SQL SELECT ... AS clause renames per column without a second pass:

ogr2ogr -f GPKG \
  s3://spatial-archive/parcels/2023/parcels_region_north.gpkg \
  /vsis3/spatial-archive/parcels/2023/parcels_region_north.shp \
  -sql "SELECT populatio AS population_density,
               populat_1 AS population_total,
               land_use_c AS land_use_code,
               parcel_own AS parcel_owner_name,
               parcel_o_1 AS parcel_owner_id,
               GEOMETRY
        FROM parcels_region_north" \
  -nln parcels_region_north

For programmatic pipelines that already hold the sidecar, drive the rename from the JSON so the mapping stays single-sourced:

import json
import geopandas as gpd

field_map = json.load(open("parcels_region_north.field_map.json"))["field_map"]

gdf = gpd.read_file("/vsis3/spatial-archive/parcels/2023/parcels_region_north.shp")
gdf = gdf.rename(columns=field_map)

gdf.to_file(
    "/vsis3/spatial-archive/parcels/2023/parcels_region_north.gpkg",
    layer="parcels_region_north",
    driver="GPKG",
)

Because GeoPackage preserves these long names cleanly, the same renamed frame flows on into columnar storage without further mapping — the naming discipline established here is what makes a later GeoParquet migration workflow round-trip-safe.

Verifying the Round Trip #

Confirm every canonical name is present and no stub leaked through. List the GeoPackage schema straight from SQLite and check it against the mapping’s values:

ogrinfo -so \
  s3://spatial-archive/parcels/2023/parcels_region_north.gpkg \
  parcels_region_north | grep -E ':' | grep -iv geometry

Expected output — full names, no ten-character stubs, no _1 collision suffixes:

population_density: Real (0.0)
population_total: Real (0.0)
land_use_code: String (0.0)
parcel_owner_name: String (0.0)
parcel_owner_id: Integer64 (0.0)

Then run a strict assertion in code that fails if any restored name is missing or any truncated stub survived, which is the round-trip guarantee the pipeline gate depends on:

import json
import fiona

expected = set(json.load(open("parcels_region_north.field_map.json"))["field_map"].values())
with fiona.open("/vsis3/spatial-archive/parcels/2023/parcels_region_north.gpkg") as src:
    actual = set(src.schema["properties"].keys())

missing = expected - actual
leaked = {n for n in actual if len(n) == 10 or n[-2:] in ("_1", "_2")}
assert not missing, f"canonical names missing: {missing}"
assert not leaked, f"truncated stubs leaked into GeoPackage: {leaked}"

This paired schema-and-assertion check is the natural sibling of the field-count-and-type diff in validating attribute schema parity after format conversion; run both so name preservation and type parity are certified in one gate.

Troubleshooting Truncation and Collisions #

Keep the sidecar and the CRS definition together as the two things that must survive every hop; the projection half of that pairing is enforced by CRS Synchronization in Pipelines. Descriptive, un-truncated names also make categorical columns self-documenting for the compression stage, where dictionary encoding for GIS attributes keys on stable field identities. For the field name rules themselves, the authoritative reference is the OGC GeoPackage Encoding Standard.

Operational Execution Checklist #

• Recover canonical field names from the originating GeoPackage, PostGIS table, or data dictionary before touching the shapefile.
• Write a version-controlled field_map sidecar keyed on GDAL’s exact truncated stubs, including _1 collision suffixes.
• Apply the rename during the GeoPackage write with ogr2ogr -sql ... AS or a rename(columns=...) call driven by the sidecar.
• List the GeoPackage schema with ogrinfo -so and confirm no ten-character stub or _1 suffix survived.
• Assert every canonical name is present and no truncated stub leaked, failing the pipeline on either.
• Preserve Integer64 and other long types explicitly during the write, since renaming does not restore type.
• Set SHAPE_ENCODING=UTF-8 so non-ASCII field names round-trip intact.

Related #

• Up: Schema Mapping & Attribute Validation — the parent reference for field-level mapping across conversions.
• Validating Attribute Schema Parity After Format Conversion — the sibling gate that detects the truncation and collisions this procedure prevents.
• Handling Attribute Loss During Spatial Format Conversion — companion coverage of the broader attribute-loss modes beyond names.
• When to Use Dictionary Encoding for Categorical GIS Fields — compression guidance for the well-named categorical fields you preserve.
• Spatial Archive Cost Modeling — the cost reference for weighing re-conversion effort against long-term archive maintainability.

Part of the Spatial Data Archival knowledge base.