Spatial Archive Cost Modeling: Retrieval, Penalties & Compression

Modeling the true cost of a geospatial archive means combining four numbers that vendor pricing pages present separately: per-gigabyte storage, retrieval and request fees, early-deletion penalties, and the compression ratio that shrinks all three. This reference is for the data engineers, cloud architects, and compliance teams who must defend a storage budget and prove that a tiering decision saves money rather than merely relocating it. Default per-GB comparisons are misleading — a tier that looks ten times cheaper can cost more once restore fees, minimum-duration billing, and read amplification against poorly sized objects are counted. The tables and formulas below give you a single, auditable model that every section of this knowledge base can reference.

Cost is not a property of a storage class; it is a property of an access pattern applied to a storage class. The same 40 TB LiDAR collection is cheap in Deep Archive if it is read once a year and ruinous if a monthly reprocessing job restores it. Everything here is built to be plugged into that access pattern.

The Total Cost Model #

Five cost components accumulate over an object’s life. Model them together, not in isolation:

The single most consequential lever is compression, because it reduces three of the five components at once. A 3:1 ratio on a vector archive does not just cut storage by two-thirds — it cuts every future restore and every egress byte by the same factor, compounding over the retention window. That is why compression tuning is a cost decision, not merely a storage decision, and why the Compression Tuning & Storage Optimization guides are upstream of every number here.

Storage Class Pricing Reference #

The table below lists representative US-region list prices for the storage classes a spatial archive typically spans. Prices drift; treat these as the model’s default coefficients and override them with your contracted rates. What rarely changes is the shape: each step down in price buys a step up in retrieval latency and minimum-duration commitment.

The per-gigabyte spread from Standard to Deep Archive is roughly 23:1, which is why aggressive cold-tiering is tempting. But the minimum-duration column is where naive models break: an object pushed to Deep Archive and deleted after 40 days still bills all 180 days. Choosing where an asset lands is the subject of Hot/Warm/Cold Tier Design for Geospatial Data, and the substrate that carries these classes is covered in Object Storage Selection for GIS Archives.

Early-Deletion Penalty Matrix #

Every tier below Standard bills a minimum number of days regardless of how long the object actually lived. The penalty for deleting — or transitioning — an object early is the remaining unmet days at that tier’s storage rate. Model it explicitly before setting any hot-to-warm or warm-to-cold transition trigger:

The practical rule this table encodes: never transition an asset into a tier whose minimum duration exceeds the time it will actually stay there. A dataset re-queried every 60 days should sit in Standard-IA, not Glacier — a too-aggressive rule that bounces it back to a hot tier on the next query pays the full 90-day Glacier minimum each round. Deriving transition days from real access telemetry rather than a static age cutoff is exactly the discipline in Setting Lifecycle Transition Thresholds from Query Telemetry.

Compression Ratio Reference #

Compression ratio converts raw dataset size into the compressed bytes every storage, retrieval, and egress fee is actually charged against. Representative ratios for spatial data, measured against uncompressed source:

Two attribute-level techniques amplify these ratios further: collapsing repetitive categorical columns with Dictionary Encoding for GIS Attributes, and matching the codec to the tier with the trade-offs in ZSTD vs LZ4 vs Snappy: Compression Trade-offs for Spatial Files. The exact level to run is governed by ZSTD Level Configuration for Spatial Files, because higher levels trade compress-time CPU for a smaller footprint — a cost you pay once against a saving you collect for the whole retention window.

A Worked Cost Model #

The following script models the total lifetime cost of an archive under a candidate tiering plan. It uses realistic coefficients and a real access pattern, so the output is directly comparable across plans.

# spatial_archive_cost.py — model lifetime cost of one dataset under a tiering plan
RATES = {
    "STANDARD":     {"gb_month": 0.023,  "min_days": 0,   "retrieval_gb": 0.0},
    "STANDARD_IA":  {"gb_month": 0.0125, "min_days": 30,  "retrieval_gb": 0.01},
    "GLACIER_IR":   {"gb_month": 0.004,  "min_days": 90,  "retrieval_gb": 0.03},
    "DEEP_ARCHIVE": {"gb_month": 0.00099,"min_days": 180, "retrieval_gb": 0.02},
}
EGRESS_GB = 0.09  # data transfer out to internet

def object_cost(raw_gb, ratio, plan, restores, egress_restores):
    """plan = list of (storage_class, days_resident); restores = GB read from cold."""
    comp_gb = raw_gb / ratio
    storage = requests = retrieval = penalty = 0.0
    for cls, days in plan:
        r = RATES[cls]
        storage += r["gb_month"] * comp_gb * (days / 30.0)
        if days < r["min_days"]:                      # early-deletion / transition penalty
            penalty += r["gb_month"] * comp_gb * ((r["min_days"] - days) / 30.0)
    retrieval = sum(RATES[c]["retrieval_gb"] * comp_gb for c in restores)
    egress = EGRESS_GB * comp_gb * egress_restores
    return round(storage + requests + retrieval + penalty + egress, 2)

# 2 TB vector estate, 6:1 GeoParquet, 1 yr Standard-IA then 4 yr Deep Archive,
# restored from Deep Archive twice with one full egress:
plan = [("STANDARD_IA", 365), ("DEEP_ARCHIVE", 1460)]
print(object_cost(raw_gb=2048, ratio=6.0, plan=plan,
                  restores=["DEEP_ARCHIVE", "DEEP_ARCHIVE"], egress_restores=1))

Run it and compare plans side by side:

python3 spatial_archive_cost.py

Expected output — the modeled five-year cost of the 2 TB estate under this plan:

228.53

Swap ratio=6.0 for ratio=3.0 and the same command returns roughly double, making the compression-versus-cost relationship concrete: the storage, retrieval, and egress lines all scale inversely with the ratio, while the request line does not.

Cost Trade-off Analysis #

Reading the model’s output back into decisions:

• Storage dominates for cold, rarely-read data. For a 5-year Deep Archive object read once, storage is >90% of cost; shaving it with a higher compression level is the only meaningful lever.
• Retrieval dominates for warm, frequently-read data. If a dataset is restored monthly, retrieval and egress can exceed a year of storage — keep it in Standard-IA or Glacier Instant, not Flexible or Deep Archive.
• Penalties dominate for churning data. Assets rewritten or re-tiered before their minimum duration bleed penalty cost invisibly. Incremental update patterns like Incremental GeoParquet Updates Without a Full Rewrite exist partly to avoid rewriting cold objects that would trigger these penalties.
• Object sizing changes request and read amplification. Millions of tiny partitions inflate PUT/GET counts and lifecycle-transition request fees; oversized objects amplify partial-read restore cost. Aligning file and row-group size to the retrieval tier is covered in Sizing Row Groups for Glacier Retrieval of GeoParquet.

Compliance & Retention Cost Interaction #

Retention requirements set a floor under the model. When a regulatory mandate forces a 10-year hold, the cost question narrows to “the cheapest class whose minimum duration and retrieval SLA I can tolerate for 10 years.” Object Lock in COMPLIANCE mode, detailed in Configuring S3 Object Lock for Compliance Spatial Archives, removes early deletion from the table entirely for the locked window — which means the early-deletion penalty column becomes irrelevant and Deep Archive’s low storage rate wins outright, provided the ~12-hour restore is acceptable for audit response. Model the retrieval fee against the realistic number of audit or legal-discovery restores per year, not the theoretical maximum. Consult the official AWS S3 storage pricing documentation and S3 Storage Classes documentation for authoritative current coefficients before committing a budget.

Operational Execution Checklist #

• Model all five components — storage, requests, retrieval, penalties, and egress — never per-GB storage alone.
• Apply the real compression ratio measured on a representative sample, not a vendor headline number.
• Check every transition against minimum duration — never move an asset into a tier it will leave before the minimum-duration clock expires.
• Derive transition days from access telemetry, not static age cutoffs, to avoid penalty churn.
• Bound object and row-group size to the retrieval tier to control request counts and partial-read amplification.
• Re-run the model per storage class and pick the plan with the lowest modeled total, not the lowest headline rate.
• Refresh rate coefficients against contracted pricing at least annually and re-validate the chosen plan.

Related #

• Compression Tuning & Storage Optimization — the ratios that scale three of the five cost components; upstream of every number here.
• Spatial Archival Architecture & Tiering Strategy — the tier model this cost reference prices out end to end.
• Format Conversion & Pipeline Automation — converting to columnar formats is what unlocks the compression ratios modeled above.
• Hot/Warm/Cold Tier Design for Geospatial Data — where the transition triggers this model prices are actually set.
• ZSTD Level Configuration for Spatial Files — tuning the compress-time CPU cost against lifetime storage savings.

Part of the Spatial Data Archival knowledge base.