Multi-Region Replication & Disaster Recovery for Spatial Archives

A spatial archive that exists in one region is one control-plane incident, one mistaken lifecycle rule, or one compromised credential away from being unavailable — and for geospatial holdings the recovery is rarely as simple as copying bytes back. Point-cloud tiles run to hundreds of thousands of objects, imagery collections carry catalogues that must be rebuilt in step with the data, and retention-locked objects cannot be re-created at all if their lock configuration is lost. This topic is for the cloud architects and archive operators who have a working hot/warm/cold tier design and now need to state, defensibly, what happens when the region holding it stops answering.

The Failure Mode: A Copy That Is Not a Recovery #

The common failing is not the absence of a second copy but the absence of a rehearsed path back from it. Cross-region replication is easy to enable and produces a reassuring object count in another region; what it does not produce is a working archive. The catalogue still points at the primary region’s URLs. The lifecycle rules that governed transitions were never applied to the destination, so every replicated object sits in whatever class the replication configuration specified — often Standard, quietly costing several times the primary. Retention locks may or may not have been replicated depending on how the rule was written. And the restore automation, the credentials it uses, and the network paths it assumes all name the region that is now unavailable.

The result is a second copy that satisfies an auditor’s checkbox and fails the first real test. A recovery is a sequence of steps that ends with a user able to answer the question they were asking before the incident, and every step in that sequence has to work in the region that survived.

Prerequisite Context #

Three things should be in place before replication is designed. The archive needs a stable object layout, because replication configuration is expressed in prefixes and tags and re-keying afterwards means re-replicating. It needs versioning enabled, since replication depends on it and since versioning is the control that protects against the deletion failures replication does not. And it needs the catalogue to be generated from the objects rather than maintained alongside them, because a catalogue built by a job can be rebuilt in the surviving region while one maintained by hand cannot.

Where the archive already applies retention locks, the destination bucket must be created with Object Lock enabled before any replication begins. That flag cannot be added later, so a destination created without it forces a second migration at exactly the moment nobody wants one.

Concept & Design Decisions #

The design reduces to four choices, and each has a defensible answer that depends on what the archive is protecting against rather than on a general notion of resilience.

Scope. Replicate everything, or only what cannot be regenerated? Derived products — the delivery copies, the tile pyramids, the warm analytical extracts — can be rebuilt from the archive of record, and paying to replicate them doubles the storage cost of material that a rebuild job could restore in hours. Replicating only the authoritative copies typically covers 30–60% of a spatial archive’s volume and all of its irreplaceable content.

Destination class. The replica’s storage class is set by the replication rule and does not inherit the source’s lifecycle. Setting it to match the source’s eventual class — Glacier for material that will end there — avoids paying Standard prices for a copy that will never be read until an incident. The cost is restore latency during a recovery, which is a recovery-objective decision rather than a storage one.

Isolation. A replica in a second region under the same account protects against regional failure and nothing else. A replica in a separate account, with a role that permits writes from replication and nothing else, additionally protects against credential compromise and against a mistaken policy change in the primary account. The second arrangement costs no more in storage and is materially harder to destroy.

Direction. One-way replication with a documented promotion procedure is simpler to reason about than bidirectional synchronisation, and for an archive — where writes are append-only and infrequent — it gives up almost nothing. Bidirectional configurations introduce conflict semantics that an archive has no natural way to resolve.

Implementation #

The replication rule itself is a small piece of infrastructure-as-code, and its important parts are the ones that are easy to omit: the storage class on the destination, the explicit inclusion of delete markers (or, deliberately, their exclusion), and replica modification sync so that later changes to the source object’s metadata reach the copy.

# Terraform: selective cross-region, cross-account replication for the archive of record
resource "aws_s3_bucket_replication_configuration" "archive_of_record" {
  bucket = aws_s3_bucket.spatial_archive.id
  role   = aws_iam_role.replication.arn

  rule {
    id       = "archive-of-record-to-eu-west-1"
    status   = "Enabled"
    priority = 10

    filter {
      and {
        prefix = "archive/"
        tags = {
          copy_class = "authoritative" # derived products are tagged differently and never match
        }
      }
    }

    # Delete markers are NOT replicated: a delete in the primary must not
    # propagate to the isolated copy. This is the whole point of the isolation.
    delete_marker_replication { status = "Disabled" }

    destination {
      bucket        = "arn:aws:s3:::spatial-archive-dr-euw1"
      account       = "210987654321" # separate account
      storage_class = "GLACIER"      # do not pay STANDARD for a copy nobody reads

      access_control_translation { owner = "Destination" }

      metrics {
        status = "Enabled"
        event_threshold { minutes = 15 } # alert if replication lags beyond RPO
      }
      replication_time {
        status = "Enabled"
        time { minutes = 15 } # S3 Replication Time Control — a contractual RPO
      }
    }

    source_selection_criteria {
      sse_kms_encrypted_objects { status = "Enabled" }
      replica_modifications     { status = "Enabled" }
    }
  }
}

Two clauses carry most of the design. Disabling delete-marker replication is what makes the replica a protection against accidental deletion rather than a faithful mirror of one; it means a delete in the primary leaves the replica intact and a genuine deletion must be performed in both places deliberately. Enabling replication time control converts replication lag from a hope into a measurable objective with an alarm attached.

Validation Gate #

Replication is verified by counting, not by observing that it is enabled. The check that matters compares object inventories on both sides, because a rule that silently matches nothing looks identical to one that has nothing to replicate.

# Compare source and replica inventories for the authoritative prefix.
# Both inventory reports are generated daily by the storage platform — no LIST calls.
aws s3api list-objects-v2 --bucket spatial-archive --prefix archive/ \
  --query 'length(Contents)' --output text          # 412037

aws s3api list-objects-v2 --bucket spatial-archive-dr-euw1 --prefix archive/ \
  --query 'length(Contents)' --output text --profile dr-account   # 412037

# Spot-check replication status on a recently written object
aws s3api head-object --bucket spatial-archive \
  --key archive/imagery/2024/scene_0417.tif \
  --query '{status:ReplicationStatus,class:StorageClass}'

Expected output is {"status": "COMPLETED", "class": "GLACIER"} on the source side and matching counts on both. A status of PENDING on objects older than the replication-time objective is the signal that the rule is failing rather than lagging — most often because the replication role lost permission on a new prefix, which produces no error anywhere except in this field.

Cost & Performance Trade-offs #

Replication has three cost components and one of them is frequently forgotten. Storage in the destination is the obvious one and is minimised by class selection and selective scope. Replication requests and the cross-region transfer of each object are charged once per object and scale with object count, so an archive of many small objects pays disproportionately — the same argument that governs partition sizing applies here. The forgotten component is replication time control, which carries a per-gigabyte premium in exchange for a contractual objective; it is worth it for the material whose recovery objective is measured in minutes and wasteful for the rest.

The last row is the recommendation for most institutional archives: the same price as the row above it, materially stronger isolation, and a recovery time that is acceptable precisely because the archive it protects is already cold.

Failure Modes & Edge Cases #

Replication configured before Object Lock. A destination bucket created without Object Lock cannot hold retention-locked replicas, and the failure is silent — objects replicate, locks do not. The only fix is a new destination bucket and a full re-replication.

Existing objects are not replicated. Replication applies to objects written after the rule is enabled. Backfilling requires a batch operation over the existing inventory, which is an explicit job with its own cost and duration; assuming the rule covers history is the single most common gap found in a first audit.

KMS keys are regional. An object encrypted with a customer-managed key in region A cannot be decrypted in region B without a key there and a replication configuration that re-encrypts. A replica that cannot be decrypted during an incident is not a recovery.

Catalogue drift between regions. The replica’s objects are identical; their URLs are not. A catalogue rebuilt in region B must rewrite every asset href, and a catalogue that stores absolute URLs rather than deriving them from a base is much harder to move.

Operational Execution Checklist #

• Create the destination bucket with versioning and Object Lock enabled before any data lands
• Tag authoritative objects at ingest so replication scope is a tag filter, not a guess
• Set the destination storage class explicitly; never inherit Standard by omission
• Disable delete-marker replication so deletions do not propagate to the isolated copy
• Place the replica in a separate account with a write-only replication role
• Provision a KMS key in the destination region and enable replica encryption
• Backfill pre-existing objects with a batch operation and reconcile inventories afterwards
• Deploy the restore automation and its credentials into the destination region
• Rebuild the catalogue against the replica and verify a sample of asset hrefs resolve
• Rehearse a cutover annually and record the measured recovery time

Deciding When to Fail Over #

The hardest part of a disaster-recovery design is not the replication and not the runbook — it is the decision to invoke it. Failing over too early to what turns out to be a fifteen-minute regional blip means promoting a replica, accepting the write loss, and later reconciling two divergent copies for no benefit. Failing over too late means an outage measured in hours where the archive was recoverable in one.

The decision should be written down before it is needed, as a small set of conditions with an owner attached to each. Two signals are worth waiting on: whether the provider has acknowledged the incident and given an estimated duration, and whether the archive’s own health checks against the primary region have failed consistently rather than intermittently. Two are worth acting on immediately: any indication that data has been lost rather than merely made unavailable, and any confirmation that the outage exceeds the recovery-time objective the archive has published.

Name the person who can make the call and the deputy who can make it at three in the morning. A technically flawless replication design that waits four hours for someone with authority to wake up has an RTO of four hours, whatever the runbook says.

Frequently Asked Questions

Related #

• Configuring Cross-Region Replication for Spatial Archives — the step-by-step provisioning walkthrough for the rule sketched above.
• Replicating Glacier-Class Spatial Objects Across Regions — what changes when the source objects are already in an archive tier.
• Rebuilding a Spatial Archive After Regional Bucket Loss — the recovery runbook this design exists to make possible.
• Measuring RPO and RTO for Geospatial Archives — turning recovery objectives from assertions into measurements.
• Archive Integrity Verification for Spatial Data — the fixity discipline that proves the replica is as good as the primary.
• Spatial Archive Cost Modeling — where the standing cost of a replica belongs in the archive’s budget.

Up one level: Spatial Archival Architecture & Tiering Strategy.