Spatial Data Archival · Cold Storage

Tuning COG Internal Tile Size for Range Requests

Internal tile size is the parameter that decides how much a reader over-fetches. Too small and a window request costs dozens of round trips; too large and each round trip drags back pixels nobody asked for. This walkthrough is for the engineer choosing that number for an imagery archive with a known access pattern, and it replaces the usual “512 is fine” with a measurement against the windows readers actually request.

Over-Fetch Is the Quantity to Minimise

A reader asks for a rectangle; the file can only supply whole tiles. The bytes fetched are therefore the area of the tiles that intersect the request, not the area of the request — and the ratio between them is the over-fetch factor that tile size controls.

Tiles touched and bytes over-fetched for one window A 900 by 700 pixel window overlaid on three tile grids of 256, 512 and 1024 pixels, showing the number of tiles touched, the megapixels fetched, and the over-fetch ratio for each. 256 px tiles 20 tiles · 1.31 MP · 2.1× 20 ranged requests 512 px tiles 6 tiles · 1.57 MP · 2.5× 6 ranged requests 1024 px tiles 2 tiles · 2.10 MP · 3.3× 2 ranged requests

The trade is between bytes and round trips, and against object storage the round trips usually matter more than the bytes: twenty requests at forty milliseconds each is 800 ms of latency for a saving of 0.8 megapixels, which over a typical link takes under 100 ms to transfer.

Step-by-Step Procedure

Step 1 — Characterise the windows readers actually request

Access logs carry the byte ranges served, and those ranges are the ground truth about window sizes. Do not design against an imagined access pattern.

-- Distribution of served range sizes over 90 days, by user agent class
SELECT CASE WHEN user_agent LIKE '%QGIS%'  THEN 'desktop GIS'
            WHEN user_agent LIKE '%titiler%' THEN 'tile server'
            ELSE 'other' END                              AS client,
       approx_percentile(bytes_sent, 0.5)                 AS p50,
       approx_percentile(bytes_sent, 0.95)                AS p95,
       COUNT(*)                                           AS requests
FROM spatial_access_logs
WHERE operation = 'REST.GET.OBJECT' AND key LIKE 'archive/imagery/%'
  AND dt >= date_add('day', -90, current_date)
GROUP BY 1 ORDER BY requests DESC;

Step 2 — Compute over-fetch for candidate tile sizes 

import math

def overfetch(win_w, win_h, tile, bytes_per_px=3):
    tx = math.ceil((win_w + tile - 1) / tile) + (1 if win_w % tile else 0)
    ty = math.ceil((win_h + tile - 1) / tile) + (1 if win_h % tile else 0)
    fetched = tx * ty * tile * tile
    return {"tiles": tx * ty,
            "fetched_mb": round(fetched * bytes_per_px / 1e6, 2),
            "ratio": round(fetched / (win_w * win_h), 2)}

for tile in (256, 512, 1024):
    print(tile, overfetch(900, 700, tile))

Step 3 — Measure the real thing, not the model

The model ignores latency, HTTP overhead and reader behaviour. Time the actual reads through the same client stack the archive serves.

for T in 256 512 1024; do
  /usr/bin/time -f "tile=$T  %e s" \
    gdal_translate -q -srcwin 4000 3200 900 700 \
      "/vsis3/spatial-archive/bench/ortho_tile${T}.tif" /dev/null
done
Measured read latency by tile size for three access patterns Three client patterns measured against three tile sizes: a tile server favours 256-pixel tiles, a desktop GIS favours 512, and a batch analytic favours 1024, with 512 the best compromise across all three. client pattern256 px512 px1024 pxbest tile server · 256 px windows many small requests 88 ms124 ms238 ms 256 desktop GIS · 900×700 pans moderate windows, interactive 341 ms196 ms274 ms 512 batch analytic · 4,000 px blocks few large sequential reads 2.9 s1.8 s1.4 s 1024 No size wins everywhere. 512 is never best and never bad, which is why it is the right default for a mixed-audience archive. Where one client dominates by an order of magnitude, tune for that client instead.

Validation & Verification

Confirm the written file carries the tile size you intended — the parameter is easy to set and easy to have overridden by a driver default.

gdalinfo -json /vsis3/spatial-archive/archive/imagery/2026/ortho_n5432_e0871.tif \
  | jq '.bands[0].block'
# [512, 512]

A block reported as [24000, 1] means the file is stripped rather than tiled and no tile-size tuning applies at all — it is a plain GeoTIFF that will be read in full for any request.

Troubleshooting

Symptom Root cause Fix
Block size reports [width, 1] TILED=YES omitted, or a non-COG driver used Convert with -of COG, which tiles by definition
Small tiles but slow reads Round-trip latency dominating Increase tile size, or enable request merging in the client
Large tiles and high egress Over-fetch on small windows Reduce tile size toward the p50 window dimension
Tile size differs across the collection Per-scene conversion without a pinned parameter Pin it in the job definition, not the operator’s command
Reads faster locally than through the archive Benchmark run against a cached local copy Benchmark through the same client and network path as production

Operational Execution Checklist

Standardising Across a Collection

Tile size is a per-file property, and an archive that varies it scene by scene behaves unpredictably for readers and is impossible to reason about in a cost model. Standardising per collection — not per archive — is the arrangement that keeps both manageable.

One tile size per collection, chosen from its dominant reader Four collections each with a single standard tile size chosen from the way that collection is read. collectiondominant readerstandard tile aerial orthophotos public tile server256 px satellite scenes desktop GIS and notebooks512 px elevation models bulk analytical jobs1024 px point-cloud derived rasters mixed512 px

Record the standard in the collection’s profile alongside its compression and overview settings, and have the conversion job read it from there rather than taking a command-line argument. A collection whose files disagree about tile size usually got that way because someone converted a backlog by hand.

Frequently Asked Questions

Does tile size affect compression ratio?

Slightly. Larger tiles give the codec a longer run of similar pixels and typically compress two to four percent better; smaller tiles add per-tile overhead to the file’s internal offsets. Neither effect is large enough to influence the choice, which should be made on read behaviour.

Should overviews use the same tile size as the full-resolution data?

Yes in almost every case, and the COG driver does this automatically. A different tile size in the overviews complicates readers without helping: the overview levels are small enough that per-tile overhead is negligible, and consistency means one number describes the file’s behaviour at every zoom.

What tile size suits imagery served through a tile server?

256 pixels, matching the tile server’s own output grid, so one output tile maps onto one internal tile with no over-fetch. This is the one case where the alignment argument beats the general compromise — but only if the tile server is the dominant reader, since the same file then behaves poorly for bulk analytical access.

Does tile size interact with the compression choice?

Slightly, and in a predictable direction: larger tiles give the codec more context and compress marginally better, while smaller tiles produce more independently compressed blocks and therefore slightly more overhead. The effect is a few percent and should not influence the tile-size decision, which is dominated by read behaviour.

What tile size suits imagery that is only ever retrieved whole?

Any of them, and 512 for consistency. Where no reader ever requests a window, the internal layout is irrelevant to performance — but choosing the collection standard anyway costs nothing and means the file behaves well if that assumption changes, which over an archive’s lifetime it usually does.

How does tile size affect memory on the writing side?

The writer holds one tile per band in memory while compressing it, so a 1024-pixel tile on a 16-bit four-band raster is about 128 MB before any pipelining. On a large conversion fleet that multiplies by the number of concurrent scenes per worker, which is the usual cause of a job that runs fine on a workstation and is killed in a container.

Is there a way to serve two access patterns from one file?

Partly, through overviews rather than tile size. A client requesting a zoomed-out view reads a small overview level regardless of the tile size, so the tile-size compromise mainly affects full-resolution reads. Where two full-resolution patterns genuinely conflict — a tile server and a bulk analytic — the honest answer is two derived copies, and the delivery copy is the one that should carry the smaller tiles.

How much does the wrong tile size actually cost?

On a served collection, roughly the ratio of over-fetch: a 3.3× over-fetch against a 2.1× one is about fifty percent more transferred bytes for every window request, which on a busy collection is a visible line in the egress bill. On an unserved collection it costs nothing at all, which is why the decision should follow the reader rather than a general rule.

Does the choice need revisiting when a collection moves storage class?

Only if it moves into a class that cannot serve ranges, in which case tile size stops affecting reads until the object is restored. That is worth noting in the collection profile rather than acting on: the tile size remains correct for the restored copy, and rewriting the collection to change it would cost far more than the difference.

Up one level: Cloud-Optimized GeoTIFF Conversion Pipelines.

Related pages

Choosing Overview Levels and Resampling for Archived Imagery Converting GeoTIFF to Cloud-Optimized GeoTIFF at Scale Validating COG Structure Before Archival