Generating Vector Tiles From Lakehouse Tables

This guide builds a vector-tile pipeline on top of an ordinary spatial lakehouse table, reusing the same derived columns and partitioning the rest of the platform relies on, and precomputing only the tiles that are actually requested.

Context and prerequisites

A tile is an aggregate over a fixed grid cell, which makes tile generation structurally identical to the summaries covered in spatial aggregation and tiling rather than a separate discipline. This recipe uses PySpark 3.5 with Sedona for the geometric work and writes tiles to object storage; the simplification it applies per zoom level is covered in simplifying geometries for analytical layers.

The pipeline

Complete working solution

from pyspark.sql import functions as F
from sedona.spark import SedonaContext

# Tolerance per zoom: roughly one screen pixel at that zoom, in degrees.
TOLERANCE = {z: 360.0 / (256 * 2 ** z) for z in range(0, 16)}
BUFFER_FRACTION = 0.05          # clip slightly beyond the tile to avoid seam artefacts

def tile_bounds(z: int, x: int, y: int) -> tuple[float, float, float, float]:
    import math
    n = 2 ** z
    lon1 = x / n * 360.0 - 180.0
    lon2 = (x + 1) / n * 360.0 - 180.0
    lat1 = math.degrees(math.atan(math.sinh(math.pi * (1 - 2 * y / n))))
    lat2 = math.degrees(math.atan(math.sinh(math.pi * (1 - 2 * (y + 1) / n))))
    return (lon1, min(lat1, lat2), lon2, max(lat1, lat2))

def build_tiles(spark, table: str, z: int, tiles: list[tuple[int, int]]):
    tol = TOLERANCE[z]
    for x, y in tiles:
        minx, miny, maxx, maxy = tile_bounds(z, x, y)
        bx = (maxx - minx) * BUFFER_FRACTION
        by = (maxy - miny) * BUFFER_FRACTION

        features = (spark.table(table)
            .where(
                (F.col("bbox_max_x") >= minx - bx) & (F.col("bbox_min_x") <= maxx + bx) &
                (F.col("bbox_max_y") >= miny - by) & (F.col("bbox_min_y") <= maxy + by))
            .selectExpr(
                "feature_id", "category",
                f"ST_Intersection("
                f"  ST_SimplifyPreserveTopology(ST_GeomFromWKB(geom_wkb), {tol}),"
                f"  ST_MakeEnvelope({minx-bx}, {miny-by}, {maxx+bx}, {maxy+by})"
                f") AS geom")
            .where("NOT ST_IsEmpty(geom)"))

        payload = encode_tile(features.collect(), z, x, y)
        write_object(f"tiles/{z}/{x}/{y}.mvt", payload)

Step-by-step walkthrough

Filter on the bounding-box columns. This is the stage that determines whether tile generation is affordable, and it is the same numeric predicate every other query on the platform uses. A tile pipeline that filters with ST_Intersects against the tile envelope reads the whole table per tile.
Simplify before clipping, not after. Simplifying the clipped fragment produces artefacts at the tile edge, because the clip introduces vertices on the boundary that the simplifier then moves. Simplifying first and clipping second keeps edges aligned between adjacent tiles.
Clip with a small buffer. Features clipped exactly at the tile boundary produce visible seams when rendered, because line widths and symbols extend beyond the geometry. A five-percent buffer is a common default and the renderer discards the overflow.
Drop empty results. A feature whose simplified geometry no longer intersects the tile contributes nothing and should not be encoded. At low zooms this removes a large fraction of candidates.
Key the output by zoom, column and row. The conventional path layout means any tile server or static host can serve the result without an index.

Common errors and fixes

Symptom	Cause	Fix
Generation is extremely slow	Filtering with a geometry predicate instead of bbox columns	Use the numeric covering; it is the whole optimisation
Visible seams between tiles	Clipped exactly at the boundary	Clip with a buffer and let the renderer trim
Edges misaligned between adjacent tiles	Simplified after clipping	Simplify first, then clip
Small features vanish at low zoom	Tolerance exceeds the feature size	Expected; filter by size and drop them explicitly for clarity
Tile payloads enormous at low zoom	No simplification, or too many features	Increase tolerance with zoom; consider a feature-count cap per tile

Precomputing only what is browsed

Precomputing a full pyramid to zoom 14 is a common early decision and is almost always wasted: the overwhelming majority of tiles cover ocean, empty land or areas nobody looks at. Request logs from even a few days of real use identify the hot set, and it is typically a few percent of the theoretical space.

The workable arrangement is to precompute the hot set on the summary refresh schedule, generate anything else on first request, and cache the result. A cold tile costs a second or two to generate — acceptable for an unusual view — and becomes warm for everyone afterwards.

Verification

def verify_tile(payload, z, x, y, source_count_in_bounds):
    tile = decode_tile(payload)
    assert tile.layers, f"tile {z}/{x}/{y} has no layers"
    # Every geometry must lie within the tile plus its buffer.
    minx, miny, maxx, maxy = tile_bounds(z, x, y)
    for feature in tile.features:
        fminx, fminy, fmaxx, fmaxy = feature.bounds
        assert fminx >= minx - 1e-6 and fmaxx <= maxx + 1e-6, "feature escapes the tile"
    # Feature count must not exceed the source count in the same window.
    assert len(tile.features) <= source_count_in_bounds

The second assertion catches a duplication bug that is easy to introduce when a feature spans several tiles and the pipeline replicates rather than clips. It is worth having because the visual result of that bug is invisible — the map looks correct and the payloads are several times larger than they should be.

Sample a handful of tiles at each zoom rather than checking all of them, and include one tile at a boundary and one at the antimeridian, which are the two places tile pipelines most often go wrong.

Controlling Payload Size

Tile size is the metric that determines whether a map feels responsive, and three techniques keep it bounded without degrading the view.

The attribute budget is the most commonly overlooked. A tile carrying every column of the source table ships attributes that the map style never references, and on feature-dense tiles the attribute block can exceed the geometry block. Selecting the two or three fields the style actually uses, and fetching the rest on click from an ordinary lookup, typically halves the payload.

Dropping sub-pixel features should be explicit rather than left to the simplifier. A polygon smaller than a pixel at the current zoom contributes nothing visible, and filtering on a size threshold removes it deterministically — whereas relying on simplification to collapse it produces inconsistent results and occasional invalid geometry.

Set a payload budget per tile and measure against it. A few hundred kilobytes is a common ceiling; tiles consistently exceeding it at a given zoom mean the tolerance or the threshold for that zoom needs adjusting, and the measurement makes that a tuning exercise rather than a guess.

Refreshing Tiles Incrementally

A full pyramid rebuild after every data change is untenable, and the incremental path is straightforward because the affected tiles are computable.

When a batch of features changes, compute the union of their bounding boxes, expand it into tile coordinates at each zoom level, and regenerate only those tiles. For a localised update — a district’s boundaries revised, a day of new observations in one city — this is a few hundred tiles rather than millions.

Two details make it correct. The buffer must be accounted for: a feature near a tile edge affects the neighbouring tile too, so expand the affected set by one tile in each direction. And deletions must be handled: a feature removed from the source still appears in the cached tile until that tile is regenerated, so deletions must trigger regeneration exactly as insertions do — which requires knowing where the deleted feature was, and therefore recording its extent before removing it.

Track the tile cache’s own staleness alongside the summary staleness described in the parent topic. A tile that was generated from a snapshot three days old is serving three-day-old geometry, and a client has no way to know unless the pipeline says so.

Where the Lakehouse Helps

Building tiles from a lakehouse table rather than from a dedicated spatial database has three specific advantages worth naming, because they are the reason to do it this way rather than to maintain a separate serving database.

One copy of the data. The tiles are derived from the same table the analytics query, so a boundary revision reaches the map and the report at the same moment and from the same source. Platforms that maintain a separate rendering database inevitably see the two diverge, and reconciling them is a recurring cost.

The same layout work pays twice. The bounding-box columns, the partitioning and the sort order that make analytical queries fast make tile generation fast, for exactly the same reason: both are selecting a small spatial window from a large table. No tile-specific indexing is needed.

Snapshot-consistent tiles. Generating a pyramid from a pinned snapshot means every tile in it reflects the same state, which a live database cannot guarantee during a bulk update. For a map whose tiles are compared against each other — a before-and-after, an animation across time — that consistency is the difference between a coherent result and a flickering one.

The cost is latency on the generation path, which is why the precomputation and caching described above matter. A lakehouse will not serve a cold tile in ten milliseconds; it will serve a warm one from object storage in single-digit milliseconds, which is the same thing from the client’s perspective and requires no additional database to operate.

Generating Vector Tiles From Lakehouse Tables

Related in this section