Geo-Tiles for Semantic Segmentation of Earth Observation Imagery

Sebastian Bullinger, Florian Fervers, Christoph Bodensteiner, Michael ArensarXiv 2023 · DOI · arXiv

Abstract

Semantic segmentation of large-scale Earth Observation (EO) imagery is often constrained by hardware memory, necessitating the partitioning of images into smaller tiles. Traditional pixel-based tiling methods suffer from scale inconsistency, latitudinal distortion, and accuracy degradation at tile boundaries. This paper introduces Geo-Tiles (EOT), a tiling framework defined in real-world units (meters). By incorporating geographic metadata and a "Tile Fusion" strategy, the proposed method ensures consistent spatial scaling and significantly reduces boundary prediction errors.

Introduction

Earth Observation (EO) images are typically too large to fit into GPU memory for end-to-end semantic segmentation. The standard solution is to divide these images into smaller "tiles." However, current pixel-based approaches have three major flaws:

  1. Scale Inconsistency: Pixel-based tiling ignores the Ground Sampling Distance (GSD). A fixed pixel tile covers different physical areas depending on the sensor resolution, forcing models to learn objects at wildly different scales.
  2. Latitudinal Distortion: Common map projections (like Web Mercator) cause physical area distortion at different latitudes, further complicating scale normalization.
  3. The Context Gap: Deep learning models struggle at tile edges due to a lack of surrounding spatial information, leading to visible "seams" and misclassifications at boundaries.
Standard tiling (top) often results in uneven coverage at image boundaries, whereas symmetric tiling (bottom) ensures a centered grid.

Methodology

The authors propose a mathematically formalized tiling pipeline that shifts the focus from pixels to geography.

Geo-Tiles (EOT)

Tiles are defined by their physical extent in meters. The system uses the image's GSD and Coordinate Reference System (CRS) to calculate the exact pixel coordinates for a tile. This ensures that a "75-meter tile" always covers 75 meters of ground, regardless of the sensor or location, effectively pre-normalizing the input scale for the neural network.

Optimal Coverage and Overlapping Augmentation

To avoid inefficient "sliver" tiles at image boundaries, a symmetric tiling formula is used to center the grid over the target area. During training, the system generates overlapping tiles by reducing the stride between tiles. This serves as a specialized form of data augmentation for geospatial data, exposing the model to various spatial contexts for the same ground features.

The Geo-Tiles framework enables various tiling configurations, including standard, symmetric, and overlapping grids to optimize training and inference.

Tile Fusion (Inference)

To solve the "Context Gap," the authors introduce a Tile Fusion strategy based on the observation that prediction error increases as one moves from the tile center toward the edge. During inference, overlapping tiles are processed, and "unreliable" edge pixels are replaced with the "reliable" center pixels of neighboring tiles. This straightforward substitution is computationally efficient and effectively eliminates boundary artifacts.

Datasets and Results

The framework was validated using the Open Cities AI Challenge and ISPRS 2D Potsdam datasets. Key results include:

  • Optimal Tile Size: For building segmentation on the Open Cities AI dataset, 75-meter tiles yielded the best performance (84.55 mIoU).
  • Augmentation and Fusion Gains: Using overlapping tiles during training provided consistent mIoU improvements. The Tile Fusion strategy further improved mIoU by approximately 0.22 points. While the numerical jump is modest, the qualitative improvement—specifically the removal of jagged edges at tile seams—is significant for real-world mapping.
  • Architecture Performance: Testing several state-of-the-art models revealed that UPerNet with a ConvNext backbone offered the best balance of accuracy and memory efficiency for EO data.

Conclusion

Geo-Tiles (EOT) addresses fundamental limitations in geospatial image processing by grounding tiling in physical units. By prioritizing physical scale and addressing boundary reliability through Tile Fusion, the framework enables more consistent and accurate large-scale mapping. The authors have made their full tile processing source code publicly available to support the geospatial research community.

Summary updated

Abstract

To cope with the high requirements during the computation of semantic segmentations of earth observation imagery, current state-of-the-art pipelines divide the corresponding data into smaller images. Existing methods and benchmark datasets oftentimes rely on pixel-based tiling schemes or on geo-tiling schemes employed by web mapping applications. The selection of subimages (comprising size, location and orientation) is crucial. It affects the available context information of each pixel, defines the number of tiles during training, and influences the degree of information degradation while down- and up-sampling the tile contents to the size required by the segmentation model. We propose a new segmentation pipeline for earth observation imagery relying on a tiling scheme that creates geo-tiles based on the geo-information of the raster data. This approach exhibits several beneficial properties compared to pixel-based or common web mapping approaches. The proposed tiling scheme shows flexible customization properties regarding tile granularity, tile stride and image boundary alignment. This allows us to perform a tile specific data augmentation during training and a substitution of pixel predictions with limited context information using data of overlapping tiles during inference. The generated tiles show a consistent spatial tile extent w.r.t. heterogeneous sensors, varying recording distances and different latitudes. We demonstrate how the proposed tiling system allows to improve the results of current state-of-the-art semantic segmentation models. To foster future research we make the source code publicly available.
BibTeX
@misc{bullinger2023geo,
  author = {Sebastian Bullinger and Florian Fervers and Christoph Bodensteiner and Michael Arens},
  title = {{Geo-Tiles for Semantic Segmentation of Earth Observation Imagery}},
  year = {2023},
  eprint = {2306.00823},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2306.00823},
}
Figure