EXPLORING CROSS-CITY SEMANTIC SEGMENTATION OF ALS POINT CLOUDS

Yu Xie, Konrad Schindler, Jiaojiao Tian, Xiao Xiang Zhu2021 · DOI

Abstract

Deep learning models achieve excellent semantic segmentation results for airborne laser scanning (ALS) point clouds, if sufficient training data are provided. However, models trained on a specific dataset typically exhibit poor performance on other datasets due to significant domain shifts. In this work, we study this domain shift and potential strategies to mitigate it, using the ISPRS Vaihingen and LASDU benchmarks. We analyze factors such as point cloud density, LiDAR intensity, and data augmentation. We also evaluate Deep CORAL for domain adaptation. Our experiments show that adapting point cloud density and appropriate data augmentation significantly reduce the domain gap, while intensity features can deteriorate generalization. Deep CORAL provides marginal improvements when basic strategies are already optimized.

Introduction: The Challenge of Domain Shift

In Airborne Laser Scanning (ALS), semantic segmentation—the automated labeling of points as ground, buildings, or trees—is a critical task. While deep learning has pushed the boundaries of accuracy, these models often fail to generalize across different urban environments. A model trained on a European city (e.g., Vaihingen, Germany) often performs poorly when applied to an Asian city (e.g., Lanzhou, China).

This "domain shift" is driven by variations in sensor characteristics, flight parameters (such as altitude), and distinct urban morphologies. These factors result in differing point densities, noise levels, and signal distributions. Currently, bridging this gap often requires expensive manual labeling for each new project. This study investigates whether "back-to-basics" engineering or formal domain adaptation can mitigate these effects.

Methodology

The researchers employed a state-of-the-art pipeline based on Kernel Point Convolution (KPConv) within a residual U-Net architecture. KPConv is advantageous for ALS data as it operates directly on 3D coordinates, preserving geometric relationships better than voxel-based methods.

To address domain shift, the study compared four strategies:

  1. Point Density Control: Optimizing the input grid size ($l$) to balance context and geometric detail.
  2. Data Augmentation: Implementing random rotation, scaling, and Gaussian noise to encourage feature invariance.
  3. Feature Selection: Evaluating the impact of LiDAR intensity on cross-dataset performance.
  4. Deep CORAL: An unsupervised domain adaptation method that aligns second-order statistics (covariance) between source and target features during training.
Architecture of the KPConv-based U-Net with Deep CORAL for domain adaptation.

Experimental Setup

The study utilized two major ALS benchmarks:

  • ISPRS Vaihingen (Germany): High-detail data captured at ~500m altitude.
  • LASDU (China): Data captured at ~1200m altitude over a dense urban area.

To ensure compatibility, the Vaihingen classes were mapped to the LASDU 6-class system: Ground, Building, Tree, Low Vegetation, Artifact, and Others.

Key Insights and Results

The experiments revealed several critical factors for cross-city generalization:

  • The Intensity Trap: While LiDAR intensity improves within-city accuracy, it is detrimental to generalization. Different sensors and atmospheric conditions produce inconsistent intensity scales, causing the model to learn city-specific signatures that do not transfer. Including intensity led to a 27 percentage point drop in accuracy for the Germany-to-China transfer ($VH \rightarrow LS$).
  • Optimal Grid Size: A "Goldilocks" grid size of $0.5m$ was found to be most effective for transfer. Finer grids ($0.25m$) lacked sufficient context, while coarser grids ($0.8m$) obscured the geometric differences between ground and low vegetation.
  • The Power of Augmentation: Data augmentation is essential; its absence resulted in a 6–8 percentage point drop in cross-city accuracy.
  • Deep CORAL's Role: Deep CORAL was most effective when baseline settings (like augmentation) were sub-optimal, providing a boost of 2.8 to 4.2 pp. However, its benefits diminished when basic engineering was already optimized.
Qualitative comparison of segmentation results for cross-city transfer (LS to VH) under different configurations.

Conclusion

The study concludes that fundamental data preparation—specifically optimizing point density and robust data augmentation—is more impactful for cross-city ALS segmentation than complex domain adaptation algorithms like Deep CORAL. Future work should address "class-aware" adaptation to resolve persistent confusion between geometrically similar classes and improve the detection of culturally variable "Artifacts" like cars and fences.

Summary updated

Abstract

Deep learning models achieve excellent semantic segmentation results for airborne laser scanning (ALS) point clouds, if sufficient training data are provided. Increasing amounts of annotated data are becoming publicly available thanks to contributors from all over the world. However, models trained on a specific dataset typically exhibit poor performance on other datasets. I.e., there are significant domain shifts, as data captured in different environments or by distinct sensors have different distributions. In this work, we study this domain shift and potential strategies to mitigate it, using two popular ALS datasets: the ISPRS Vaihingen benchmark from Germany and the LASDU benchmark from China. We compare different training strategies for cross-city ALS point cloud semantic segmentation. In our experiments, we analyse three factors that may lead to domain shift and affect the learning: point cloud density, LiDAR intensity, and the role of data augmentation. Moreover, we evaluate a well-known standard method of domain adaptation, deep CORAL (Sun and Saenko, 2016). In our experiments, adapting the point cloud density and appropriate data augmentation both help to reduce the domain gap and improve segmentation accuracy. On the contrary, intensity features can bring an improvement within a dataset, but deteriorate the generalisation across datasets. Deep CORAL does not further improve the accuracy over the simple adaptation of density and data augmentation, although it can mitigate the impact of improperly chosen point density, intensity features, and further dataset biases like lack of diversity.
BibTeX
@inproceedings{xie2021exploring,
  author = {Yu Xie and Konrad Schindler and Jiaojiao Tian and Xiao Xiang Zhu},
  title = {{EXPLORING CROSS-CITY SEMANTIC SEGMENTATION OF ALS POINT CLOUDS}},
  booktitle = {˜The},
  year = {2021},
  doi = {10.5194/isprs-archives-xliii-b2-2021-247-2021},
}
Figure