LiDARNet: A Boundary-Aware Domain Adaptation Model for Point Cloud Semantic Segmentation

Peng Jiang, Srikanth Saripalli

Abstract

LiDAR semantic segmentation is a cornerstone of autonomous perception, yet models often suffer from significant performance degradation when deployed across different sensor configurations or environments—a challenge known as the domain gap. We present LiDARNet, a novel unsupervised domain adaptation (UDA) framework that bridges this gap without requiring target-domain annotations. By utilizing a private-shared feature extraction architecture and integrating boundary-aware learning, LiDARNet isolates sensor-specific noise from universal geometric structures. Our evaluation across SemanticKITTI, SemanticPOSS, and the newly introduced SemanticUSL datasets demonstrates that LiDARNet achieves an 8% to 22% improvement in mean Intersection over Union (mIoU) on target domains compared to baseline methods.

Introduction

Deep learning models for 3D scene understanding are notoriously brittle. A model trained on a 64-beam Velodyne sensor in a highway environment often fails when transferred to a 64-beam Ouster sensor in an urban campus due to variations in beam angles, noise patterns, and reflectivity distributions. Manually labeling 3D point clouds to retrain these models is prohibitively expensive and time-consuming.

LiDARNet addresses this by focusing on Unsupervised Domain Adaptation (UDA). The goal is to leverage a labeled "source" dataset to achieve high performance on an unlabeled "target" dataset, ensuring that autonomous systems can adapt to new sensors and environments with minimal human intervention.

Methodology

LiDARNet employs several architectural innovations to facilitate robust cross-domain transfer:

1. 2D Range Map Projection

To optimize computational efficiency and leverage established 2D convolutional neural network (CNN) architectures, LiDARNet projects raw 3D point clouds into 2D "range maps." This representation simplifies the spatial reasoning required for segmentation while maintaining the underlying geometric relationships of the scene.

2. Private-Shared Feature Extraction

A core innovation of LiDARNet is its dual-stream feature extraction. The model utilizes two distinct extractors:

  • Domain-Shared Extractor: Captures geometry and semantic features (e.g., the shape of a vehicle) that are consistent across different datasets.
  • Domain-Private Extractor: Isolates sensor-specific noise, intensity variations, and unique scan patterns. By explicitly separating these features, the model prevents domain-specific artifacts from polluting the semantic understanding of the scene.

3. Boundary-Aware Learning

Physical boundaries—such as the edges of a building or the silhouette of a pedestrian—remain relatively constant even when point density or reflectivity changes. LiDARNet integrates a Gated-SCNN (Gated Shape CNN) to explicitly learn these boundaries within the 2D projection. This focus on shape and structure makes the model more resilient to the "blurring" effects of domain shifts.

4. Adversarial Alignment and Multi-task Loss

The framework utilizes a CycleGAN mechanism to translate data between source and target domains, ensuring the shared features are truly representative of both environments. The training process is governed by a multi-task loss function that balances segmentation accuracy, boundary detection, domain classification, and feature orthogonality (ensuring private and shared features remain distinct).

Experimental Setup and Datasets

The model was evaluated using three real-world datasets:

  • SemanticKITTI: A standard benchmark for autonomous driving in urban and highway settings.
  • SemanticPOSS: A dataset featuring complex campus scenes with high dynamic instance density.
  • SemanticUSL: Introduced in this research, this dataset was collected using a Clearpath Warthog robot at Texas A&M University. It includes diverse terrains such as walkpaths and off-road scenes, providing a rigorous test for domain adaptation.

Results and Discussion

LiDARNet consistently outperformed source-only baselines and existing pixel-level adaptation methods like CyCADA. Key findings include:

  • Significant mIoU Gains: The model achieved an 8% to 22% mIoU increase on target datasets, proving the effectiveness of the private-shared feature separation.
  • Source Performance Preservation: Unlike many UDA methods that suffer from "catastrophic forgetting," LiDARNet maintained its performance on the source domain after adapting to the target.
  • Superior Transferability: In the KITTI → USL task, the adapted LiDARNet model actually outperformed a model trained directly on the limited labeled USL data, highlighting the value of transferring knowledge from larger, high-quality source datasets.

Conclusion

LiDARNet demonstrates that boundary-awareness and explicit feature separation are powerful tools for overcoming the domain gap in 3D perception. By focusing on universal geometric truths rather than sensor-specific noise, the model provides a scalable path for deploying autonomous systems across diverse and evolving environments. Future research will explore more complex 3D representations and further optimize the architecture for real-time edge deployment.

Summary updated

Abstract

We present a boundary-aware domain adaptation model for LiDAR scan full-scene semantic segmentation (LiDARNet). Our model can extract both the domain private features and the domain shared features with a two branch structure. We embedded Gated-SCNN into the segmentor component of LiDARNet to learn boundary information while learning to predict full-scene semantic segmentation labels. Moreover, we further reduce the domain gap by inducing the model to learn a mapping between two domains using the domain shared and private features. Additionally, we introduce a new dataset (SemanticUSL1) for domain adaptation for LiDAR point cloud semantic segmentation. The dataset has the same data format and ontology as SemanticKITTI. We conducted experiments on real-world datasets SemanticKITTI, SemanticPOSS, and SemanticUSL, which have differences in channel distributions, reflectivity distributions, diversity of scenes, and sensors setup. Using our approach, we can get a single projection-based Li-DAR full-scene semantic segmentation model working on both domains. Our model can keep almost the same performance on the source domain after adaptation and get an 8%-22% mIoU performance increase in the target domain.
BibTeX
@inproceedings{jiang2021lidarnet,
  author = {Peng Jiang and Srikanth Saripalli},
  title = {{LiDARNet: A Boundary-Aware Domain Adaptation Model for Point Cloud Semantic Segmentation}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  doi = {10.1109/icra48506.2021.9561255},
}
Figure