Abstract
We focus on the domain adaptation problem for 3D semantic segmentation, addressing the challenge of data variability in point clouds collected by different LiDARs. Existing benchmarks often mix different types of datasets, which blurs and complicates segmentation evaluations. Here, we introduce a Multi-LiDARs Domain Adaptation Segmentation (MLDAS) dataset, which contains point-wise semantic annotated point clouds captured simultaneously by a 128-beam LiDAR, a 64-beam LiDAR, and a 32-beam LiDAR. We select 31,875 scans from two representative scenarios: campus and urban street. Furthermore, we evaluate current 3D segmentation unsupervised domain adaptation methods on the proposed dataset and propose the Hierarchical Segmentation Network with Spatial Consistency (HSSC) as a novel knowledge transfer method to mitigate the domain gap significantly using spatial-temporal consistency constraints. Extensive experiments show that HSSC greatly improves the state-of-the-art cross-domain semantic segmentation methods.
Introduction
In autonomous driving and robotics, 3D semantic segmentation is essential for environmental perception, allowing systems to categorize every point in a 3D scan. However, models trained on specific sensor data often fail when deployed on different hardware or in new environments—a phenomenon known as the domain gap. This gap is primarily driven by sensor disparities (resolution, beam count) and scenario variability (object distribution). Existing benchmarks frequently conflate these factors, complicating the isolation of specific challenges in cross-sensor and cross-scenario adaptation.
The MLDAS Dataset
To address these challenges, we present the Multi-LiDARs Domain Adaptation Segmentation (MLDAS) dataset. Unlike previous benchmarks like SemanticKITTI or nuScenes, MLDAS provides perfectly synchronized data across different hardware.
Hardware and Scale
The dataset utilizes three mechanical spinning LiDARs: a Hesai XT32 (32-beam), an Ouster OS64 (64-beam), and an Ouster OS128 (128-beam). We collected 31,875 scans across two distinct scenarios: a university Campus and a busy Urban Street.
Annotation and Synchronization
The dataset includes 14 semantic categories. Labels were meticulously generated for the high-resolution 128-beam data and then mathematically projected onto the 64-beam and 32-beam scans using precise sensor calibration. This simultaneous capture allows researchers to decouple the "hardware gap" from the "environment gap."
Hierarchical Segmentation Network with Spatial Consistency (HSSC)
We propose HSSC, a novel framework designed to bridge these domain gaps using a "Teacher-Student" architecture and spatial-temporal consistency.
- Hierarchical Learning: HSSC utilizes an intermediate domain to bridge the gap between a labeled source (e.g., Campus with 128-beam LiDAR) and an unlabeled target (e.g., Street with 32-beam LiDAR). This allows the model to adapt gradually to new sensor characteristics.
- Spatial Consistency Constraints: By leveraging the synchronized multi-sensor data in MLDAS, HSSC enforces a rule: if two different sensors observe the same physical space, the model must assign them identical semantic labels.
- Mean-Teacher Architecture: A "Teacher" model provides pseudo-labels for a "Student" model, which learns to maintain consistency while adapting to the target's unique data distribution.
Results
HSSC demonstrates significant improvements over existing state-of-the-art Unsupervised Domain Adaptation (UDA) methods.
In the challenging "Campus(XT32) → Street(OS128)" task, HSSC achieved a 45.19 mIOU, more than doubling the performance of the "Source-only" baseline (20.80 mIOU). It also significantly outperformed other popular methods such as CosMix (30.52 mIOU) and PolarMix (25.46 mIOU). These results highlight the effectiveness of leveraging spatial consistency and intermediate domains, especially when migrating between sensors with different beam counts.
Conclusion and Future Work
While HSSC and MLDAS represent a major step forward, some limitations remain. The HSSC training strategy currently requires synchronized multi-LiDAR data for the intermediate domain. Additionally, fast-moving dynamic objects still present challenges for point-to-point matching across different scan patterns. Future research will explore more complex transition domains and the application of multi-sensor consistency to tasks like 3D object detection and motion forecasting.
Project page: https://sychen320.github.io/projects/MLDAS
Summary updated