Abstract
Unsupervised domain adaptation for point cloud semantic segmentation has attracted great attention due to its effectiveness in learning with unlabeled data. Most of existing methods use global-level feature alignment to transfer the knowledge from the source domain to the target domain, which may cause the semantic ambiguity of the feature space. In this paper, we propose a graph-based framework to explore the local-level feature alignment between the two domains, which can reserve semantic discrimination during adaptation. Specifically, in order to extract local-level features, we first dynamically construct local feature graphs on both domains and build a memory bank with the graphs from the source domain. In particular, we use optimal transport to generate the graph matching pairs. Then, based on the assignment matrix, we can align the feature distributions between the two domains with the graph-based local feature loss. Furthermore, we consider the correlation between the features of different categories and formulate a category-guided contrastive loss to guide the segmentation model to learn discriminative features on the target domain. Extensive experiments on different synthetic-to-real and real-to-real domain adaptation scenarios demonstrate that our method can achieve state-of-the-art performance.
Introduction
Deep learning models for 3D point cloud segmentation are notoriously sensitive to the specific sensor used to collect data. A model trained on high-density, 64-beam LiDAR data (the source domain) often fails when deployed on lower-density, 32-beam data (the target domain) due to domain shift.
Existing Unsupervised Domain Adaptation (UDA) methods typically attempt to align the global feature distributions of the two domains. However, global alignment is often too coarse, leading to "semantic ambiguity" where features from distinct categories (e.g., a pole versus a tree) are blurred together in the latent space. This paper addresses the challenge by aligning domains at a local, geometric level, ensuring the model recognizes that a "car-shaped" cluster in the source domain corresponds to a similar cluster in the target domain, regardless of point density variations.
Proposed Methodology
The proposed framework shifts the focus from global distribution alignment to local-level graph matching. The architecture consists of three core components designed to maintain semantic discrimination during the adaptation process.
1. Dynamic Feature Graphs
Instead of processing the entire scene globally, the model utilizes Farthest Point Sampling (FPS) to select centroid points and constructs local graphs around them. These graphs capture k-nearest neighbors and their relative distances, effectively encoding local geometric structures that are more robust to density changes than global statistics.
2. Feature Graph Memory Bank
To overcome the limited variety of objects present in a single training batch, the authors maintain a memory bank of source-domain feature graphs. This ensures the model always has access to a rich "dictionary" of labeled source features to compare against target features, stabilizing the learning process for rare classes.
3. Graph Matching via Optimal Transport
To find which source graph matches a given target graph, the framework employs the Sinkhorn algorithm to solve an Optimal Transport (OT) problem. This produces an assignment matrix providing point-level correspondences between the two domains. Based on this alignment, a graph-based local feature loss ($L_{loc}$) and a category-guided contrastive loss ($L_{con}$) are applied. The contrastive loss pushes target features closer to "positive" (same-category) source features and further from "negative" ones, sharpening the model's discriminative power on unlabeled data.
Experimental Evaluation
The researchers evaluated their method across three distinct domain adaptation scenarios: synthetic-to-real (vKITTI $\to$ SemanticPOSS), indoor-to-indoor (S3DIS $\to$ ScanNet), and real-to-real outdoor (SemanticKITTI $\to$ nuScenes).
Key Results
The proposed method achieved state-of-the-art (SOTA) performance across all tested benchmarks, measured by mean Intersection-over-Union (mIoU):
- vKITTI $\to$ SemanticPOSS: Reached 54.9% mIoU, significantly outperforming the "Source Only" baseline (44.6%).
- S3DIS $\to$ ScanNet: Achieved 53.8% mIoU, a massive jump from the baseline (43.2%).
- SemanticKITTI $\to$ nuScenes: Reached 37.3% mIoU, beating the previous best-reported method (C&L) which sat at 31.6%.
Qualitative visualizations confirm that the method successfully clusters target-domain features by category much more cleanly than baseline approaches. As shown in the outdoor adaptation results, the model maintains high accuracy for critical categories like drivable surfaces and vehicles despite the hardware-induced domain shift.
Conclusion
This work demonstrates that local geometric consistency is a more reliable signal than global statistical consistency when adapting point cloud models across different sensors. By matching local neighborhoods through a principled Optimal Transport framework and utilizing a memory bank for stability, the proposed method effectively retains the "shape" of features across domains. Future work may explore extending this graph-matching approach to multi-modal scenarios, such as fusing 2D images with 3D point clouds, to further enhance robustness against domain shift.
Summary updated