TL;DR
This paper introduces AttProto, a novel source-free unsupervised domain adaptation method for 3D object detection. It tackles the challenge of noisy pseudo-labels during self-training by using an "attentive prototype" mechanism, powered by transformers, to identify and down-weight incorrect annotations, leading to improved detection performance on unseen target domains.
What Problem Does This Paper Address?
Autonomous driving systems rely heavily on 3D object detection, often trained on large LiDAR datasets. However, these models exhibit a significant performance drop when deployed in environments with different sensor specifications, weather conditions, or driving customs – a phenomenon known as the "domain gap." Traditional domain adaptation methods often require access to the original source training data, which may be unavailable due to privacy or storage constraints. Furthermore, existing "source-free" adaptation methods that rely on pseudo-labeling via self-training can inadvertently reinforce errors if the pseudo-labels are noisy and over-confident, especially when dealing with subtle differences between object categories. This paper addresses the critical need for effective 3D object detection adaptation in a source-free setting, specifically by mitigating the impact of noisy pseudo-labels generated during self-training.
Methods
The core of AttProto lies in its novel "attentive prototype" computation. Instead of simply averaging features from detected regions to form a class prototype, AttProto uses a transformer module. This transformer leverages self-attention to identify salient "regions-of-interest" within the detected objects, effectively learning relationships between features. This allows it to focus on more informative features for prototype creation.
Crucially, AttProto incorporates "predictive entropy" from the classifier's output. This entropy quantifies the model's uncertainty about a prediction. By weighting the attentive region features with their corresponding predictive entropy (or rather, a measure derived from it), the method down-weights features from regions where the model is uncertain, which are more likely to be mis-labeled. This "attentive class prototype" is then used to refine pseudo-labels: regions whose features are dissimilar to this prototype are considered incorrect and their associated losses are down-weighted during the self-training process. This iterative refinement helps to clean up the pseudo-labels and improve the robustness of the adaptation.
Datasets
The paper evaluates its approach on three popular large-scale autonomous driving datasets:
- Waymo Open Dataset: Used with approximately 50K frames for training and validation.
- nuScenes dataset: Utilized with 34,149 frames, split into 28K for training and 6K for validation.
- KITTI dataset: Used with 7,481 frames, split into 3K for training and 3K for validation.
These datasets are chosen for their significant domain gaps, simulating realistic deployment challenges. The experiments employ an inductive setup, meaning the target domain test data is not seen during adaptation.
Key Insights and Ideas
- Transformer for Salient Feature Extraction: The use of transformers with self-attention is key to identifying the most informative features for prototype computation, moving beyond simple averaging.
- Predictive Entropy as a Noise Filter: Leveraging the classifier's uncertainty (predictive entropy) provides a principled way to down-weight noisy or incorrect pseudo-labels, a significant improvement over simple confidence thresholding.
- Attentive Prototypes for Robustness: The "attentive prototype" acts as a more robust representation of a class, less susceptible to noise from mis-labeled examples, leading to better pseudo-label refinement.
- Source-Free Adaptation is Crucial: The focus on a source-free setting makes the method highly practical for real-world scenarios where source data is often inaccessible.
Key Results
AttProto demonstrates strong performance across various domain shift scenarios.
- On the Waymo → KITTI shift, AttProto achieves state-of-the-art results for the "Car" category with the SECOND-iou detector, outperforming methods like ST3D and SF-UDA3D.
- For the nuScenes → KITTI shift, AttProto also shows competitive results, particularly in the "easy" and "moderate" difficulty categories.
- When adapting to nuScenes → KITTI with PointRCNN, AttProto achieves the best results in most categories, surpassing weakly supervised methods and prior domain adaptation approaches.
- Qualitative results show AttProto effectively mitigates false positives and improves localization accuracy compared to direct transfer and other self-training methods.
Limitations and Future Work
The authors acknowledge that AttProto under-performs in the "hard" category, which includes objects with high occlusion, truncation, and those far from the sensor. These objects are often underrepresented in datasets and can be challenging to detect even with ground truth. Poor localization from the source-trained model can further exacerbate this issue.
Future work will focus on addressing these limitations, particularly by exploring label-free augmentation techniques. The authors also aim to provide a fully unsupervised solution for addressing domain shifts related to vehicle size differences, which are currently handled by weakly supervised methods.
Summary updated