Attentive Prototypes for Source-Free Unsupervised Domain Adaptive 3D Object Detection

Deepti Hegde, Vishal M. Patel

TL;DR

This paper introduces AttProto, a novel source-free unsupervised domain adaptation method for 3D object detection. It tackles the challenge of noisy pseudo-labels during self-training by using an "attentive prototype" mechanism, powered by transformers, to identify and down-weight incorrect annotations, leading to improved detection performance on unseen target domains.

What Problem Does This Paper Address?

Autonomous driving systems rely heavily on 3D object detection, often trained on large LiDAR datasets. However, these models exhibit a significant performance drop when deployed in environments with different sensor specifications, weather conditions, or driving customs – a phenomenon known as the "domain gap." Traditional domain adaptation methods often require access to the original source training data, which may be unavailable due to privacy or storage constraints. Furthermore, existing "source-free" adaptation methods that rely on pseudo-labeling via self-training can inadvertently reinforce errors if the pseudo-labels are noisy and over-confident, especially when dealing with subtle differences between object categories. This paper addresses the critical need for effective 3D object detection adaptation in a source-free setting, specifically by mitigating the impact of noisy pseudo-labels generated during self-training.

Methods

The core of AttProto lies in its novel "attentive prototype" computation. Instead of simply averaging features from detected regions to form a class prototype, AttProto uses a transformer module. This transformer leverages self-attention to identify salient "regions-of-interest" within the detected objects, effectively learning relationships between features. This allows it to focus on more informative features for prototype creation.

Crucially, AttProto incorporates "predictive entropy" from the classifier's output. This entropy quantifies the model's uncertainty about a prediction. By weighting the attentive region features with their corresponding predictive entropy (or rather, a measure derived from it), the method down-weights features from regions where the model is uncertain, which are more likely to be mis-labeled. This "attentive class prototype" is then used to refine pseudo-labels: regions whose features are dissimilar to this prototype are considered incorrect and their associated losses are down-weighted during the self-training process. This iterative refinement helps to clean up the pseudo-labels and improve the robustness of the adaptation.

Datasets

The paper evaluates its approach on three popular large-scale autonomous driving datasets:

  • Waymo Open Dataset: Used with approximately 50K frames for training and validation.
  • nuScenes dataset: Utilized with 34,149 frames, split into 28K for training and 6K for validation.
  • KITTI dataset: Used with 7,481 frames, split into 3K for training and 3K for validation.

These datasets are chosen for their significant domain gaps, simulating realistic deployment challenges. The experiments employ an inductive setup, meaning the target domain test data is not seen during adaptation.

Key Insights and Ideas

  • Transformer for Salient Feature Extraction: The use of transformers with self-attention is key to identifying the most informative features for prototype computation, moving beyond simple averaging.
  • Predictive Entropy as a Noise Filter: Leveraging the classifier's uncertainty (predictive entropy) provides a principled way to down-weight noisy or incorrect pseudo-labels, a significant improvement over simple confidence thresholding.
  • Attentive Prototypes for Robustness: The "attentive prototype" acts as a more robust representation of a class, less susceptible to noise from mis-labeled examples, leading to better pseudo-label refinement.
  • Source-Free Adaptation is Crucial: The focus on a source-free setting makes the method highly practical for real-world scenarios where source data is often inaccessible.

Key Results

AttProto demonstrates strong performance across various domain shift scenarios.

  • On the Waymo → KITTI shift, AttProto achieves state-of-the-art results for the "Car" category with the SECOND-iou detector, outperforming methods like ST3D and SF-UDA3D.
  • For the nuScenes → KITTI shift, AttProto also shows competitive results, particularly in the "easy" and "moderate" difficulty categories.
  • When adapting to nuScenes → KITTI with PointRCNN, AttProto achieves the best results in most categories, surpassing weakly supervised methods and prior domain adaptation approaches.
  • Qualitative results show AttProto effectively mitigates false positives and improves localization accuracy compared to direct transfer and other self-training methods.

Limitations and Future Work

The authors acknowledge that AttProto under-performs in the "hard" category, which includes objects with high occlusion, truncation, and those far from the sensor. These objects are often underrepresented in datasets and can be challenging to detect even with ground truth. Poor localization from the source-trained model can further exacerbate this issue.

Future work will focus on addressing these limitations, particularly by exploring label-free augmentation techniques. The authors also aim to provide a fully unsupervised solution for addressing domain shifts related to vehicle size differences, which are currently handled by weakly supervised methods.

Summary updated

Abstract

3D object detection networks tend to be biased towards the data they are trained on. Evaluation on datasets captured in different locations, conditions or sensors than that of the training (source) data results in a drop in model performance due to the gap in distribution with the test (or target) data. Current methods for domain adaptation either assume access to source data during training, which may not be available due to privacy or memory concerns, or require a sequence of lidar frames as an input. We propose a single-frame approach for source-free, unsupervised domain adaptation of lidar-based 3D object detectors that uses class prototypes to mitigate the effect pseudo-label noise. Addressing the limitations of traditional feature aggregation methods for prototype computation in the presence of noisy labels, we utilize a transformer module to identify outlier ROI's that correspond to incorrect, over-confident annotations, and compute an attentive class prototype. Under an iterative training strategy, the losses associated with noisy pseudo labels are down-weighed and thus refined in the process of self-training. To validate the effectiveness of our proposed approach, we examine the domain shift associated with networks trained on large, label-rich datasets (such as the Waymo Open Dataset and nuScenes) and evaluate on smaller, label-poor datasets (such as KITTI) and vice-versa. We demonstrate our approach on two recent object detectors and achieve results that out-perform the other domain adaptation works.
BibTeX
@inproceedings{hegde2024attentive,
  author = {Deepti Hegde and Vishal M. Patel},
  title = {{Attentive Prototypes for Source-Free Unsupervised Domain Adaptive 3D Object Detection}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure