Reflective Teacher: Semi-Supervised Multimodal 3D Object Detection in Bird's-Eye-View via Uncertainty Measure

Saheli Hazra, Sudip Das, Rohit Choudhary, Arindam Das, Ganesh Sistu, Ciarán Eising, Ujjwal BhattacharyaWACV 2025

TL;DR

This paper introduces "Reflective Teacher," a novel semi-supervised approach for 3D object detection in autonomous driving. It tackles the challenge of catastrophic forgetting in teacher-student learning by using a reflective mechanism and improves pseudo-label quality with an uncertainty measure, achieving state-of-the-art results with significantly less labeled data.

What Problem Does This Paper Address?

Autonomous driving systems heavily rely on accurate 3D object detection to understand their surroundings for tasks like path planning. While Bird's-Eye-View (BEV) representations are effective for fusing sensor data (like LiDAR and cameras), creating large, annotated datasets for training these models is prohibitively expensive. Semi-supervised learning (SSL) offers a solution by leveraging abundant unlabeled data. However, existing SSL methods for 3D object detection face two major hurdles:

  1. Catastrophic Forgetting: In teacher-student SSL, the teacher model's knowledge is updated by the student. Techniques like Exponential Moving Average (EMA) can lead to the teacher forgetting previously learned information, resulting in inaccurate pseudo-labels.
  2. Pseudo-Label Quality: The accuracy of pseudo-labels generated by the teacher is crucial. Noisy or incorrect pseudo-labels can severely degrade the student's performance.

This paper aims to address these limitations by proposing a more robust and efficient semi-supervised learning framework for multimodal 3D object detection in BEV.

Methods

The proposed approach, "Reflective Teacher," builds upon a teacher-student framework and introduces several key innovations:

  • Reflective Teacher Learning: To combat catastrophic forgetting, the authors introduce a "reflective" mechanism. Instead of simply updating the teacher's weights with the student's, a regularizer is used to preserve critical parameters and previously acquired knowledge. This ensures that the teacher retains its expertise while learning from the student.
  • Uncertainty Measure for Pseudo-Labels: To improve the quality of pseudo-labels, the paper incorporates an uncertainty measure. This metric evaluates the confidence of region proposals generated by the student's BEV Region Proposal Network (RPN). By considering both the Intersection over Union (IoU) with pseudo-labels and the classification probability score, the model can identify and down-weight uncertain regions during training. This helps the teacher generate more reliable pseudo-labels.
  • Geometry Aware BEV Fusion (GA-BEVFusion): To efficiently fuse features from different modalities (LiDAR and camera), the authors propose GA-BEVFusion. This module addresses feature misalignment by using a bi-linear warping strategy and deformable convolutions. It aligns BEV features from different sensors and then enhances them, leading to more robust and accurate representations.
  • Overall Architecture: The system integrates multi-view camera images and LiDAR data. A perspective 3D head is used for camera features, while LiDAR features are projected into the BEV space. The GA-BEVFusion module then combines these features. The model is trained with a combination of supervised and unsupervised losses, including a perspective loss and an alignment loss.

Datasets

The proposed method is evaluated on two widely used autonomous driving datasets:

  • nuScenes: A comprehensive multimodal dataset for autonomous driving.
  • Waymo Open Dataset (WOD): Another large-scale dataset for autonomous driving research.

The paper demonstrates performance on both fully supervised and semi-supervised settings, highlighting the effectiveness of their approach with varying percentages of labeled data.

Key Insights and Ideas

  • Catastrophic forgetting is a significant bottleneck in teacher-student SSL for 3D detection. The proposed "reflective" mechanism offers a principled way to mitigate this by actively preserving past knowledge.
  • Uncertainty estimation is key to improving pseudo-label quality. By quantifying the confidence of region proposals, the model can focus on more reliable information, leading to better teacher guidance for the student.
  • Geometric misalignment between sensor modalities is a persistent challenge in BEV fusion. The GA-BEVFusion module provides an effective solution by explicitly aligning features, leading to improved performance.
  • The proposed semi-supervised approach can achieve performance comparable to fully supervised methods with a fraction of the labeled data. This has significant implications for reducing data annotation costs in real-world applications.

Key Results

The Reflective Teacher model achieves state-of-the-art results on both nuScenes and Waymo datasets.

  • nuScenes: The model achieves 79.3% mAP and 80.4% NDS on the test set in a fully supervised setting. Crucially, in a semi-supervised setting, it achieves equivalent performance with only 25% of the labeled data.
  • Waymo: Similarly, on the Waymo dataset, the model achieves comparable results with only 22% of the labeled data.
  • Qualitative results demonstrate the model's ability to detect distant, small, and occluded objects, even in challenging nighttime conditions with bright light occlusion.
  • Ablation studies confirm the effectiveness of GA-BEVFusion, perspective supervision, temporal inputs, and the uncertainty measure. The uncertainty measure, in particular, provides a notable improvement in mAP when trained with limited labeled data.

Limitations and Future Work

  • Limitations: The paper acknowledges that while their method significantly reduces the need for labeled data, it doesn't entirely eliminate it. The performance still benefits from some level of supervised data. The paper also mentions that their ablation studies for comparison primarily focused on Unbiased Teacher and Active Teacher due to code availability.
  • Future Work: The authors suggest that their framework can be extended to other multimodal 3D object detection architectures. They also highlight the potential for further improvements by exploring more advanced uncertainty estimation techniques and more sophisticated fusion strategies. The effectiveness of their approach in fully supervised settings also opens avenues for further research in that domain.

Summary updated

BibTeX
@inproceedings{hazra2025reflective,
  author = {Saheli Hazra and Sudip Das and Rohit Choudhary and Arindam Das and Ganesh Sistu and Ciarán Eising and Ujjwal Bhattacharya},
  title = {{Reflective Teacher: Semi-Supervised Multimodal 3D Object Detection in Bird's-Eye-View via Uncertainty Measure}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
}
Figure