Breaking the SSL-AL Barrier: A Synergistic Semi-Supervised Active Learning Framework for 3D Object Detection

Zengran Wang, Yanan Zhang, Jiaxin Chen, Di Huang

Abstract

To address the annotation burden in LiDAR-based 3D object detection, active learning (AL) methods offer a promising solution. However, traditional active learning approaches solely rely on a small amount of labeled data to train an initial model for data selection, overlooking the potential of leveraging the abundance of unlabeled data. Recently, attempts to integrate semi-supervised learning (SSL) into AL with the goal of leveraging unlabeled data have faced challenges in effectively resolving the conflict between the two paradigms, resulting in less satisfactory performance. To tackle this conflict, we propose a Synergistic Semi-Supervised Active Learning framework, dubbed as S-SSAL. Specifically, from the perspective of SSL, we propose a Collaborative PseudoScene Pre-training (CPSP) method that effectively learns from unlabeled data without introducing adverse effects. From the perspective of AL, we design a Collaborative Active Learning (CAL) method, which complements the uncertainty and diversity methods by model cascading. This allows us to fully exploit the potential of the CPSP pre-trained model. Extensive experiments conducted on KITTI and Waymo demonstrate the effectiveness of our S-SSAL framework. Notably, on the KITTI dataset, utilizing only 2% labeled data, S-SSAL can achieve performance comparable to models trained on the full dataset.

Introduction

LiDAR-based 3D object detection is a cornerstone of autonomous vehicle perception systems. However, these models are notoriously "data-hungry," requiring millions of hand-labeled 3D bounding boxes to achieve high accuracy. Annotating point clouds is significantly more labor-intensive than 2D image annotation; for instance, the Waymo Open Dataset contains over 10 million ground-truth boxes, representing a massive human effort.

To mitigate these costs, researchers typically employ two strategies: Active Learning (AL), which identifies the most informative samples for human labeling, and Semi-Supervised Learning (SSL), which leverages unlabeled data through pseudo-labeling. Despite their individual successes, combining them is difficult. SSL thrives on high-confidence "easy" samples to maintain stability, whereas AL targets high-uncertainty "hard" samples. Naively merging them often results in noisy SSL pseudo-labels corrupting the AL selection process. S-SSAL is designed to resolve this conflict by creating a synergistic workflow that decouples confidence from uncertainty.

The S-SSAL Framework

The S-SSAL framework operates through a three-stage pipeline: Temporary Model Updating (TMU), Unlabeled Sample Selecting (USS), and Final Model Delivering (FMD).

The S-SSAL framework overview, illustrating the transition from Temporary Model Updating (TMU) to Unlabeled Sample Selecting (USS) and finally to Final Model Delivering (FMD).

Collaborative PseudoScene Pre-training (CPSP)

In the TMU stage, the framework employs CPSP to leverage unlabeled data without introducing noise. Traditional SSL can be unstable because it trains on entire scenes that may contain many low-confidence, noisy objects. CPSP solves this by extracting only high-confidence objects from unlabeled data and storing them in a Certain Box Bank.

Detailed workflow of Collaborative PseudoScene Pre-training (CPSP), showing how confident objects are extracted and "pasted" into background scenes to create clean training data.

As shown in the detailed workflow, the system generates "background scenes" by removing potential objects from LiDAR frames. It then populates these backgrounds with boxes from the bank to create "PseudoScenes." This allows the model to pre-train on clean, high-confidence data, resulting in a much better-calibrated model for the subsequent active learning phase.

Collaborative Active Learning (CAL)

During the USS stage, the framework selects the next batch of data for human annotation using three sub-strategies:

  1. Ensemble-based Entropy ($E^2\_Unc$): By combining predictions from a standard pre-trained model and the CPSP model, the system gains a more reliable measure of uncertainty.
  2. Box-level Diversity ($B\_Div$): This ensures the selected samples are not redundant by measuring the cosine similarity of box features against the existing labeled pool.
  3. Class Balance Sampling (CBS): To prevent the model from becoming biased toward common objects like cars, CBS dynamically adjusts weights to prioritize rare but safety-critical classes like pedestrians and cyclists.

Final Model Delivering (FMD)

In the final stage, the framework utilizes standard SSL techniques, such as Hybrid Semi-Supervised Data Augmentation (HSSDA), to maximize performance using the now-optimized pool of labeled and unlabeled data.

Experimental Results

The framework was evaluated on the KITTI and Waymo Open Dataset benchmarks, demonstrating remarkable data efficiency.

  • KITTI Efficiency: Using only 350 labeled boxes (approximately 2% of the training set), S-SSAL achieved a moderate mAP of 70.5. This is strikingly close to the 72.2 mAP achieved by models trained on 100% of the labeled data.
  • Waymo Performance: On the large-scale Waymo dataset, the CPSP pre-training alone provided a 1.2% mAP boost over standard pre-training methods. The framework also showed significant improvements in heading-weighted AP (mAPH), indicating a better understanding of object orientation.
  • Calibration and Rare Classes: Ablation studies confirmed that CPSP produces a better-calibrated model, meaning its confidence scores accurately reflect its true precision. Furthermore, the CBS strategy proved vital for maintaining high detection rates for vulnerable road users.

Conclusion

S-SSAL successfully bridges the gap between Semi-Supervised and Active Learning for 3D object detection. By utilizing PseudoScene pre-training to denoise the SSL process and an ensemble-based strategy to stabilize AL selection, the framework achieves state-of-the-art performance with a fraction of the labeling effort. Future work will explore integrating more complex SSL architectures to further close the performance gap with fully supervised models.

Summary updated

Abstract

To address the annotation burden in LiDAR-based 3D object detection, active learning (AL) methods offer a promising solution. However, traditional active learning approaches solely rely on a small amount of labeled data to train an initial model for data selection, overlooking the potential of leveraging the abundance of unlabeled data. Recently, attempts to integrate semi-supervised learning (SSL) into AL with the goal of leveraging unlabeled data have faced challenges in effectively resolving the conflict between the two paradigms, resulting in less satisfactory performance. To tackle this conflict, we propose a Synergistic Semi-Supervised Active Learning framework, dubbed as S-SSAL. Specifically, from the perspective of SSL, we propose a Collaborative PseudoScene Pre-training (CPSP) method that effectively learns from unlabeled data without introducing adverse effects. From the perspective of AL, we design a Collaborative Active Learning (CAL) method, which complements the uncertainty and diversity methods by model cascading. This allows us to fully exploit the potential of the CPSP pre-trained model. Extensive experiments conducted on KITTI and Waymo demonstrate the effectiveness of our S-SSAL framework. Notably, on the KITTI dataset, utilizing only 2% labeled data, S-SSAL can achieve performance comparable to models trained on the full dataset.
BibTeX
@misc{wang2025breaking,
  author = {Zengran Wang and Yanan Zhang and Jiaxin Chen and Di Huang},
  title = {{Breaking the SSL-AL Barrier: A Synergistic Semi-Supervised Active Learning Framework for 3D Object Detection}},
  year = {2025},
  eprint = {2501.15449},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.15449},
}
Figure