Dual-Curriculum Teacher for Domain-Inconsistent Object Detection in Autonomous Driving

Longhui Yu, Yifan Zhang, Lanqing Hong, Fei Chen, Zhenguo Li

Abstract

Object detection for autonomous vehicles has advanced rapidly, yet the high cost of labeled data remains a significant bottleneck. Semi-supervised object detection (SSOD) offers a solution by leveraging large volumes of unlabeled data. However, standard SSOD methods typically assume that labeled and unlabeled data share the same distribution (the IID assumption). In real-world autonomous driving, this assumption is frequently violated as data is collected across diverse weather conditions, times of day, and geographic locations. This paper introduces Dual-Curriculum Teacher (DucTeacher), a framework designed to handle domain-inconsistent SSOD. By employing a domain evolving curriculum and a distribution matching curriculum, DucTeacher effectively manages data distribution discrepancies and class distribution shifts, achieving state-of-the-art performance on the SODA10M and COCO benchmarks.

Introduction

In the controlled environments of research benchmarks, SSOD models excel by using a "Teacher" model to generate pseudo-labels for a "Student" model. However, the transition to autonomous driving introduces "domain inconsistency." For instance, a model trained on sunny, urban data may struggle to interpret unlabeled data from a rainy night.

This inconsistency manifests in two primary ways:

  1. Data Distribution Shift: Variations in lighting, weather, and sensor noise that alter the pixel-level appearance of objects.
  2. Class Distribution Shift: Variations in the frequency of object classes across different environments (e.g., more trucks on highways versus more pedestrians in city centers).

When standard SSOD models encounter these shifts, they generate noisy, inaccurate pseudo-labels that degrade the performance of the student model. DucTeacher addresses these challenges by progressively adapting to new domains and dynamically adjusting detection thresholds.

The DucTeacher Framework

DucTeacher utilizes a dual-curriculum approach to filter noise and calibrate the learning process for unlabeled data.

1. Domain Evolving Curriculum (DEC)

To address data distribution shifts, the DEC adopts a "start simple" strategy. Rather than training on all unlabeled data simultaneously, the model prioritizes domains that are most similar to the labeled source data.

  • Similarity Estimation: The framework calculates a Domain Similarity score based on the average confidence of the Teacher model's bounding box predictions. High confidence suggests the domain is similar to the training set.
  • Progressive Learning: The model begins with "easy" (high-similarity) domains. As the model’s feature representation matures, it gradually incorporates "harder" (low-similarity) domains, such as night or rainy scenes. This prevents the model from being overwhelmed by low-quality pseudo-labels in the early stages of training.

2. Distribution Matching Curriculum (DMC)

Standard SSOD methods often use a fixed confidence threshold (e.g., 0.7) to filter pseudo-labels. This is problematic when class distributions shift; a rare class in a difficult domain might never meet a high fixed threshold, leading to significant false negatives.

  • Dynamic Thresholding: The DMC estimates the specific class distribution for each unlabeled domain. It then dynamically adjusts the confidence threshold for each class and domain. If the model expects a certain frequency of a class (like "cyclists") but is under-detecting them, it lowers the threshold to capture more potential instances, thereby balancing the learning signal.

Experimental Evaluation

Dataset: SODA10M

The researchers evaluated DucTeacher on SODA10M, the largest public semi-supervised autonomous driving dataset. The setup included 5,000 labeled images from a single domain (Daytime, City, Clear) and 10 million unlabeled images spanning 48 different domains.

Performance Results

DucTeacher demonstrated significant improvements over existing baselines:

  • Overall Accuracy: On SODA10M, DucTeacher achieved 48.4 mAP, representing a +10.5 improvement over the supervised-only baseline and a +2.2 improvement over the previous state-of-the-art (Unbiased Teacher).
  • Robustness in Hard Scenarios: The most notable gains occurred in the "Night" domain. While the supervised baseline achieved only 21.1 mAP, DucTeacher reached 40.7 mAP, nearly doubling the performance in low-light conditions.
  • Generalization: On the MS-COCO benchmark, DucTeacher provided a consistent boost of approximately 0.8 mAP across various labeling ratios, proving its utility even in datasets with less extreme domain shifts.

Conclusion and Future Work

DucTeacher provides a robust framework for semi-supervised learning in the presence of real-world domain shifts. By using confidence as a proxy for domain similarity and implementing adaptive thresholding, it effectively bridges the gap between controlled training sets and diverse deployment environments.

Future research directions include reducing the computational overhead of the Teacher-Student EMA framework and extending the dual-curriculum approach to 3D LiDAR-based detection. Additionally, developing methods to automatically cluster domains without relying on metadata labels would further enhance the framework's versatility in the field.

Summary updated

Abstract

Object detection for autonomous vehicles has received increasing attention in recent years, where labeled data are often expensive while unlabeled data can be collected readily, calling for research on semi-supervised learning for this area. Existing semi-supervised object detection (SSOD) methods usually assume that the labeled and unlabeled data come from the same data distribution. In autonomous driving, however, data are usually collected from different scenarios, such as different weather conditions or different times in a day. Motivated by this, we study a novel but challenging domain inconsistent SSOD problem. It involves two kinds of distribution shifts among different domains, including (1) data distribution discrepancy, and (2) class distribution shifts, making existing SSOD methods suffer from inaccurate pseudo-labels and hurting model performance. To address this problem, we propose a novel method, namely Dual-Curriculum Teacher (DucTeacher). Specifically, DucTeacher consists of two curriculums, i.e., (1) domain evolving curriculum seeks to learn from the data progressively to handle data distribution discrepancy by estimating the similarity between domains, and (2) distribution matching curriculum seeks to estimate the class distribution for each unlabeled domain to handle class distribution shifts. In this way, DucTeacher can calibrate biased pseudo-labels and handle the domain-inconsistent SSOD problem effectively. DucTeacher shows its advantages on SODA10M, the largest public semi-supervised autonomous driving dataset, and COCO, a widely used SSOD benchmark. Experiments show that DucTeacher achieves new state-of-the-art performance on SODA10M with 2.2 mAP improvement and on COCO with 0.8 mAP improvement.
BibTeX
@inproceedings{yu2022dual,
  author = {Longhui Yu and Yifan Zhang and Lanqing Hong and Fei Chen and Zhenguo Li},
  title = {{Dual-Curriculum Teacher for Domain-Inconsistent Object Detection in Autonomous Driving}},
  booktitle = {BMVC},
  year = {2022},
  url = {https://openreview.net/forum?id=_6sYR6vGUtm},
}
Figure