Cross-Domain Object Detection by Dual Adaptive Branch

Xinyi Liu, Baofeng Zhang, Na LiuSensors 2023 · DOI

TL;DR

This paper tackles the challenge of cross-domain object detection, where models trained on one dataset struggle with data from a different domain. They introduce a "Dual Adaptive Branch" (DAB) framework within a teacher-student setup that simultaneously aligns domain-invariant features and suppresses domain-specific ones, leading to significant performance improvements.

What Problem Does This Paper Address?

Object detection models typically assume that training and testing data come from the same distribution. However, in real-world scenarios, this assumption often breaks down due to variations in lighting, appearance, background, and other factors. This "domain shift" can drastically degrade model performance. Cross-domain object detection aims to bridge this gap by enabling models to generalize to new, unseen domains with minimal or no labeled data. While existing methods have made progress, they often struggle to effectively disentangle and leverage domain-invariant features while mitigating the influence of domain-specific ones, leading to persistent errors and false positives.

Methods

The proposed approach, called the Dual Adaptive Branch (DAB), is built upon a teacher-student framework. The teacher model, trained on target domain data, generates pseudo-labels to guide the student model. The core novelty lies within the student model's architecture, which features two parallel branches:

  • Domain-Invariant Feature Alignment Branch: This branch aims to align features that are common across both source and target domains. It maps features to a high-dimensional space and uses a distribution difference metric function to minimize discrepancies. This is crucial for learning generalizable object representations.
  • Domain-Specific Feature Suppression Branch: This branch is designed to actively suppress features that are unique to a specific domain (e.g., background clutter or stylistic differences). By constraining these domain-specific features, the model can better focus on the essential object characteristics.

The paper also introduces a domain adaptive kernel function and a distribution difference measure function to facilitate this alignment and suppression process.

Datasets

The experiments were conducted on several standard cross-domain object detection benchmarks:

  • Pascal VOC: A real-world dataset commonly used for detection and segmentation.
  • Clipart: An artistic image dataset.
  • Watercolor: Contains watercolor-style images.
  • DT Clipart: Created by applying CycleGAN to Pascal VOC to mimic the Clipart style, ensuring identical annotations.
  • Cityscapes: A semantic segmentation dataset of urban scenes.
  • Foggy Cityscapes: A variation of Cityscapes with synthetic fog added.

The paper also evaluates domain generalization on an unknown domain (Clipart) using Pascal VOC and Watercolor as training data.

Key Insights and Ideas

  • Disentangling Features is Key: The most significant insight is that simply aligning all features is insufficient. Effectively separating and handling domain-invariant versus domain-specific features is critical for robust cross-domain object detection.
  • Suppression is as Important as Alignment: The feature suppression branch proved to be as vital as the alignment branch, demonstrating that actively reducing the influence of domain-specific noise can significantly boost performance.
  • Teacher-Student Synergy: The teacher-student framework, combined with pseudo-labeling, provides a strong foundation for transferring knowledge from the source to the target domain.
  • Error Reduction: The method effectively reduces common errors like missed detections (Miss), classification errors (Cls), and localization errors (Loc), as evidenced by their error analysis.

Key Results

The proposed Dual Adaptive Branch (DAB) method demonstrated strong performance across various cross-domain scenarios:

  • Pascal VOC → Clipart: Achieved 48.4% mAP, outperforming previous state-of-the-art methods by a significant margin. The individual branches also showed competitive results, with the feature suppression branch alone achieving 45.9% mAP.
  • Pascal VOC → DT Clipart: Achieved 54.7% mAP, again outperforming other methods.
  • Cityscapes → Foggy Cityscapes: Achieved 47.4% mAP, a substantial improvement over existing methods, particularly in adverse weather conditions.
  • Domain Generalization: Showed competitive performance on an unknown target domain (Clipart) when trained on Pascal VOC and Watercolor.
  • Qualitative Results: Visualizations showed a marked improvement in detection accuracy and localization compared to baseline methods, with fewer false positives and missed objects.

Limitations and Future Work

  • Category-Specific Performance: While overall performance is strong, the paper notes that the method doesn't perform best in all categories (e.g., 'chair', 'cow', 'mbike' in one experiment). However, the differences are small, and the proposed method still outperforms others in these categories.
  • Scale Variation: The paper investigates scale variation and shows that DAB is robust, but the feature alignment branch is noted to be more robust to scale shifts than the suppression branch.
  • Trade-off Parameters: The study includes an ablation on trade-off parameters (λ and μ), suggesting that optimal performance is achieved with specific values (e.g., 0.099 for both). Fine-tuning these parameters is important.
  • Future Work: The paper implicitly suggests that further exploration of feature disentanglement and more sophisticated domain adaptation strategies could lead to even greater improvements. The robustness across different scales and adverse weather conditions also opens avenues for more challenging real-world applications.

Summary updated

Abstract

The object detection task usually assumes that the training and test samples obey the same distribution, and this assumption is not valid in reality, therefore the study of cross-domain object detection is proposed. Compared with image classification, the cross-domain object detection task presents the greater challenge, which requires both accurate classification and localization of samples in the target domain. The teacher-student framework (the student model is supervised by pseudo-labels from the teacher model) has produced a large accuracy improvement in cross-domain object detection. Feature-level adversarial training is used in the student model, which allows features in the source and target domains to share a similar distribution. However, the direction and gradient of the weights can be divided into domain-specific and domain-invariant features, and the purpose of domain adaptive is to focus on the domain-invariant features while eliminating interference from the domain-specific features. Inspired by this, we propose a teacher-student framework named dual adaptive branch (DAB), which uses domain adversarial learning to address the domain distribution. Specifically, we ensure that the student model aligns domain-invariant features and suppresses domain-specific features in this process. We further validate our method based on multiple domains. The experimental results demonstrate that our proposed method significantly improves the performance of cross-domain object detection and achieves the competitive experimental results on common benchmarks.
BibTeX
@inproceedings{liu2023cross,
  author = {Xinyi Liu and Baofeng Zhang and Na Liu},
  title = {{Cross-Domain Object Detection by Dual Adaptive Branch}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23031199},
}
Figure