Domain Adaptive Object Detection via Dual-Stream Bilevel-Cycle Optimization

Yannan Chen, Wei Wang, Wenqiang Wang, Ruoyu Chen, Jiancheng Wang, Mingbo Yang, Yaowei Wang, Xiaochun CaoarXiv 2026 · arXiv

TL;DR

The Dual-Stream Bilevel-Cycle Optimization (DSBCO) framework successfully extends Cycle Self-Training to unsupervised domain adaptive object detection. By decoupling classification and regression into separate cycle streams and introducing a critical regression normalization mechanism, DSBCO overcomes the training instability and exploding losses that previously stalled cycle-based adaptation in object detection.

What Problem Does This Paper Address?

Imagine training an autonomous driving object detector in sunny Germany (using the Cityscapes dataset) and deploying it in a heavy fog or a completely different city layout. The drastic shift in data distribution—known as a domain gap—often causes standard detectors to fail.

Unsupervised Domain Adaptive Object Detection (DAOD) attempts to fix this by using labeled source data alongside unlabeled target data. A popular approach is Self-Training (ST), where the model generates "pseudo-labels" for the target domain and learns from them. However, standard ST hits a performance ceiling because these pseudo-labels are notoriously unreliable, suffering from incorrect classifications, inaccurate bounding boxes, or both.

While Cycle Self-Training (CST) has successfully broken this performance barrier in image classification by using an inner-outer loop strategy to filter out bad labels, extending it to object detection introduces major roadblocks:

  1. Multiple Failure Modes: Pseudo-labels can be wrong in classification, localization, or both simultaneously.
  2. Training Instability: Dense object detectors (like FCOS) predict many objects per image, making direct CST application highly unstable.
  3. Exploding Losses: Bounding box coordinates have a wide, unbounded numerical range, which causes gradients to explode during cycle optimization.
Challenges of extending CST to object detection, including performance upper bounds, training instability, and exploding losses.

Methods

To solve these challenges, the authors propose DSBCO, a framework built on top of the stable Mean Teacher (MT) paradigm. Instead of a single optimization loop, DSBCO decouples the task into two distinct feature-level cycle streams:

  • Classification Cycle Stream: This stream aligns semantic category features across domains. Instead of using unstable matrix inversions common in traditional CST, it constructs a robust covariance matrix along the feature dimension to map target features back to the source domain.
  • Regression Cycle Stream with Normalization: To prevent exploding losses caused by unbounded bounding box coordinates, the authors introduce a regression normalization mechanism. It computes the mean and standard deviation of target domain regression offsets, projecting them into a standardized normal distribution space before solving the cycle optimization.

Additionally, the framework utilizes Instance Consistency Distillation as a baseline, filtering out background noise by using strict classification and localization (centerness) thresholds to ensure only high-quality pseudo-labels are used for training.

Overview of the proposed DSBCO framework featuring the Mean Teacher paradigm, Instance Consistency Distillation, and Dual-Stream Feature Bilevel-Cycle Optimization.

Datasets

The authors evaluated DSBCO across four standard cross-domain scenarios to test its robustness against weather, layout, camera, and synthetic-to-real shifts:

  • Adverse Weather Adaptation: Cityscapes $\rightarrow$ Foggy Cityscapes
  • Diverse Context Adaptation: Cityscapes $\rightarrow$ BDD100K (daytime driving subset)
  • Distinct Camera Adaptation: KITTI $\rightarrow$ Cityscapes
  • Synthetic Source Adaptation: Sim10K $\rightarrow$ Cityscapes

Key Insights and Ideas

  • Decoupling is Essential: Object detection cannot be treated as a unified classification problem for domain adaptation. Separating localization and classification into independent streams prevents errors in one task from corrupting the other.
  • Normalization Stabilizes Continuous Spaces: Cycle optimization inherently struggles with infinite output spaces. Projecting bounding box coordinates into a standard normal distribution is the "secret sauce" that prevents gradient explosion.
  • Bypassing the Hessian: Standard bilevel optimization is computationally heavy because it repeatedly calculates large Hessian matrices. DSBCO bypasses this by using a compact feature-based covariance matrix estimation and a linear solver, making it highly efficient.

Key Results

DSBCO achieved state-of-the-art results, outperforming all existing FCOS-based methods across the benchmarks:

  • Foggy Weather: Reached 64.7% mAP on Foggy Cityscapes, outperforming the second-best FCOS method (Harmonious Teacher) by a massive 14.3%.
  • Context & Camera Shifts: Achieved a 41.9% mAP on BDD100K and a leading 62.2% AP for cars on the KITTI $\rightarrow$ Cityscapes task.
  • Computational Efficiency: Despite running bilevel optimization, DSBCO maintains a training speed close to standard self-training (only ~1.2$\times$ the time cost), proving it is highly scalable.
Qualitative detection results comparing Standard Self-Training and DSBCO, demonstrating robust improvements against bounding box shifts and misclassifications.

Limitations and Future Work

While DSBCO excels with anchor-free convolutional detectors like FCOS, the authors note two primary limitations:

  1. Architecture Compatibility: The dual-stream optimization framework has not yet been adapted or tested on modern Vision Transformer (ViT) based detectors like DETR.
  2. Fixed Thresholding: The framework relies on fixed confidence thresholds to filter out background noise, which might struggle in exceptionally noisy or complex target environments. Future work will explore more adaptive thresholding strategies.

Summary updated

Abstract

Cycle self-training (CST) breaks the shared classifier assumption of the standard self-training framework, which is effective for unsupervised domain adaptation and exploits unlabeled target data by training with target pseudo-labels. CST introduces a target classifier and employs an inner-outer loop updating strategy, addressing the issue of unreliable pseudo-labels and enabling pseudo-labels to generalize across domains. Despite its success in image classification, extending CST to object detection faces three main challenges. First, the upper bound of CST in object detection is constrained by three types of unreliable pseudo-labels, such as classification error alone, localization error alone, and their combination. Second, since object detection involves detecting multiple target objects, directly applying CST leads to training insta bility. Third, a wider numerical range of regression coordinates leads to exploding losses. To this end, we apply CST to both classification and regression and propose the Dual-Stream Bilevel-Cycle Optimization framework. Specifically, we construct CST upon Mean Teacher to prevent training instability and use extra normalization to map the regression bounding box into a standardized space, effectively addressing exploding losses. Also, we provide a theoretical derivation of the regression bound. Extensive experiments across four cross domain standard scenarios demonstrate that our framework achieves considerable results.
BibTeX
@misc{chen2026domain,
  author = {Yannan Chen and Wei Wang and Wenqiang Wang and Ruoyu Chen and Jiancheng Wang and Mingbo Yang and Yaowei Wang and Xiaochun Cao},
  title = {{Domain Adaptive Object Detection via Dual-Stream Bilevel-Cycle Optimization}},
  year = {2026},
  eprint = {2606.31373},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.31373},
}
Figure