Universal Domain Adaptive Object Detector

Wenxu Shi, Lei Zhang, Weijie Chen, Shiliang Pu

Introduction

Domain Adaptive Object Detection (DAOD) has traditionally operated under a "Closed-set" assumption, where the source domain (labeled) and the target domain (unlabeled) share an identical set of categories. However, real-world applications rarely satisfy this condition. In practical scenarios, the target domain often contains categories not present in the source (Open-set) or only a subset of the source categories (Partial-set). This discrepancy, known as Category Shift, often leads to "negative transfer," where the model's performance degrades as it attempts to align unrelated features.

Furthermore, existing DAOD methods frequently overlook Scale Shift. Objects of the same category can appear at vastly different scales due to perspective and camera distance. Aligning a small, distant object in the source domain with a large, close-up object in the target domain results in poor feature representation. To address these dual challenges, this paper introduces UniDAOD (Universal Domain Adaptive Object Detection) and a novel framework: the Universal Scale-aware Domain Adaptation Framework (US-DAF).

The US-DAF Framework

Built upon the Faster R-CNN architecture, US-DAF introduces two primary innovations designed to handle category and scale discrepancies simultaneously.

1. Filter Mechanism (FM)

To mitigate negative transfer, the Filter Mechanism acts as a gatekeeper for adversarial alignment. Instead of attempting to align all features, the framework utilizes the domain discriminator’s confidence as a similarity quantifier.

If the discriminator is highly confident that a sample belongs to either the source or the target domain, the sample is likely a "private" class (unique to one domain) and is filtered out. Conversely, samples that fall near the "uncertainty boundary" (a probability near 0.5) are identified as "common" classes. Only these common samples are used for adversarial alignment, ensuring the model focuses on shared semantic information.

2. Multi-Label Scale-Aware Adapter (SAA)

Standard DAOD methods typically use a binary domain classifier (Source vs. Target). US-DAF replaces this with a Multi-Label Scale-Aware Adapter. This adapter utilizes a four-element label vector: [Domain, Small, Medium, Large]. For instance, a small object in the source domain is encoded as [0, 1, 0, 0], while a large object in the target domain might be [1, 0, 0, 1].

By incorporating scale as a structured label rather than noise, the network is forced to learn domain-invariant features that are specifically aligned by their physical scale. This prevents the misalignment of features across different sizes, leading to more robust detection across varying perspectives.

Experimental Evaluation

The researchers evaluated US-DAF across several cross-domain benchmarks to test its versatility in Open-set, Partial-set, and Closed-set scenarios:

  • Open-set: Pascal VOC → Clipart1k and Watercolor → Pascal VOC. These tests involved varying degrees of category overlap, measured by the Jaccard index (ξ).
  • Partial-set: Pascal VOC → Watercolor, where the target domain categories are a subset of the source.
  • Closed-set: Cityscapes → Foggy Cityscapes, a standard benchmark used to ensure the model maintains high performance when labels match perfectly.

Key Results and Insights

US-DAF achieved state-of-the-art results across all tested scenarios, outperforming established baselines such as DAF, MAF, and HTCN.

  • Performance Gains: On the Clipart1k and Watercolor datasets, the model demonstrated relative improvements in mean Average Precision (mAP) of 7.1% and 5.9%, respectively, over the standard DAF baseline.
  • Robustness to Category Shift: In the Pascal VOC → Clipart1k task (ξ=0.5), US-DAF reached 40.0% mAP, a significant jump from the source-only Faster R-CNN (30.6%) and the standard DAF (33.2%).
  • Scale-Wise Precision: Quantitative analysis confirmed that the SAA improved mAP across all object sizes (small, medium, and large), proving that explicit scale alignment is superior to uniform alignment methods.

Conclusion

The US-DAF framework demonstrates that effective domain adaptation in object detection requires a selective and structured approach. By treating the domain discriminator as a similarity quantifier and elevating scale to a primary feature label, US-DAF successfully bridges the gap between disparate domains. While the current mechanism requires some parameter tuning for the filter threshold, it represents a significant step toward "in-the-wild" object detection where category and scale shifts are the norm rather than the exception.

Summary updated

BibTeX
@inproceedings{shi2022universal,
  author = {Wenxu Shi and Lei Zhang and Weijie Chen and Shiliang Pu},
  title = {{Universal Domain Adaptive Object Detector}},
  booktitle = {ACM Multimedia},
  year = {2022},
  url = {https://openreview.net/forum?id=2grPGSvnUoY},
}
Figure