TL;DR
This paper introduces the first framework specifically designed for Domain Adaptive Object Detection (DAOD) under noisy source annotations. The proposed Noise Latent Transferability Exploration (NLTE) framework effectively recaptures missing annotations and leverages class-corrupted samples to achieve robust performance, improving mAP by up to 8.4% and nearly matching the performance of models trained on perfectly clean data.
What Problem Does This Paper Address?
Domain Adaptive Object Detection (DAOD) is the art of training a detector on a labeled "source" domain (like clear sunny photos) so that it performs well on an unlabeled "target" domain (like foggy or artistic images). Standard DAOD methods rest on a shaky assumption: that source labels are perfectly clean.
In reality, large-scale datasets are riddled with errors. As shown in Figure 1, even benchmark datasets like Cityscapes contain miss-annotated instances (objects completely missed by humans) and class-corrupted instances (objects labeled as the wrong category).
When these noisy labels are used for domain adaptation, they bias the source distribution, leading the model to learn incorrect transformations. While "learning with noisy labels" is a hot topic in image classification, those methods often fail in detection because they ignore the domain shift or struggle with the extreme foreground-background imbalance inherent in object detection.
Methods
The authors propose the NLTE framework, which consists of three main pillars designed to turn "noise" into "knowledge":
- Potential Instance Mining (PIM): Since the Region Proposal Network (RPN) is class-agnostic, it often finds objects even if a human missed them. PIM looks at high-confidence background proposals that don't overlap with existing ground truth and "recaptures" them as potential foreground instances to enrich source semantics.
- Morphable Graph Relation Module (MGRM): Instead of just trying to "fix" a wrong label, MGRM models how categories relate across domains. It uses graph-based feature aggregation to build a Global Relation Matrix (representing ideal class-wise transitions) and uses it to regularize a Local Relation Matrix derived from noisy samples. This allows the model to extract useful domain-invariant features even from incorrectly labeled objects.
- Entropy-Aware Gradient Reconcilement (EAGR): This is the "peacekeeper" of the training process. It uses a first-order meta-learning update to ensure that the gradients produced by noisy samples are consistent with those from clean samples. By aligning these gradients, the model ensures that all data points contribute toward a unified, domain-invariant representation.
Datasets
The researchers evaluated NLTE across several challenging scenarios:
- Synthetic Noise: They took the clean Pascal VOC dataset and manually injected noise at various rates (20% to 80%) before adapting to Clipart1k and Watercolor2k.
- Real-world Noise: They used Cityscapes (which has inherent labeling errors) as the source and Foggy Cityscapes as the target.
Key Insights and Ideas
- Noise as a Resource: Unlike classification-based denoising that filters out noisy samples, NLTE recognizes that even a mislabeled source image contains valuable domain-specific information (like lighting or texture) that is crucial for domain alignment.
- RPN as a Denoising Tool: The realization that a class-agnostic RPN can "out-detect" a noisy human annotator is a clever way to handle missing labels without needing a separate pre-trained model.
- Gradient Harmony: By framing the problem as a gradient consistency task via meta-learning, the authors avoid the need to explicitly separate "clean" from "noisy" samples during every training step, which is computationally expensive and error-prone.
Key Results
The results demonstrate that NLTE is remarkably robust. In the Pascal VOC to Clipart1k task, NLTE significantly outperformed standard baselines (DAF) and general noise-robust methods (SCE, GCE).
- Significant Gains: At a 60% noise rate, NLTE improved the baseline mAP by 8.4%.
- Approaching the Upper Bound: Remarkably, at 60% noise, NLTE's performance was only 2% lower than a model trained on a 100% clean source dataset.
- Real-world Success: On the Cityscapes to Foggy Cityscapes benchmark, NLTE achieved an mAP of 45.4%, outperforming recent state-of-the-art DAOD methods like SSAL.
Limitations and Future Work
While the paper presents a comprehensive solution, a few areas remain for exploration:
- Computational Complexity: The use of graph modules and meta-learning updates likely increases training time and memory requirements compared to standard detectors.
- RPN Dependency: The effectiveness of PIM is tied to the quality of the RPN. In extremely difficult domains where the RPN fails to generate high-confidence proposals, the benefits of PIM might diminish.
- Future Directions: The authors suggest that addressing noisy annotations is a highly effective alternative to standard adaptation techniques, hinting that future DAOD research should perhaps focus more on data quality and label robustness.
Summary updated