Introduction
Domain Adaptive Object Detection (DAOD) is a critical task in computer vision, aimed at transferring knowledge from a labeled source domain to an unlabeled target domain. While standard DAOD methods assume that source domain annotations are perfectly clean, real-world datasets often contain significant noise. This noise typically manifests in two ways: miss-annotated instances (objects treated as background) and class-corrupted instances (objects assigned incorrect labels).
When models are trained on such "dirty" data, they learn biased distributions that cause domain alignment processes to fail. This paper introduces NLTE (Noise Latent Transferability Exploration), the first comprehensive framework designed to address noisy source annotations in DAOD by mining missing objects, modeling label transitions, and reconciling gradients via meta-learning.
The Challenge of Noisy Source Labels
In traditional object detection, a missing label is often more damaging than a wrong label because the model actively learns to suppress those features as "background." In the context of domain adaptation, simply discarding noisy samples—a common strategy in classification—is counterproductive. Even a mislabeled object contains valuable domain-specific information (e.g., lighting, texture, and weather conditions) that is essential for aligning the source and target feature spaces.
The NLTE Framework
To leverage noisy data effectively while mitigating its negative impact on detection accuracy, the NLTE framework employs three integrated components:
1. Potential Instance Mining (PIM)
Since the Region Proposal Network (RPN) is class-agnostic, it frequently identifies objects that human annotators may have overlooked. PIM identifies high-confidence proposals that do not overlap with existing ground truth boxes. By "recapturing" these instances, the framework prevents the model from incorrectly learning that these foreground objects belong to the background category.
2. Morphable Graph Relation Module (MGRM)
The MGRM utilizes graph neural networks to model the complex relationships between different classes across domains. It constructs a "global relation matrix" to quantify the probability of one class being mistaken for another. This matrix serves as a regularizer for the features of noisy samples, ensuring that their transition probabilities remain consistent with the broader data structure and preventing the model from overfitting to specific annotation errors.
3. Entropy-Aware Gradient Reconcilement (EAGR)
EAGR is an optimization strategy designed to ensure that noisy samples do not pull the model's parameters in the wrong direction. It utilizes an entropy-aware discriminator to incorporate class confidence into the domain alignment process. By applying a first-order meta-learning update, EAGR ensures that gradients produced by noisy samples are reconciled—or aligned—with gradients from clean samples, fostering the learning of robust, domain-invariant features.
Experimental Evaluation
The researchers evaluated NLTE across several challenging benchmarks, using both synthetic and natural noise:
- Synthetic Noise (Pascal VOC → Clipart1k / Watercolor2k): Source labels were injected with varying levels of corruption (0% to 80%). NLTE demonstrated remarkable robustness; for instance, on the Clipart1k task with 60% noise, NLTE improved the Mean Average Precision (mAP) by 8.4% over the baseline.
- Natural Noise (Cityscapes → Foggy Cityscapes): This benchmark tested the framework against real-world annotation errors inherent in the Cityscapes dataset. NLTE achieved 45.4% mAP, outperforming state-of-the-art DAOD methods originally designed for clean data.
Key Insights
- Noisy Samples are Informative: Unlike classification-only methods, NLTE demonstrates that noisy samples are useful for domain alignment if their class-specific errors are properly managed.
- Background Recovery is Essential: In object detection, the "background trap" (missing labels) is a primary source of performance degradation. Mining these instances is critical for recovery.
- Meta-Learning for Robustness: Treating the consistency between clean and noisy gradients as a meta-objective allows the model to learn robust features without requiring explicit knowledge of which labels are incorrect.
Conclusion and Future Work
NLTE provides a robust solution for DAOD in the presence of imperfect source data. While the framework introduces some complexity through its use of graph modules and meta-learning, it significantly narrows the gap between models trained on noisy data and those trained on ideal, clean datasets. Future research may explore instance-dependent noise—where errors are tied to image difficulty rather than random class swaps—and further optimize the hyperparameter sensitivity of the multi-component loss function.
Summary updated