Towards Robust Adaptive Object Detection under Noisy Annotations

Xinyu Liu, Wuyang Li, Qiushi Yang, Baopu Li, Yixuan Yuan

Introduction

Domain Adaptive Object Detection (DAOD) is a critical task in computer vision, aimed at transferring knowledge from a labeled source domain to an unlabeled target domain. While standard DAOD methods assume that source domain annotations are perfectly clean, real-world datasets often contain significant noise. This noise typically manifests in two ways: miss-annotated instances (objects treated as background) and class-corrupted instances (objects assigned incorrect labels).

When models are trained on such "dirty" data, they learn biased distributions that cause domain alignment processes to fail. This paper introduces NLTE (Noise Latent Transferability Exploration), the first comprehensive framework designed to address noisy source annotations in DAOD by mining missing objects, modeling label transitions, and reconciling gradients via meta-learning.

The Challenge of Noisy Source Labels

In traditional object detection, a missing label is often more damaging than a wrong label because the model actively learns to suppress those features as "background." In the context of domain adaptation, simply discarding noisy samples—a common strategy in classification—is counterproductive. Even a mislabeled object contains valuable domain-specific information (e.g., lighting, texture, and weather conditions) that is essential for aligning the source and target feature spaces.

The NLTE Framework

To leverage noisy data effectively while mitigating its negative impact on detection accuracy, the NLTE framework employs three integrated components:

1. Potential Instance Mining (PIM)

Since the Region Proposal Network (RPN) is class-agnostic, it frequently identifies objects that human annotators may have overlooked. PIM identifies high-confidence proposals that do not overlap with existing ground truth boxes. By "recapturing" these instances, the framework prevents the model from incorrectly learning that these foreground objects belong to the background category.

2. Morphable Graph Relation Module (MGRM)

The MGRM utilizes graph neural networks to model the complex relationships between different classes across domains. It constructs a "global relation matrix" to quantify the probability of one class being mistaken for another. This matrix serves as a regularizer for the features of noisy samples, ensuring that their transition probabilities remain consistent with the broader data structure and preventing the model from overfitting to specific annotation errors.

3. Entropy-Aware Gradient Reconcilement (EAGR)

EAGR is an optimization strategy designed to ensure that noisy samples do not pull the model's parameters in the wrong direction. It utilizes an entropy-aware discriminator to incorporate class confidence into the domain alignment process. By applying a first-order meta-learning update, EAGR ensures that gradients produced by noisy samples are reconciled—or aligned—with gradients from clean samples, fostering the learning of robust, domain-invariant features.

Experimental Evaluation

The researchers evaluated NLTE across several challenging benchmarks, using both synthetic and natural noise:

  • Synthetic Noise (Pascal VOC → Clipart1k / Watercolor2k): Source labels were injected with varying levels of corruption (0% to 80%). NLTE demonstrated remarkable robustness; for instance, on the Clipart1k task with 60% noise, NLTE improved the Mean Average Precision (mAP) by 8.4% over the baseline.
  • Natural Noise (Cityscapes → Foggy Cityscapes): This benchmark tested the framework against real-world annotation errors inherent in the Cityscapes dataset. NLTE achieved 45.4% mAP, outperforming state-of-the-art DAOD methods originally designed for clean data.

Key Insights

  1. Noisy Samples are Informative: Unlike classification-only methods, NLTE demonstrates that noisy samples are useful for domain alignment if their class-specific errors are properly managed.
  2. Background Recovery is Essential: In object detection, the "background trap" (missing labels) is a primary source of performance degradation. Mining these instances is critical for recovery.
  3. Meta-Learning for Robustness: Treating the consistency between clean and noisy gradients as a meta-objective allows the model to learn robust features without requiring explicit knowledge of which labels are incorrect.

Conclusion and Future Work

NLTE provides a robust solution for DAOD in the presence of imperfect source data. While the framework introduces some complexity through its use of graph modules and meta-learning, it significantly narrows the gap between models trained on noisy data and those trained on ideal, clean datasets. Future research may explore instance-dependent noise—where errors are tied to image difficulty rather than random class swaps—and further optimize the hyperparameter sensitivity of the multi-component loss function.

Summary updated

Abstract

Domain Adaptive Object Detection (DAOD) models a joint distribution of images and labels from an annotated source domain and learns a domain-invariant transformation to estimate the target labels with the given target domain images. Existing methods assume that the source domain labels are completely clean, yet large-scale datasets often contain error-prone annotations due to instance ambiguity, which may lead to a biased source distribution and severely degrade the performance of the domain adaptive detector de facto. In this paper, we represent the first effort to formulate noisy DAOD and propose a Noise Latent Transferability Exploration (NLTE) framework to address this issue. It is featured with 1) Potential Instance Mining (PIM), which leverages eligible proposals to recapture the miss-annotated instances from the background; 2) Morphable Graph Relation Module (MGRM), which models the adaptation feasibility and transition probability of noisy samples with relation matrices; 3) Entropy-Aware Gradient Reconcilement (EAGR), which incorporates the semantic information into the discrimination process and enforces the gradients provided by noisy and clean samples to be consistent towards learning domain-invariant representations. A thorough evaluation on benchmark DAOD datasets with noisy source annotations validates the effectiveness of NLTE. In particular, NLTE improves the mAP by 8.4% under 60% corrupted annotations and even approaches the ideal upper bound of training on a clean source dataset. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Code is available at https://github.com/CityU-AIM-Group/NLTE.
BibTeX
@misc{liu2022towards,
  author = {Xinyu Liu and Wuyang Li and Qiushi Yang and Baopu Li and Yixuan Yuan},
  title = {{Towards Robust Adaptive Object Detection under Noisy Annotations}},
  year = {2022},
  eprint = {2204.02620},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.02620},
}
Figure