Towards Robust Adaptive Object Detection under Noisy Annotations

Xinyu Liu, Wuyang Li, Qiushi Yang, Baopu Li, Yixuan Yuan

TL;DR

This paper introduces the first framework specifically designed for Domain Adaptive Object Detection (DAOD) under noisy source annotations. The proposed Noise Latent Transferability Exploration (NLTE) framework effectively recaptures missing annotations and leverages class-corrupted samples to achieve robust performance, improving mAP by up to 8.4% and nearly matching the performance of models trained on perfectly clean data.

What Problem Does This Paper Address?

Domain Adaptive Object Detection (DAOD) is the art of training a detector on a labeled "source" domain (like clear sunny photos) so that it performs well on an unlabeled "target" domain (like foggy or artistic images). Standard DAOD methods rest on a shaky assumption: that source labels are perfectly clean.

In reality, large-scale datasets are riddled with errors. As shown in Figure 1, even benchmark datasets like Cityscapes contain miss-annotated instances (objects completely missed by humans) and class-corrupted instances (objects labeled as the wrong category).

Figure 1. Examples of noisy annotations in Cityscapes dataset. Miss-annotated samples: The bicycle in (a); the rider and car in (c). Class-corrupted samples: The rider and bicycle are labeled as person in (a); the motorcycle is labeled as bicycle in (b).

When these noisy labels are used for domain adaptation, they bias the source distribution, leading the model to learn incorrect transformations. While "learning with noisy labels" is a hot topic in image classification, those methods often fail in detection because they ignore the domain shift or struggle with the extreme foreground-background imbalance inherent in object detection.

Methods

The authors propose the NLTE framework, which consists of three main pillars designed to turn "noise" into "knowledge":

  1. Potential Instance Mining (PIM): Since the Region Proposal Network (RPN) is class-agnostic, it often finds objects even if a human missed them. PIM looks at high-confidence background proposals that don't overlap with existing ground truth and "recaptures" them as potential foreground instances to enrich source semantics.
  2. Morphable Graph Relation Module (MGRM): Instead of just trying to "fix" a wrong label, MGRM models how categories relate across domains. It uses graph-based feature aggregation to build a Global Relation Matrix (representing ideal class-wise transitions) and uses it to regularize a Local Relation Matrix derived from noisy samples. This allows the model to extract useful domain-invariant features even from incorrectly labeled objects.
  3. Entropy-Aware Gradient Reconcilement (EAGR): This is the "peacekeeper" of the training process. It uses a first-order meta-learning update to ensure that the gradients produced by noisy samples are consistent with those from clean samples. By aligning these gradients, the model ensures that all data points contribute toward a unified, domain-invariant representation.
Figure 3. Overview of our NLTE framework, which includes PIM, MGRM and EAGR. c⃝is the concatenation operation.

Datasets

The researchers evaluated NLTE across several challenging scenarios:

  • Synthetic Noise: They took the clean Pascal VOC dataset and manually injected noise at various rates (20% to 80%) before adapting to Clipart1k and Watercolor2k.
  • Real-world Noise: They used Cityscapes (which has inherent labeling errors) as the source and Foggy Cityscapes as the target.

Key Insights and Ideas

  • Noise as a Resource: Unlike classification-based denoising that filters out noisy samples, NLTE recognizes that even a mislabeled source image contains valuable domain-specific information (like lighting or texture) that is crucial for domain alignment.
  • RPN as a Denoising Tool: The realization that a class-agnostic RPN can "out-detect" a noisy human annotator is a clever way to handle missing labels without needing a separate pre-trained model.
  • Gradient Harmony: By framing the problem as a gradient consistency task via meta-learning, the authors avoid the need to explicitly separate "clean" from "noisy" samples during every training step, which is computationally expensive and error-prone.

Key Results

The results demonstrate that NLTE is remarkably robust. In the Pascal VOC to Clipart1k task, NLTE significantly outperformed standard baselines (DAF) and general noise-robust methods (SCE, GCE).

Figure 2. Performance comparison of baseline domain adaptive detector DAF [6], adding noise-robust learning method SCE [45], and adding the proposed NLTE under different noisy rates in Pas- cal VOC & Noisy Pascal VOC →Clipart1k.
  • Significant Gains: At a 60% noise rate, NLTE improved the baseline mAP by 8.4%.
  • Approaching the Upper Bound: Remarkably, at 60% noise, NLTE's performance was only 2% lower than a model trained on a 100% clean source dataset.
  • Real-world Success: On the Cityscapes to Foggy Cityscapes benchmark, NLTE achieved an mAP of 45.4%, outperforming recent state-of-the-art DAOD methods like SSAL.

Limitations and Future Work

While the paper presents a comprehensive solution, a few areas remain for exploration:

  • Computational Complexity: The use of graph modules and meta-learning updates likely increases training time and memory requirements compared to standard detectors.
  • RPN Dependency: The effectiveness of PIM is tied to the quality of the RPN. In extremely difficult domains where the RPN fails to generate high-confidence proposals, the benefits of PIM might diminish.
  • Future Directions: The authors suggest that addressing noisy annotations is a highly effective alternative to standard adaptation techniques, hinting that future DAOD research should perhaps focus more on data quality and label robustness.

Summary updated

Abstract

Domain Adaptive Object Detection (DAOD) models a joint distribution of images and labels from an annotated source domain and learns a domain-invariant transformation to estimate the target labels with the given target domain images. Existing methods assume that the source domain labels are completely clean, yet large-scale datasets often contain error-prone annotations due to instance ambiguity, which may lead to a biased source distribution and severely degrade the performance of the domain adaptive detector de facto. In this paper, we represent the first effort to formulate noisy DAOD and propose a Noise Latent Transferability Exploration (NLTE) framework to address this issue. It is featured with 1) Potential Instance Mining (PIM), which leverages eligible proposals to recapture the miss-annotated instances from the background; 2) Morphable Graph Relation Module (MGRM), which models the adaptation feasibility and transition probability of noisy samples with relation matrices; 3) Entropy-Aware Gradient Reconcilement (EAGR), which incorporates the semantic information into the discrimination process and enforces the gradients provided by noisy and clean samples to be consistent towards learning domain-invariant representations. A thorough evaluation on benchmark DAOD datasets with noisy source annotations validates the effectiveness of NLTE. In particular, NLTE improves the mAP by 8.4\% under 60\% corrupted annotations and even approaches the ideal upper bound of training on a clean source dataset.
BibTeX
@misc{liu2022towards,
  author = {Xinyu Liu and Wuyang Li and Qiushi Yang and Baopu Li and Yixuan Yuan},
  title = {{Towards Robust Adaptive Object Detection under Noisy Annotations}},
  year = {2022},
  eprint = {2204.02620},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.02620},
}
Figure