Learning with Twin Noisy Labels for Visible-Infrared Person Re-Identification

Mouxing Yang, Zhenyu Huang, Peng Hu, Taihao Li, Jiancheng Lv, Xi Peng

Introduction

Visible-Infrared Person Re-identification (VI-ReID) is a critical task for modern surveillance systems, enabling the tracking of individuals across visible-light cameras during the day and infrared cameras at night. While standard ReID has seen significant progress, VI-ReID faces a unique and often overlooked challenge: the quality of data annotations. Infrared images are frequently grainy and lack color information, making accurate human labeling difficult. This paper identifies a fundamental problem in this domain termed Twin Noisy Labels (TNL) and proposes a robust training framework called DART (DuAlly Robust Training) to mitigate its effects.

The Twin Noisy Label Problem

Most existing ReID algorithms operate under the assumption of near-perfect identity labels. However, in real-world VI-ReID datasets, two types of noise inevitably emerge:

  1. Noisy Annotations (NA): This refers to incorrect identity labels assigned to images, often due to the difficulty of distinguishing individuals in low-quality infrared footage.
  2. Noisy Correspondence (NC): In multi-modal learning, cross-modal pairs are constructed based on identity labels. If an identity is mislabeled in one modality, the resulting pairing between the visible and infrared images becomes a "mismatch."

Together, these form the "Twin" problem: identity noise in one domain automatically contaminates the correspondence between domains. Standard models attempting to force-learn these incorrect associations often suffer from severe performance degradation.

Figure 1: Illustration of Noisy Annotations (NA) and Noisy Correspondence (NC). NA occurs when identities are mislabeled (red boxes), which leads to NC where the model incorrectly attempts to pull together different people (FP) or push apart the same person (FN).

DART: DuAlly Robust Training

To address TNL, the authors introduce the DART framework, which focuses on identifying and rectifying noisy samples rather than simply discarding them. The framework consists of three integrated stages:

1. Co-modeling (The Clean Filter)

DART employs two identical networks (A and B) that train simultaneously. This approach leverages the "memorization effect" of deep neural networks—the tendency for models to learn simple, clean patterns before overfitting to noise. Each network estimates the "clean confidence" of samples for its counterpart, providing a peer-review mechanism that avoids the confirmation bias inherent in single-network self-correction.

2. Pair Division

Instead of treating all noisy data as unusable, DART categorizes image pairs into four distinct groups based on the confidence scores: True Positives (TP), True Negatives (TN), False Positives (FP), and False Negatives (FN). This fine-grained division allows the model to understand how a label is noisy.

3. Dually Robust Loss

The core of DART is its specialized loss function:

  • Soft Identification Loss: This weights the standard classification loss by the estimated confidence, effectively reducing the influence of likely-mismatched samples.
  • Adaptive Quadruplet Loss: This is the framework's primary innovation. While standard triplet losses pull labeled positives together and push negatives apart, DART’s adaptive loss changes behavior based on the pair category. For instance, if a pair is identified as a False Positive (labeled as the same person but predicted to be different), the loss adaptively switches to push them apart, rectifying the supervision signal.
Figure 3: The DART training pipeline and framework. It uses co-teaching between two networks (A and B) to divide pairs and apply a dually robust loss consisting of soft identification and adaptive quadruplet components.

Experimental Results

The researchers evaluated DART on the SYSU-MM01 and RegDB benchmarks, simulating various noise levels by randomly flipping identity labels. The results demonstrate remarkable resilience:

  • Superiority in High Noise: On the SYSU-MM01 dataset with a 20% noise ratio, DART achieved a 66.31% Rank-1 accuracy, significantly outperforming baseline models like ADP, which dropped to approximately 25%.
  • Extreme Robustness: Even at a 50% noise ratio—where half of the labels are incorrect—DART maintained a Rank-1 accuracy of 60.27%. This performance is comparable to some state-of-the-art models operating on perfectly clean data.
  • Plug-and-Play Capability: The authors demonstrated that DART can be integrated into other existing architectures (such as AGW), providing a significant boost in performance whenever label noise is present.

Conclusion

DART represents a significant step forward in making VI-ReID systems practical for real-world deployment where perfect data is rare. By recognizing the twin nature of noise in multi-modal tasks and opting to rectify rather than discard noisy samples, the framework maintains high accuracy even under extreme label corruption. Future work may explore how this approach handles "structured noise," where similar-looking individuals are consistently mislabeled, further bridging the gap between laboratory benchmarks and real-world application.

Summary updated

Abstract

In this paper, we study an untouched problem in visible-infrared person re-identification (VI-ReID), namely, Twin Noise Labels (TNL) which refers to as noisy annotation and correspondence. In brief, on the one hand, it is inevitable to annotate some persons with the wrong identity due to the complexity in data collection and annotation, e.g., the poor recognizability in the infrared modality. On the other hand, the wrongly annotated data in a single modality will eventually contaminate the cross-modal correspondence, thus leading to noisy correspondence. To solve the TNL problem, we propose a novel method for robust VI-ReID, termed DuAlly Robust Training (DART). In brief, DART first computes the clean confidence of annotations by resorting to the memorization effect of deep neural networks. Then, the proposed method rectifies the noisy correspondence with the estimated confidence and further divides the data into four groups for further utilizations. Finally, DART employs a novel dually robust loss consisting of a soft identification loss and an adaptive quadruplet loss to achieve robustness on the noisy annotation and noisy correspondence. Extensive experiments on SYSU-MM01 and RegDB datasets verify the effectiveness of our method against the twin noisy labels compared with five state-of-the-art methods. The code could be accessed from https://github.com/XLearning-SCU/2022-CVPR-DART.
BibTeX
@inproceedings{yang2022learning,
  author = {Mouxing Yang and Zhenyu Huang and Peng Hu and Taihao Li and Jiancheng Lv and Xi Peng},
  title = {{Learning with Twin Noisy Labels for Visible-Infrared Person Re-Identification}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  doi = {10.1109/cvpr52688.2022.01391},
}
Figure