Co-Mining: Deep Face Recognition With Noisy Labels

Xiaobo Wang, Shuo Wang, Hailin Shi, Jun Wang, Tao MeiICCV 2019 · DOI

Abstract

Deep face recognition models require massive datasets, which are often plagued by noisy labels due to automated web-scraping. We present Co-Mining, a novel training strategy that utilizes two neural networks to peer-review and exchange high-confidence samples. This approach effectively identifies noisy data, prevents error loops through sample exchange, and prioritizes clean images to build highly discriminative face representations even on "dirty" datasets.

Introduction

Deep learning for face recognition is a data-hungry field. To build state-of-the-art models, researchers utilize massive datasets like MS-Celeb-1M, which contains millions of images. However, because these datasets are often scraped automatically from the web, they are frequently riddled with noisy labels.

The authors categorize this noise into two distinct types:

  1. Label Flips (Closed-set): An image of Person A is mistakenly labeled as Person B, where Person B is also an identity present in the dataset.
  2. Outliers (Open-set): An image of a person who does not belong to any category in the dataset is assigned a random label.
Visualization of label flips and outliers in face recognition datasets.

Standard training algorithms, such as ArcFace, assume labels are perfect. When they are not, models tend to "memorize" these mistakes, which significantly degrades their ability to generalize to real-world faces. Since manual cleaning is prohibitively expensive, an algorithmic solution is required to handle large-scale label noise.

Methodology: The Co-Mining Framework

The proposed Co-Mining framework trains two "peer" networks (Backbone-1 and Backbone-2) simultaneously. The core logic relies on the "memorization effect": neural networks tend to learn simple, clean patterns before they begin memorizing noisy outliers.

The Co-Mining training framework featuring dual backbones and sample exchange.

The method consists of three primary modules:

  • Noisy Label Detection: In every mini-batch, each network calculates the loss for its samples. Samples with the highest loss are flagged as potential noise and dropped from the current update.
  • High-Confidence Clean Faces Exchanging: To avoid "sample-selection bias"—where a network only learns what it already knows, reinforcing its own errors—the two networks exchange their "high-confidence" clean samples. If Network A is confident a sample is clean while Network B is uncertain, Network B uses that sample to learn, and vice versa.
  • Clean Faces Re-weighting: When both networks agree a sample is clean (the intersection of their clean sets), the algorithm assigns that sample extra weight in the loss function. This ensures that the most reliable data dominates the training process.

Additionally, the system employs an Adaptive Noise Rate. It begins by keeping most data and gradually increases the "drop rate" as the networks become more confident in identifying noise.

Experimental Results

The researchers evaluated Co-Mining using several industry-standard datasets, including CASIA-WebFace, MS-Celeb-1M, and VggFace2. They created "refined" versions (e.g., MsCeleb-R) to ensure fair open-set evaluation. Testing was conducted across a wide battery of benchmarks, including LFW, CFP, and the MegaFace challenge.

Co-Mining consistently outperformed both standard baselines and other state-of-the-art noise-handling methods:

  • MS-Celeb-1M: On this notoriously noisy dataset, Co-Mining achieved an average accuracy of 92.02% across multiple benchmarks, compared to 88.89% for the standard Arc-Softmax baseline.
  • Synthetic Noise: In experiments with 30% intentional noise, Co-Mining maintained an average accuracy of 81.32%, while the baseline plummeted to 58.81%.
  • MegaFace: On the MegaFace challenge (trained on VggFace2-R), Co-Mining achieved 81.51% Rank-1 accuracy, significantly outperforming other robust training methods like De-Coupling and Co-teaching.

Conclusion and Future Work

Co-Mining demonstrates that loss values can serve as a reliable proxy for label quality and that peer-review between dual networks prevents the "self-evolution" trap of overconfidence in errors. While the method effectively handles both closed-set and open-set noise, it does introduce computational overhead by requiring the simultaneous training of two networks. Future work may involve extending this "co-mining" philosophy to other computer vision tasks, such as fine-grained image classification or object detection in noisy environments.

Summary updated

BibTeX
@inproceedings{wang2019co,
  author = {Xiaobo Wang and Shuo Wang and Hailin Shi and Jun Wang and Tao Mei},
  title = {{Co-Mining: Deep Face Recognition With Noisy Labels}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00945},
}
Figure