TL;DR
This paper introduces a novel three-stage graph-based approach using community detection to automatically clean extremely large, noisy face datasets. By identifying and removing mislabeled images while preserving data diversity, they created the largest publicly available clean face dataset, C-MS-Celeb, which significantly improves face recognition model performance.
What Problem Does This Paper Address?
Building massive face recognition datasets, like the widely used MS-Celeb-1M, is often done by scraping images from the internet. While this provides a vast number of images, it inevitably leads to a significant number of mislabeled entries. Manually cleaning these datasets is prohibitively time-consuming and expensive, especially for datasets with millions of images. Existing automated cleaning methods often struggle with the inherent diversity of human faces (e.g., variations in age, expression, lighting), leading to the removal of correctly labeled but visually different images, or they lack tolerance for the high proportion of noisy labels common in web-scraped data. This paper aims to address the challenge of automatically cleaning large-scale face datasets to improve both the accuracy and diversity of the data, ultimately benefiting face recognition model training.
Methods
The authors propose a three-stage cleaning pipeline:
- Face Alignment and Feature Extraction: Pre-trained deep Convolutional Neural Networks (CNNs) are used to detect, align, and extract a 128-dimensional feature vector for each face image. This step standardizes the input for subsequent analysis.
- Community-Based Cleaning:
- Face Similarity Graph Construction: For images of the same identity, a graph is built where nodes represent face features and edges represent cosine similarity between them. Stronger similarities result in higher edge weights.
- Community Detection: Weak edges (below a certain threshold determined by open-set face identification theory) are removed to isolate noisy images. Then, a fast community detection algorithm is applied. Communities (groups of similar faces) that are too small (below a threshold 'p') are considered to represent mislabeled images and are deleted. Larger communities are retained. This stage leverages the idea that correctly labeled faces of the same person will form strong, cohesive communities, while mislabeled faces will be more isolated or form smaller, disparate groups.
- Relabeling Previously Deleted Images: Images deleted in Stage 2 are given a chance to be relabeled. Their features are compared to the centers of the reserved communities from Stage 2. If a deleted image's feature is sufficiently similar to a community's center (above a threshold 'η'), it is assigned the identity of that community. This step aims to recover potentially useful data that was initially discarded.
Datasets
- MS-Celeb-1M: This is the primary dataset used for cleaning. It's a large-scale dataset containing approximately 8.4 million images of nearly 100,000 celebrities, known for its high proportion of noisy labels.
- C-MS-Celeb: This is the newly created clean version of the MS-Celeb-1M dataset, containing 6,464,016 images of 94,682 celebrities.
- LFW (Labeled Faces in the Wild): Used for evaluating the performance of face recognition models trained on the cleaned datasets and for determining parameters (thresholds τ and η) for the cleaning process.
Key Insights and Ideas
- Community Structure as a Proxy for Identity: The core idea is that correctly labeled images of the same person will form dense, interconnected "communities" in a feature space, while mislabeled images will be outliers or form very small, disconnected groups.
- Balancing Cleanness and Diversity: The method is designed to achieve high cleanness (removing mislabeled images) without sacrificing data diversity (retaining varied images of the same person). Representing a single identity by multiple communities allows for capturing different facets of a person's appearance.
- Leveraging Open-Set Identification Theory: Using principles from open-set face identification to set thresholds for edge removal and relabeling helps ensure robustness against noisy data and maintain high accuracy.
- Incremental Cleaning for Better Results: The paper demonstrates that a multi-stage cleaning process, including relabeling deleted images, leads to a better final dataset and improved model performance compared to single-stage cleaning.
- Largest Publicly Available Clean Dataset: The creation of C-MS-Celeb is highlighted as a significant contribution, providing a valuable resource for the research community.
Key Results
- The proposed method successfully cleaned the MS-Celeb-1M dataset, resulting in C-MS-Celeb with 6,464,016 images of 94,682 celebrities, where approximately 97.3% of images are correctly labeled.
- Training an Inception-ResNet-v1 model on C-MS-Celeb achieved an Equal Error Rate (EER) of 99.67% on the LFW benchmark, outperforming models trained on the original MS-Celeb-1M and other state-of-the-art results.
- The C-MS-Celeb dataset is claimed to be the largest clean face dataset publicly available at the time of publication.
- The method demonstrated superior performance in terms of both cleanness and diversity compared to other cleaning methods like Maximal Subgraph Method (MSM), K-means Clustering Method (KCM), Fixed Proportion Removal (FPR), and Light CNN Cleaning (LCNN).
Limitations and Future Work
- Prerequisites for Relabeling: The relabeling stage (Stage 3) relies on the assumption that most mislabeled images are actually celebrity faces and that the dataset contains a large number of distinct celebrities. This assumption holds for MS-Celeb-1M but might not be universally applicable to all noisy datasets.
- Parameter Sensitivity: While the paper uses open-set identification theory to guide parameter selection (τ and η), the choice of 'p' for community size threshold in Stage 2 is still empirical and might require tuning for different datasets.
- Future Work: The paper implicitly suggests that further refinement of the cleaning process and exploration of different community detection algorithms could yield even better results. The creation of C-MS-Celeb itself opens avenues for further research in face recognition using a large-scale, clean dataset.
Summary updated