Introduction
Face recognition has traditionally been a "data-hungry" field, relying on massive datasets like MS-Celeb-1M with millions of manually labeled images. While self-supervised learning (SSL) has revolutionized general computer vision, it has historically struggled with faces. Standard SSL treats every image as a unique "instance," whereas face recognition requires grouping different images of the same person together.
UCoL (Uncertainty-aware Contrastive Learning) is a framework that allows models to learn discriminative facial features without a single human-provided label. It bridges the performance gap with supervised methods by using a dual-path momentum network and a novel uncertainty-aware $k$-nearest neighbors (kNN) strategy to identify "pseudo-positive" pairs across different images.
Addressing the Identity Gap
The primary challenge in facial recognition is learning features that distinguish similar-looking strangers while recognizing the same person across variations in age or lighting. Standard contrastive learning (e.g., MoCo) makes an image's representation similar to its own augmentations (intra-instance) but treats all other images as negatives. In face recognition, these "negatives" often include other photos of the same person, which destroys the model's ability to cluster identities. UCoL addresses this by identifying inter-instance pairs online during training.
Methodology
UCoL builds on the Momentum Contrast (MoCo) framework with two major innovations:
- Dual-path Momentum Network: The framework utilizes two training flows. Path 1 (Intra-instance) follows the standard MoCo approach, comparing an image to its own augmentations. Path 2 (Inter-instance) compares an image to other images in the dataset predicted to represent the same identity.
- Uncertainty-aware Pairwise Self-labeling: To find inter-instance pairs without labels, UCoL identifies $k$-nearest neighbors in the feature space. To filter noise, it employs two consistency checks:
- Multi-view Consistency: A pair is considered "positive" only if they remain neighbors across different random augmentations.
- MC-Dropout Consistency: Multiple stochastic forward passes are run using Dropout. If the neighbor relationship remains stable, confidence in the pair is high.
- Margin-based InfoNCE: The standard contrastive loss is replaced with a version incorporating a margin (similar to CosFace), forcing the model to push different identities further apart and create tighter clusters.
Experimental Setup
The authors evaluated UCoL on major face recognition benchmarks:
- Training: MS-Celeb-1M (5.1M images) with all identity labels removed to simulate an unsupervised environment.
- Evaluation: LFW (Labeled Faces in the Wild) for verification, MegaFace (1 million distractors) for large-scale challenge, and IJB-C for varying image quality.
- Scaling: Pre-training on the massive WebFace260M dataset.
Key Results
UCoL significantly outperforms previous unsupervised methods and rivals semi-supervised ones. It effectively "solves" the identity collapse problem inherent in standard SSL for faces.
- LFW Accuracy: Achieved 99.00%, remarkably close to supervised ArcFace (99.82%) and far exceeding the MoCo baseline (84.53%).
- MegaFace (Rank-1): Achieved 81.02%, a massive jump from standard MoCo's 6.53%.
- Ablation Studies: The uncertainty-aware consistency check provided a ~3% boost on MegaFace, demonstrating the value of selective pseudo-labeling.
- Scalability: When pre-trained on WebFace260M, the MegaFace Rank-1 score reached 89.71%, showing the method's effectiveness on massive unlabeled data.
Conclusion and Future Work
UCoL demonstrates that discriminative identity-level features can be learned entirely from unlabeled data by leveraging uncertainty to filter pseudo-labels. While it nearly matches semi-supervised performance, a small gap remains compared to the best supervised models. Future work may explore more robust initialization strategies to improve early-stage kNN search and further reduce the computational overhead of stochastic passes.
Summary updated