Abstract: In recent years, significant progress has been made in face recognition, which can be partially attributed to the availability of large-scale labeled face datasets. However, since the faces in these datasets usually contain limited degree and types of variation, the resulting trained models generalize poorly to more realistic unconstrained face datasets. While collecting labeled faces with larger variations could be helpful, it is practically infeasible due to privacy and labor cost. In comparison, it is easier to acquire a large number of unlabeled faces from different domains, which could be used to regularize the learning of face representations. We present an approach to use such unlabeled faces to learn generalizable face representations, where we assume neither the access to identity labels nor domain labels for unlabeled images. Experimental results on unconstrained datasets show that a small amount of unlabeled data with sufficient diversity can (i) lead to an appreciable gain in recognition performance and (ii) outperform the supervised baseline when combined with less than half of the labeled data. Compared with the state-of-the-art face recognition methods, our method further improves their performance on challenging benchmarks, such as IJB-B, IJB-C and IJB-S.
The Reality Gap in Face Recognition
Modern face recognition (FR) models perform exceptionally well on high-quality, web-crawled celebrity photos. However, these models often struggle when deployed in real-world, "unconstrained" environments—such as low-resolution surveillance footage, low-light mobile photos, or extreme camera angles. This discrepancy is known as the domain gap.
While state-of-the-art models are trained on massive labeled datasets like MS-Celeb-1M, these datasets lack the environmental variety found in the real world. Collecting and labeling millions of images for every possible deployment scenario is practically impossible. This research explores how to leverage the abundance of diverse, unlabeled face data to bridge this gap and regularize the learning of face representations.
A Semi-Supervised Framework for Robustness
The authors propose a framework that utilizes unlabeled images to make models more robust through two primary mechanisms:
1. Domain Adversarial Regularization (DA)
To ensure the model learns features that are useful across different environments, the system employs a discriminator. This discriminator attempts to distinguish between features extracted from labeled "clean" data and unlabeled "unconstrained" data. The embedding network is trained to "fool" this discriminator, forcing it to find a domain-agnostic feature space where a blurry surveillance face and a studio portrait of the same person are represented similarly.
2. Multi-mode Augmentation Network (AN)
This novel component is an unsupervised image-to-image translation module. It learns the "styles" (such as specific types of blur, lighting, or occlusions) present in the unlabeled data and applies them to the labeled celebrity images. By creating "hard" synthetic versions of known identities, the model can practice recognizing individuals under difficult conditions that mimic real-world noise.
Key Insights and Results
The study reveals several critical insights into the nature of face recognition training:
- Diversity Over Quantity: A striking finding is that adding a small amount of highly diverse unlabeled data (e.g., 70k images from the WiderFace dataset) is more effective for generalization than adding millions of additional labeled images that are similar to the existing training set.
- Saturation of Clean Data: Supervised learning on celebrity photos eventually hits a plateau. Once a model has seen enough "clean" identities, the bottleneck for performance becomes a lack of environmental variety rather than a lack of unique faces.
- State-of-the-Art Performance: The method achieved significant gains on challenging benchmarks. On the IJB-C dataset, the True Accept Rate (TAR) at a 1e-7 False Accept Rate jumped from a baseline of 62.90% to 77.39%. On the difficult IJB-S (Surveillance-to-Still) benchmark, Rank-1 accuracy improved from 53.23% to 59.29%.
Conclusion
By treating real-world degradation as a "style" to be learned and using adversarial alignment to normalize features, this approach significantly boosts the performance of face recognition models in unconstrained settings. The results demonstrate that unlabeled data, when sufficiently diverse, is a powerful and cost-effective tool for building more generalizable computer vision systems.
Summary updated