Abstract
As a widely used loss function in deep face recognition, the softmax loss cannot guarantee that the minimum positive sample-to-class similarity is larger than the maximum negative sample-to-class similarity. As a result, no unified threshold is available to separate positive sample-to-class pairs from negative sample-to-class pairs. To bridge this gap, we design a UCE (Unified Cross-Entropy) loss for face recognition model training, which is built on the vital constraint that all the positive sample-to-class similarities shall be larger than the negative ones. Our UCE loss can be integrated with margins for a further performance boost. The face recognition model trained with the proposed UCE loss, UniFace, was intensively evaluated using a number of popular public datasets like MFR, IJB-C, LFW, CFP-FP, AgeDB, and MegaFace. Experimental results show that our approach outperforms SOTA methods like SphereFace, CosFace, ArcFace, Partial FC, etc. Especially, till the submission of this work (Mar. 8, 2023), the proposed UniFace achieves the highest TAR@MR-All on the academic track of the MFR-ongoing challenge.
Introduction: The Threshold Gap in Face Recognition
In deep face recognition, models are typically trained to map faces into a feature space where "same-identity" pairs are close and "different-identity" pairs are far apart. Most modern systems use Softmax loss or its popular variants like ArcFace and CosFace to achieve this.
However, a critical flaw exists in the standard Softmax objective: it only ensures that a sample is closer to its correct class proxy than to any other proxy in a given batch. It does not guarantee a "unified threshold"—a single similarity value that can perfectly separate all positive pairs from all negative pairs across the entire dataset. In real-world deployment (e.g., biometric authentication), a fixed threshold is required to decide if two faces match. Without it, the similarity distributions of matches and non-matches overlap, leading to higher error rates.
Methods: Unified Cross-Entropy (UCE) Loss
The core contribution of this work is the Unified Cross-Entropy (UCE) loss. Unlike standard Softmax, which focuses on "row dominance" (picking the right class for one sample), UCE encourages "column dominance" (ensuring a class proxy is only similar to its own samples).
Key technical components include:
- Learnable Threshold ($t$): For the first time, the authors incorporate the unified threshold as an automatic, learnable parameter within the loss function. This allows the model to find the optimal boundary between identities during training.
- Marginal UCE: The authors extend UCE by adding a cosine margin ($m$), similar to the logic in ArcFace, to further push the boundaries between classes for better discriminative power.
- Balanced UCE: To handle the massive class imbalance (one positive pair vs. $N-1$ negative pairs in a batch), they introduce re-weighting ($\lambda$) and sampling ($r$) strategies to prevent the gradients from being overwhelmed by negative samples.
Key Insights and Ideas
Aligning Training with Verification
The authors realized that while Softmax loss is effective for classification, it doesn't align perfectly with the verification task. By forcing the model to respect a global threshold $t$ during training, they align the training objective directly with the deployment goal.
Robustness to Hyperparameters
One of the most significant practical advantages of UCE is its robustness to hyperparameter tuning. While standard marginal softmax performance often "crashes" if the margin $m$ is set too high, UCE maintains stable performance across a wider range of values.
Experimental Results
UniFace was trained on massive datasets including CASIA-WebFace (0.5M images), Glint360K (17.1M images), and WebFace42M (42.5M images). It consistently outperformed SOTA methods like ArcFace, CosFace, and SphereFace across multiple benchmarks:
- MFR-ongoing: UniFace achieved the highest TAR@MR-All (True Accept Rate on the Multi-Racial set) on the academic track at the time of submission.
- IJB-C: On a ResNet-50 backbone, UCE improved the TAR@FAR=1e-4 from 71.53% (Softmax) to 74.80%.
- MegaFace: Achieved 99.27% Rank-1 identification and 99.19% verification accuracy when trained on Glint360K, setting a new bar for the challenge.
Conclusion
UniFace introduces a paradigm shift by incorporating the verification threshold directly into the training loss. By bridging the gap between classification-based training and threshold-based verification, UCE loss provides a more robust and high-performing framework for large-scale face recognition. Future work may explore identity-adaptive thresholds to further reduce demographic bias and improve fairness in diverse datasets.
Summary updated