Introduction
Deep face recognition aims to map facial images onto a hypersphere where features of the same identity are clustered together and those of different identities are separated. Modern approaches typically employ cosine-based softmax loss functions, such as ArcFace or CosFace, which introduce a scale parameter ($s$) and an angular margin ($m$) to enhance discriminative power. However, these hyperparameters are notoriously difficult to tune, often requiring extensive trial-and-error. This paper introduces AdaCos, a hyperparameter-free alternative that adaptively scales cosine logits to achieve state-of-the-art performance without manual intervention.
The Problem: Hyperparameter Sensitivity
In standard cosine-based losses, the scale parameter $s$ determines the sharpness of the probability distribution, while the margin $m$ enforces a gap between classes. If $s$ is too small, the model lacks confidence; if too large, the loss becomes insensitive to errors, leading to poor gradients. Finding the optimal balance is a "black art" that depends heavily on the dataset size and complexity. AdaCos addresses this "tuning fatigue" by making the process fully automatic.
Methodology: Analyzing the Probability Curve
The authors analyzed how $s$ and $m$ affect the classification probability $P_{i,y_i}$. They observed that both parameters essentially serve the same purpose: modulating the mapping from the angle $\theta$ (the distance between a feature and its class center) to the probability.
As shown in the figure above, increasing $s$ or $m$ "squeezes" the probability curve, making it steeper. The authors argue that the explicit margin $m$ is redundant if $s$ is appropriately managed. Based on this, they developed two versions of AdaCos:
- Fixed AdaCos: An optimal fixed scale is derived based on the number of classes $C$: $\tilde{s}_f \approx \sqrt{2} \cdot \log(C - 1)$. This ensures the probability curve is steepest at a neutral point ($\pi/4$), providing a robust baseline.
- Dynamic AdaCos: This version automatically adjusts $s$ during training. By monitoring the median angle of positive pairs in each mini-batch, the loss function "senses" the model's convergence.
Dynamic Convergence Sensing
The core innovation of Dynamic AdaCos is its ability to tighten supervision as the model improves. As training progresses, the angles between features and their correct class centers ($\theta_{i,y_i}$) naturally decrease.
By using the median angle as a proxy for training progress, AdaCos dynamically increases $s$ to keep the probability curve in the "active" region where gradients are most effective. This eliminates the need for manual scheduling or margin tuning.
Key Results
AdaCos consistently outperformed previous state-of-the-art methods (ArcFace, CosFace, $l2$-softmax) across major benchmarks:
- LFW Accuracy: Achieved 99.71%, surpassing ArcFace (99.45%).
- MegaFace Identification: At the $10^6$ distractor level, AdaCos reached 97.41% accuracy, beating ArcFace (96.88%) and CosFace (95.50%).
- IJB-C Verification: At a False Accept Rate of $10^{-4}$, AdaCos achieved a True Accept Rate of 92.40%, the highest in the comparison.
- Training Stability: AdaCos showed much faster and more stable convergence in the early stages of training compared to traditional methods.
Conclusion
AdaCos effectively removes the need to tune $s$ and $m$ by treating the scale parameter as a dynamic participant in the optimization process. By simplifying the loss landscape and automatically sensing convergence, it provides a more robust and efficient framework for deep face recognition. Future work could involve extending this adaptive logic to other metric learning tasks such as person re-identification or image retrieval.
Summary updated