Gradient-Enhanced Softmax for Face Recognition

Linjun Sun, Weijun Li, Xin Ning, Liping Zhang, Xiaoli Dong, Wei HeIEICE Trans. Information and Systems 2020 · OpenReview

Abstract

This letter proposes a gradient-enhanced softmax supervisor for face recognition (FR) based on a deep convolutional neural network (DCNN). The proposed supervisor conducts the constant-normalized cosine to obtain the score for each class using a combination of the intra-class score and the soft maximum of the inter-class scores as the objective function. This mitigates the vanishing gradient problem in the conventional softmax classifier. The experiments on the public Labeled Faces in the Wild (LFW) database denote that the proposed supervisor achieves better results when compared with those achieved using the current state-of-the-art softmax-based approaches for FR.

The "Lazy" Supervisor Problem

In deep learning-based face recognition, the loss function (or "supervisor") is responsible for guiding the model's weight adjustments. The standard tool for this is the softmax classifier. However, a fundamental flaw exists: as a model becomes proficient at classification, the gradient—the signal used for learning—approaches zero.

Visualization of feature distributions (a) and gradient decay (b) in standard softmax. As the model gains confidence, the learning signal vanishes, leading to suboptimal feature clustering.

Once the model is "confident enough" that a face belongs to a specific person, it stops trying to improve. This "vanishing gradient" prevents the model from further shrinking intra-class variation or expanding inter-class variation. Essentially, the model becomes "lazy" once it reaches a basic level of accuracy, leaving potential performance gains on the table.

Methodology: Gradient-Enhanced Softmax (GESoftmax)

To address this, we propose GESoftmax, which modifies the traditional softmax framework in two key ways:

  1. Constant-Normalized Cosine Scores: Instead of raw dot products, we use normalized cosine similarity multiplied by a temperature parameter ($\alpha$). This prevents feature magnitude from dominating the classification process.
  2. Gradient Enhancement: The loss function ($L_{GES}$) is redefined as a weighted combination of the target class score and the "soft maximum" of all other class scores.
The architecture of the proposed GESoftmax loss, which introduces a constant gradient term to maintain a strong supervisory signal throughout training.

The core innovation is the hyperparameter $\lambda$. Unlike standard softmax, where the gradient disappears as loss decreases, the gradient of GESoftmax for the target class remains a negative constant ($\lambda - 1$). This provides a persistent "push" that keeps the training process active, forcing the network to continue refining feature distributions long after a standard classifier would have plateaued.

Experimental Results

The proposed method was evaluated against several state-of-the-art approaches across multiple benchmarks:

  • LFW Accuracy: GESoftmax ($\lambda=0.475$) achieved 98.97%, surpassing AM-Softmax (98.62%).
  • BLUFR Performance: On the challenging "Detection Identification Rate" (DIR @ FAR=1%), GESoftmax achieved 88.16%, a significant jump over AM-Softmax’s 80.20%.
  • Efficiency: These gains were achieved using MobileFaceNet, a lightweight architecture, proving that the improvement comes from the loss function logic rather than raw computing power.

Conclusion

GESoftmax effectively addresses the vanishing gradient problem in face recognition by ensuring a persistent supervisory signal. This forces the network to learn more compact and discriminative facial features, leading to superior performance on benchmarks like LFW and BLUFR. Future work will focus on automated methods for determining the optimal $\lambda$ for different datasets.

Summary updated

Abstract

This letter proposes a gradient-enhanced softmax supervisor for face recognition (FR) based on a deep convolutional neural network (DCNN). The proposed supervisor conducts the constant-normalized cosine to obtain the score for each class using a combination of the intra-class score and the soft maximum of the inter-class scores as the objective function. This mitigates the vanishing gradient problem in the conventional softmax classifier. The experiments on the public Labeled Faces in the Wild (LFW) database denote that the proposed supervisor achieves better results when compared with those achieved using the current state-of-the-art softmax-based approaches for FR.
BibTeX
@article{sun2020gradient,
  author = {Linjun Sun and Weijun Li and Xin Ning and Liping Zhang and Xiaoli Dong and Wei He},
  title = {{Gradient-Enhanced Softmax for Face Recognition}},
  journal = {IEICE Trans. Information and Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=LzAEKzH59x},
}
Figure