Additive Margin Softmax for Face Verification

Feng Wang, Weiyang Liu, Haijun Liu, Jian Cheng

TL;DR

This paper introduces AM-Softmax, a new loss function for deep face verification that adds an "additive margin" to the standard softmax loss. This simple yet effective modification improves the separation between different identities while keeping features of the same identity compact, leading to state-of-the-art performance on challenging benchmarks.

What Problem Does This Paper Address?

Deep learning models for face verification aim to learn feature representations that are highly discriminative, meaning features from the same person are very similar, while features from different people are very different. Traditional softmax loss is good at separating different classes but struggles to reduce the variation within a single class. Prior work like Large-margin Softmax and Angular Softmax (SphereFace) introduced multiplicative margins to address this, but they can be complex to implement and tune. This paper tackles the challenge of creating a more interpretable and easier-to-implement loss function that achieves superior face verification performance by enforcing a clear margin between identities.

Methods

The core contribution is the Additive Margin Softmax (AM-Softmax) loss function. Unlike previous methods that incorporated margins multiplicatively (e.g., by modifying the angle $\theta$), AM-Softmax introduces an additive margin by subtracting a fixed value '$m$' directly from the cosine similarity between the feature vector and the class weight vector: $\psi(\theta) = \cos\theta - m$.

Key aspects of their method include:

  • Additive Margin: This is the primary novelty, offering a more intuitive and interpretable way to enforce separation compared to multiplicative margins.
  • Feature and Weight Normalization: Crucially, they emphasize the importance of normalizing both the feature vectors and the weight vectors to unit length. This normalization ensures that the additive margin '$m$' directly translates to a consistent angular margin on the hypersphere, making the margin effective.
  • Global Scale Factor 's': Similar to other recent methods, they use a global scale factor '$s$' to amplify the logits before the softmax, further enhancing discriminability. They found fixing '$s$' to a large value (e.g., 30) stabilizes training.
  • Simpler Implementation: The additive nature of the margin means the gradient calculation for backpropagation is simpler ($\psi'(x) = 1$), making it easier to integrate into existing frameworks compared to methods like SphereFace.

Datasets

  • Training: CASIA-Webface [22]
  • Evaluation:
    • LFW (Labeled Faces in the Wild) [3] with various protocols (e.g., BLUFR).
  • MegaFace [5] (a large-scale dataset with many distractors).

They also performed experiments on Fashion MNIST [21] for visualization purposes, using a simplified 3D feature space.

Key Insights and Ideas

  • Simplicity is Key: The paper demonstrates that a conceptually simple modification (additive margin) can yield significant performance gains, often outperforming more complex methods.
  • Normalization is Crucial: The effectiveness of the additive margin is highly dependent on feature and weight normalization. Without it, the margin '$m$' doesn't guarantee a large angular separation.
  • Feature Normalization and Image Quality: They observe that feature normalization is particularly beneficial for low-quality images, as it prevents features with small norms (often from low-quality images) from dominating the gradient and potentially causing explosion. This is akin to a form of hard-sample mining.
  • Cosine vs. Angular Margin: While angular margins are conceptually appealing, they argue that cosine margins, as used in AM-Softmax, are computationally more efficient and achieve similar goals, especially when dealing with the non-linearity of cosine functions.

Key Results

  • AM-Softmax consistently outperforms state-of-the-art methods like A-Softmax (SphereFace) on LFW BLUFR and MegaFace benchmarks, using the same network architecture and training data.
  • For example, on MegaFace (Rank1@1e6), AM-Softmax with $m=0.35$ achieves 72.47% accuracy, surpassing A-Softmax's 67.41%.
  • The paper shows that the optimal performance is achieved with a margin '$m$' around 0.35-0.4.
  • Visualizations on a toy dataset (Fashion MNIST) show that AM-Softmax produces well-separated feature clusters on a hypersphere, similar to the best SphereFace configurations.

Limitations and Future Work

  • Hyperparameter Tuning: While AM-Softmax is simpler than some prior methods, the margin '$m$' is still a manually tuned global hyper-parameter.
  • Future Directions: The authors suggest exploring more creative ways to specify the margin function $\psi(\theta)$, potentially moving beyond simple multiplication and addition. They also propose investigating automatic margin determination and incorporating class-specific or sample-specific margins. They also mention the potential for re-weighting strategies to mitigate gradient explosion risks, especially when feature norms are very small.

Summary updated

Abstract

In this paper, we propose a conceptually simple and geometrically interpretable objective function, i.e. additive margin Softmax (AM-Softmax), for deep face verification. In general, the face verification task can be viewed as a metric learning problem, so learning large-margin face features whose intra-class variation is small and inter-class difference is large is of great importance in order to achieve good performance. Recently, Large-margin Softmax and Angular Softmax have been proposed to incorporate the angular margin in a multiplicative manner. In this work, we introduce a novel additive angular margin for the Softmax loss, which is intuitively appealing and more interpretable than the existing works. We also emphasize and discuss the importance of feature normalization in the paper. Most importantly, our experiments on LFW BLUFR and MegaFace show that our additive margin softmax loss consistently performs better than the current state-of-the-art methods using the same network architecture and training dataset. Our code has also been made available at https://github.com/happynear/AMSoftmax
BibTeX
@article{wang2018additive,
  author = {Feng Wang and Weiyang Liu and Haijun Liu and Jian Cheng},
  title = {{Additive Margin Softmax for Face Verification}},
  journal = {IEEE Signal Processing Letters},
  year = {2018},
  url = {https://openreview.net/forum?id=DDZHtLUt9o},
}
Figure