DAM: Discrepancy Alignment Metric for Face Recognition

Jiaheng Liu, Yudong Wu, Yichao Wu, Chuming Li, Xiaolin Hu, Ding Liang, Mengyu Wang

Face recognition (FR) has achieved remarkable performance through deep learning, yet a fundamental inconsistency persists between how these models are trained and how they are deployed. While state-of-the-art loss functions utilize relative margins to separate identities, inference typically relies on absolute similarity scores. Researchers from Beihang University, SenseTime, and Tsinghua University have introduced the Discrepancy Alignment Metric (DAM) to bridge this "training-inference gap."

The Training-Inference Gap

Most modern FR loss functions, such as ArcFace or CosFace, are designed to be relative. They focus on ensuring that a "positive" similarity score ($s_p$) exceeds a "negative" similarity score ($s_n$) by a specific margin. However, during real-world verification, we apply a global absolute threshold (e.g., 0.7) to determine matches.

This approach fails to account for the non-uniform distribution of the feature space. Some identities reside in "crowded" regions where similarity scores are naturally lower, while others occupy "sparse" areas. As illustrated in Figure 1, an identity (ID2) might maintain a sufficient relative margin, but because its absolute scores fall below the global threshold, its positive pairs are incorrectly classified as false negatives.

Figure 1: Comparison of standard cosine similarity (a) and calibrated similarity (b). A global threshold in (a) causes false negatives for ID2, whereas DAM calibration in (b) aligns the distributions to improve accuracy.

Methodology: Discrepancy Alignment Metric (DAM)

To resolve this, the authors propose DAM, a "plug-and-play" inference metric that replaces standard cosine similarity. DAM normalizes similarity scores based on the Local Inter-class Discrepancy (LID) of each face, effectively calibrating the score according to the local density of the feature space.

The DAM Pipeline

The core formula for DAM is defined as: $$DAM(f_1, f_2) = e^{s \langle f_1, f_2 \rangle} \cdot \left( \frac{1}{G(f_1)} + \frac{1}{G(f_2)} \right)$$ Where $G(f)$ represents the LID—a measure of how much a face embedding overlaps with its nearest neighbors from other classes.

Figure 5: The DAM inference pipeline. Features extracted from an FR model are used to estimate Local Inter-class Discrepancy (LID) using a neighboring embedding set, resulting in a calibrated similarity score.

Estimating LID (LIDE)

The authors propose two practical methods for estimating $G(f)$:

  1. Reference-based (DAM-R): Uses an "anchor embedding set" of diverse face features. For any new face, the system searches for its $k$-nearest neighbors in this set to estimate local discrepancy. To address privacy concerns, the authors demonstrated that GAN-generated "fake" faces work as effectively as real biometric data.
  2. Learning-based (DAM-L): Employs a lightweight, two-layer regressor to predict the LID value directly from a feature vector, eliminating the need for a neighbor search during inference.

Key Results and Insights

DAM consistently improves performance across major benchmarks without requiring model retraining. Key findings include:

  • Performance Gains: On the rigorous IJB-C dataset, adding DAM-R to ArcFace improved the True Accept Rate (TAR) from 95.63% to 95.78% at a False Accept Rate of 1e-4. On the LFW dataset, accuracy rose from 98.73% to 99.03%.
  • Fairness and Robustness: Testing on the RFW (Racial Faces in the Wild) dataset showed that DAM-R improved average accuracy across different ethnicities from 94.63% to 95.08%, helping to mitigate performance disparities caused by non-uniform data distributions.
  • The "L-Shaped" Curve: The research revealed that only a few "hard" negative classes significantly impact discrepancy. This justifies using a small number of top-$k$ neighbors for LID estimation, keeping the computational overhead manageable.

Conclusion

DAM provides a mathematically grounded solution to ensure that the relative margins learned during training are preserved during deployment. By shifting from a rigid global threshold to a locally-aware metric, DAM offers a simple yet powerful way to enhance the accuracy and fairness of existing face recognition systems.

Summary updated

Abstract

The field of face recognition (FR) has witnessed remarkable progress with the surge of deep learning. The effective loss functions play an important role for FR. In this paper, we observe that a majority of loss functions, including the widespread triplet loss and softmax-based cross-entropy loss, embed inter-class (negative) similarity s <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</inf> and intra-class (positive) similarity s <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</inf> into similarity pairs and optimize to reduce (s <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</inf> − s <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</inf> ) in the training process. However, in the verification process, existing metrics directly take the absolute similarity between two features as the confidence of belonging to the same identity, which inevitably causes a gap between the training and verification process. To bridge the gap, we propose a new metric called Discrepancy Alignment Metric (DAM) for verification, which introduces the Local Inter-class Discrepancy (LID) for each face image to normalize the absolute similarity score. To estimate the LID of each face image in the verification process, we propose two types of LID Estimation (LIDE) methods, which are reference-based and learning-based estimation methods, respectively. The proposed DAM is plug-and-play and can be easily applied to the most existing methods. Extensive experiments on multiple popular face recognition benchmark datasets demonstrate the effectiveness of our proposed method.
BibTeX
@inproceedings{liu2021dam,
  author = {Jiaheng Liu and Yudong Wu and Yichao Wu and Chuming Li and Xiaolin Hu and Ding Liang and Mengyu Wang},
  title = {{DAM: Discrepancy Alignment Metric for Face Recognition}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00379},
}
Figure