Abstract
Fair biometric algorithms have similar verification performance across different demographic groups given a single decision threshold. Unfortunately, for state-of-the-art face recognition networks, score distributions differ between demographics. Contrary to work that tries to align those distributions by extra training or fine-tuning, we solely focus on score post-processing methods. As proved, well-known sample-centered score normalization techniques, Z-norm and T-norm, do not improve fairness for high-security operating points. Thus, we extend the standard Z/T-norm to integrate demographic information in normalization. Additionally, we investigate several possibilities to incorporate cohort similarities for both genuine and impostor pairs per demographic to improve fairness across different operating points. We run experiments on two datasets with different demographics (gender and ethnicity) and show that our techniques generally improve the overall fairness of five state-of-the-art pre-trained face recognition networks, without downgrading verification performance. We also indicate that an equal contribution of False Match Rate (FMR) and False Non-Match Rate (FNMR) in fairness evaluation is required for the highest gains.
Introduction
Modern face recognition (FR) systems are frequently biased due to imbalances in training data, which often skew toward specific demographics. This leads to the "Other Race Effect," where systems exhibit higher False Match Rates (FMR) or False Non-Match Rates (FNMR) for underrepresented groups. While traditional fixes involve computationally expensive retraining or fine-tuning, this paper demonstrates that demographic bias can be significantly mitigated through score normalization—a post-processing step that requires no changes to the underlying AI models.
Methodology: Demographic-Aware Normalization
The core idea is to adjust the raw similarity score between two faces based on how that score compares to a "cohort" (a reference set of unrelated faces). By aligning the score distributions of different demographic groups, the system can apply a single decision threshold more fairly.
The researchers explored several variations of this approach:
- Identity-based (Z-norm/T-norm): Standard techniques that normalize scores based on an individual's typical similarity to a general cohort.
- Demographics-based (M1.1 - M2.2): Extensions that restrict the cohort to the same demographic group as the subject being verified.
- Pure Cohort-based (M3): Uses the statistics of all impostor scores within a specific demographic group to shift the distribution.
- Genuine and Impostor Cohorts (M4 & M5): Incorporates both impostor and "genuine" (same-person) scores to better align both ends of the distribution using techniques like Platt Scaling or Bimodal-CDF.
Experimental Results
The methods were tested on five state-of-the-art pre-trained networks (including ArcFace and MagFace) using the VGGFace2 and Racial Faces in the Wild (RFW) benchmarks.
Key Findings:
- Significant Fairness Gains: Demographic-based normalization (M1.1, M1.2, and M3) consistently reduced bias. For example, on the VGGFace2 ethnicity task, the Worst-case Error Rate (WERM) dropped from 3.81 to 1.52 for the E1 network.
- Performance Stability: Unlike training-based interventions, these post-processing techniques rarely downgraded accuracy. The True Match Rate (TMR) remained stable or improved slightly across most experiments.
- Efficiency of M3: The pure cohort-based method (M3) proved highly effective and computationally efficient, as its statistics can be pre-computed during the enrollment phase.
Discussion and Conclusion
This research highlights that post-processing is a powerful, model-agnostic tool for improving fairness. A critical insight is that normalizing based on demographic groups is more effective than individual identity-based normalization. Furthermore, the authors caution that "balancing" datasets through sub-sampling can actually worsen fairness by reducing the data available to model minority distributions.
Future work will focus on refining these models to better capture the "tails" of score distributions, where critical errors occur, and adapting the WERM metric for varied real-world security scenarios.
Summary updated