QMagFace: Simple and Accurate Quality-Aware Face Recognition

Philipp Terhörst, Malte Ihlefeld, Marco Huber, Naser Damer, Florian Kirchbuchner, Kiran Raja, Arjan KuijperWACV 2023

TL;DR

This paper introduces QMagFace, a novel face recognition system that incorporates face image quality directly into the comparison score. By leveraging a magnitude-aware angular margin loss (MagFace) and a simple, linear quality-weighting function, QMagFace significantly improves recognition accuracy, especially in challenging unconstrained scenarios like varying poses, ages, and image qualities.

What Problem Does This Paper Address?

Face recognition systems are increasingly deployed in real-world scenarios, which often involve unconstrained environments. These environments present significant challenges due to variations in illumination, pose, expression, and crucially, image quality. While image quality is a known determinant of recognition performance, existing methods either ignore it during comparison or use quality estimates that aren't well-suited for the task. This leads to a loss of valuable information or inaccurate adjustments, particularly when dealing with difficult conditions. The paper aims to bridge this gap by developing a method that effectively integrates model-specific face image quality into the comparison process to enhance robustness and accuracy.

Methods

The core of QMagFace lies in its two main components:

  1. Quality-Aware Comparison Scoring: Instead of relying solely on the standard cosine similarity between face embeddings, QMagFace modifies the comparison score by incorporating the "quality" of the face images. This quality is derived from the magnitude of the face embeddings themselves, a concept introduced by the MagFace loss. The proposed quality-aware score $\hat{s}$ is a linear combination of the standard score $s$ and the minimum of the two image qualities ($q_1, q_2$), weighted by a learned function $w(s)$: $\hat{s} = w(s) \cdot \min\{q_1, q_2\} + s$ The quality-weighting function $w(s)$ is designed to be active only for lower scores, effectively down-weighting comparisons involving low-quality images.

  2. MagFace Loss: The system is built upon a face recognition model trained with the MagFace loss. This loss function is designed to incorporate magnitude-aware angular margins, which inherently link the embedding magnitude (and thus, face quality) to the recognition process. This provides a natural source of model-specific quality information.

The quality-weighting function $w(s)$ is learned through a three-step process:

  • Determining optimal weights for various decision thresholds.
  • Defining a relevant threshold range based on desired False Match Rates (FMRs).
  • Learning the parameters of a linear quality-weighting function ($w(s) = \beta \cdot s - \alpha$) by minimizing the difference between the learned weights and the optimal weights.

Datasets

The paper utilizes several datasets for training and evaluation:

  • Training: Adience dataset was used for training the quality-weighting function due to its wide range of image qualities.
  • Evaluation Benchmarks:
    • Single-Image: LFW (Labeled Faces in the Wild), AgeDB-30, CFP-FP (Cross-Pose Face Verification), XQLFW (Cross-Quality LFW).
  • Video-Based: IJB-B (IARPA Janus Benchmark-B), IJB-C (IARPA Janus Benchmark-C).
  • Additional Datasets for Analysis: Morph, ColorFeret.

Key Insights and Ideas

  • Quality is King (Especially for Low Quality): The paper strongly emphasizes that incorporating face image quality is most beneficial when dealing with low-quality images. The QMagFace approach effectively adjusts scores to mitigate the negative impact of poor quality.
  • Linearity is Sufficient: A key finding is that a simple linear relationship between face image quality and comparison scores is sufficient for effective weighting. This simplicity makes the approach generalizable and computationally efficient.
  • Model-Specific Quality Matters: The MagFace loss provides embeddings whose magnitudes directly reflect the quality as perceived by the specific recognition model. This model-specific quality is more valuable than generic quality metrics.
  • Generalizability: The learned quality-weighting function demonstrates high generalizability across different MagFace models and even across different databases, suggesting a robust underlying principle.

Key Results

QMagFace consistently improves face recognition performance across various benchmarks and FMR ranges.

  • State-of-the-Art Performance: QMagFace achieves state-of-the-art results on challenging benchmarks, particularly in:
  • Cross-age recognition (AgeDB): 98.50%
    • Cross-pose recognition (CFP-FP): 98.74%
    • Cross-quality recognition (XQLFW): 83.95%
  • Consistent Improvement: Across multiple databases and FMR levels, QMagFace variants (especially those based on iResNet-100) consistently outperform their MagFace counterparts, often by significant margins. For instance, on the Morph database at an FMR of $10^{-5}$, QMagFace-100 achieves 99.88% accuracy compared to MagFace-100's 99.58%.
  • Robustness: The quality-weighting function shows robustness, with learned parameters being similar across different training databases.

Limitations and Future Work

  • Diminishing Returns at Very Low FMRs: The performance gains from QMagFace are less pronounced at extremely low FMRs (e.g., $10^{-7}$), as the benefit of quality information is most significant for lower-quality images.
  • Video-Based Quality Aggregation: The paper notes that aggregating quality information for video embeddings (using weighted sums) does not perfectly reflect the combined embedding quality. Future work should focus on more advanced quality-based fusion techniques for video frames.
  • Bias in Quality Estimates: The authors highlight the ethical consideration that biased quality estimates can lead to unfair matching decisions. Ensuring unbiased quality assessment is crucial for equitable face recognition systems.

Summary updated

BibTeX
@inproceedings{terhorst2023qmagface,
  author = {Philipp Terhörst and Malte Ihlefeld and Marco Huber and Naser Damer and Florian Kirchbuchner and Kiran Raja and Arjan Kuijper},
  title = {{QMagFace: Simple and Accurate Quality-Aware Face Recognition}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
}
Figure