TL;DR
This paper reveals that Face Quality Assessment (FQA) algorithms—tools designed to filter out "bad" images to improve recognition—unintentionally inherit and amplify the demographic biases of the face recognition (FR) systems they support. By analyzing state-of-the-art FQA methods, the authors demonstrate that subgroups already marginalized by FR systems (such as specific ethnicities and age groups) are systematically assigned lower quality scores, potentially leading to discriminatory outcomes during enrollment and authentication.
What Problem Does This Paper Address?
Modern face recognition systems are highly accurate but notoriously biased, often performing significantly worse on certain demographic groups (e.g., non-white ethnicities, young children) or under specific conditions like profile poses. To boost performance, researchers use FQA to estimate an image's "utility"—how useful it is for a specific recognition model.
The prevailing wisdom is that FQA should be "model-specific" to be most effective. However, the authors identify a critical gap: if an FQA algorithm is tailored to a biased FR model, does it also become biased? This "bias transfer" could lead to a "double whammy" for marginalized groups: not only is the system less likely to recognize them correctly, but it might also reject their images entirely during enrollment, labeling them as "low quality" simply because the underlying model struggles to process their features.
Methods
The researchers conducted a comprehensive audit of the relationship between FR bias and FQA bias. They evaluated four prominent FQA solutions:
- COTS: A commercial off-the-shelf industry product.
- Best-Rowden: An SVM-based approach trained on comparison scores.
- FaceQnet: A deep learning regressor trained to predict quality labels.
- SER-FIQ: A recent unsupervised method that measures quality based on the "robustness" of embeddings under stochastic variations (dropout).
These were tested against two widely used FR models, FaceNet and ArcFace, across two datasets representing controlled and uncontrolled environments. The primary evaluation tool was the Error-versus-Reject Curve (ERC), which measures how much the verification error drops as the lowest-quality images are discarded.
Datasets
- ColorFERET: A controlled dataset of 14,000 images from 1,200 individuals, featuring labels for pose, ethnicity, and age.
- Adience: An uncontrolled, "in-the-wild" dataset of 26,500 images from 2,200 individuals, primarily used for age and gender analysis.
- Training Data: The FQA models were variously trained or evaluated using MORPH, VGGFace2, and MS1M to ensure a broad demographic spread.
Key Insights and Ideas
- Quality is Utility, Not Just Aesthetics: The paper emphasizes that "quality" in biometrics isn't about how pretty a picture is; it's about how well a specific algorithm can use it. If a model is "blind" to certain features, it perceives images with those features as low utility.
- The Bias Transfer Effect: The most effective FQA methods (like SER-FIQ) are the ones most tightly coupled to the FR model. Ironically, this tight coupling makes them the most prone to bias transfer.
- Embeddings Over Training Data: The authors found that the bias in FQA originates more from the underlying FR embeddings than from the specific data used to train the FQA regressor itself.
- Ethical Implications: Biased FQA creates systemic barriers. For example, in automated border control, a biased FQA might constantly flag a specific demographic for manual inspection, not because their photos are blurry, but because the system is poorly trained on their demographic.
Key Results
The study confirmed significant biases in the underlying FR models. For instance, FaceNet showed a staggering 80.02% False Non-Match Rate (FNMR) for children aged 0-2, compared to much lower rates for adults. Profile poses resulted in error rates 10 times higher than frontal views.
When applying FQA thresholds, the bias transfer became clear:
- Subgroup Attrition: As the quality threshold increases, the proportion of "biased" subgroups (e.g., Black individuals, young children) in the remaining pool drops sharply. This indicates the FQA is disproportionately labeling these groups as "low quality."
- Performance vs. Fairness: While SER-FIQ provided the best quality estimation performance (see Figure 1), it also exhibited the clearest separation in quality distributions between biased and unbiased groups.
Limitations and Future Work
The authors acknowledge that the current definition of biometric quality—utility for recognition—is fundamentally at odds with demographic fairness if the recognition models themselves are biased.
Future directions include: * Developing FQA solutions that are "demographic-blind" or specifically designed to resist bias transfer. * Prioritizing the development of unbiased FR models, which would naturally lead to more equitable model-specific FQA. * Re-evaluating the ethical frameworks and standards (like ISO/IEC) that define biometric quality to include fairness as a core metric.
Summary updated