Troubleshooting Ethnic Quality Bias with Curriculum Domain Adaptation for Face Image Quality Assessment

Fu-Zhao Ou, Baoliang Chen, Chongyi Li, Shiqi Wang, Sam Kwong

Introduction

Face Image Quality Assessment (FIQA) serves as a critical gatekeeper in biometric systems, determining whether an input image possesses sufficient clarity, lighting, and pose for accurate processing. However, current FIQA models suffer from a significant ethnic quality bias. Because primary training datasets like CASIA-Webface and MS-Celeb-1M are predominantly composed of Caucasian faces (often exceeding 74%), models frequently fail to generalize to other ethnicities. This imbalance leads to non-Caucasian faces receiving lower quality scores regardless of actual image clarity, creating a bottleneck for fairness in global face recognition systems.

Two primary technical hurdles complicate the mitigation of this bias. First, FIQA is typically framed as a regression task, which is notoriously difficult to transfer between domains due to shifts in feature scales. Second, the lack of objective "ground truth" labels for face quality makes it challenging to ensure a model isn't simply learning the noise or specific quirks of the source domain.

Methodology: The EQBM Framework

To address these challenges, the authors propose the Ethnic-Quality-Bias Mitigating (EQBM) framework. This approach shifts the paradigm from simple scalar regression to a distribution-based, curriculum-driven domain adaptation strategy.

The EQBM framework architecture, illustrating the four-stage process from annotation mapping to ranking-based adversarial training.

From Scalars to Likert Scales

Instead of predicting a single quality score, EQBM maps quality into a Likert-scale probability distribution (e.g., probabilities for levels like "poor," "fair," and "excellent"). This soft-mapping stabilizes the learning process and bypasses the feature scaling issues inherent in traditional regression-based domain adaptation, providing a more robust foundation for cross-ethnic transfer.

Curriculum Domain Adaptation

Recognizing that not all images are equally informative, the framework employs an "easy-to-hard" training scheduler. Samples are prioritized based on two metrics:

  1. Inter-domain Uncertainty: Identifying samples that look similar across races in the feature space.
  2. Intra-domain Quality Margin: Selecting pairs where the quality difference is clear and unambiguous. By starting with "easy" pairs, the model builds a stable foundation before tackling complex cases where ethnic features might otherwise distract from quality assessment.

Ranking-based Domain Adversarial Network

To overcome the absence of ground-truth labels, the model utilizes a pairwise ranking loss. Rather than guessing an absolute score, the network learns to identify which of two images is of higher quality. This is integrated with a Gradient Reversal Layer (GRL) to ensure that the learned features are "race-blind" while remaining highly "quality-aware."

Experimental Results

The EQBM framework was validated using the Racial Faces in the Wild (RFW) benchmark, which contains 10,000 images for each of four ethnic groups: Caucasian, Indian, Asian, and African. Training was conducted on the BUPT-Transferface dataset, using Caucasian images as the source and unlabeled images of other races as the target domain.

Performance comparison across different face recognition backbones (FaceNet, ArcFace, GAC) and ethnic groups, showing FNMR vs. ratio of unconsidered images.

Using the Area Over Curve (AOC) metric, the EQBM-S variant demonstrated substantial improvements over state-of-the-art (SOTA) methods like FaceQAN and SDD-FIQA:

  • African Group: 13.2% improvement over the best competitor.
  • Indian Group: 41.4% improvement in accuracy.
  • Asian Group: 21.9% improvement.

Crucially, the model maintained high performance on general benchmarks like IJB-C (AOC of 0.632), proving that mitigating ethnic bias does not necessitate a trade-off in overall system performance.

Conclusion and Future Work

The EQBM framework successfully addresses ethnic bias in FIQA by treating quality as a distribution and employing a curriculum-based ranking strategy. While this represents a significant step toward fairer biometrics, the authors note that gender and age bias remain open challenges. Future research will explore extending this curriculum approach to a multi-attribute framework that simultaneously addresses race, gender, and age to ensure universal reliability in face recognition technology.

Summary updated

Abstract

Face Image Quality Assessment (FIQA) lays the foundation for ensuring the stability and accuracy of face recognition systems. However, existing FIQA methods mainly formulate quality relationships within the training set to yield quality scores, ignoring the generalization problem caused by ethnic quality bias between the training and test sets. Domain adaptation presents a potential solution to mitigate the bias, but if FIQA is treated essentially as a regression task, it will be limited by the challenge of feature scaling in transfer learning. Additionally, how to guarantee source risk is also an issue due to the lack of ground-truth labels of the source domain for FIQA. This paper presents the first attempt in the field of FIQA to address these challenges with a novel Ethnic-Quality-Bias Mitigating (EQBM) framework. Specifically, to eliminate the restriction of scalar regression, we first compute the Likert-scale quality probability distributions as source domain annotations. Furthermore, we design an easy-to-hard training scheduler based on the inter-domain uncertainty and intra-domain quality margin as well as the ranking-based domain adversarial network to enhance the effectiveness of transfer learning and further reduce the source risk in domain adaptation. Extensive experiments demonstrate that the EQBM significantly mitigates the quality bias and improves the generalization capability of FIQA across races on different datasets.
BibTeX
@inproceedings{ou2023troubleshooting,
  author = {Fu-Zhao Ou and Baoliang Chen and Chongyi Li and Shiqi Wang and Sam Kwong},
  title = {{Troubleshooting Ethnic Quality Bias with Curriculum Domain Adaptation for Face Image Quality Assessment}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.01894},
}
Figure