Abstract: In this paper, we investigate the problem of prediction confidence in face and kinship verification. Most existing face and kinship verification methods focus on accuracy performance while ignoring confidence estimation for their prediction results. However, confidence estimation is essential for modeling reliability and trustworthiness in such high-risk tasks. To address this, we introduce an effective confidence measure that allows verification models to convert a similarity score into a confidence score for any given face pair. We further propose a confidence-calibrated approach, termed Angular Scaling Calibration (ASC). ASC is easy to implement and can be readily applied to existing verification models without model modifications, yielding accuracy-preserving and confidence-calibrated probabilistic verification models. In addition, we introduce the uncertainty in the calibrated confidence to boost the reliability and trustworthiness of the verification models in the presence of noisy data. To the best of our knowledge, our work presents the first comprehensive confidence-calibrated solution for modern face and kinship verification tasks. We conduct extensive experiments on four widely used face and kinship verification datasets, and the results demonstrate the effectiveness of our proposed approach.
Introduction
In high-stakes biometric applications such as forensic kinship analysis or secure access control, the reliability of an AI's prediction is as critical as its accuracy. While modern face verification models excel at determining whether two images represent the same person or a biological relative, they typically output raw similarity scores that do not reflect true probabilistic confidence. This lack of calibration means a model might assign a high similarity score to a pair it incorrectly identifies, leading to potential failures in real-world trust. This paper addresses this gap by introducing a framework to estimate and calibrate confidence while accounting for data noise.
The Proposed Framework
The researchers propose a "plug-and-play" approach that can be applied to existing verification models without retraining. The framework consists of three main components: a new confidence measure, Angular Scaling Calibration (ASC), and an uncertainty integration module.
- Confidence Measure: A mapping function converts the distance between a similarity score ($s$) and a decision threshold ($\tau$) into a probabilistic confidence score ($c$).
- Angular Scaling Calibration (ASC): Since state-of-the-art models like ArcFace operate in angular space, ASC learns a linear transformation of the angle between face features using a small validation set. This post-processing step fixes miscalibration while preserving the original model's accuracy.
- Uncertainty Integration: By utilizing Probabilistic Face Embeddings (PFE), the model represents faces as distributions rather than single points. This allows the system to penalize confidence when image quality is low (e.g., due to blur or poor lighting), providing a more "honest" assessment.
Experimental Results
The approach was evaluated on major benchmarks including Families In the Wild (FIW), KinFaceW, LFW, and IJB-C. A key finding was the "Deeper is Less Calibrated" paradox: while deeper models like ResNet101 are more accurate, they are often more overconfident and poorly calibrated compared to shallower architectures like VGG16.
The primary metric, Expected Calibration Error (ECE), saw dramatic improvements. On the FIW dataset, ASC reduced the ECE of a ResNet101 model from 22.85% to just 2.08%. As shown in the reliability diagrams, the calibrated models (right) align much more closely with the ideal diagonal, where predicted confidence matches actual accuracy. Furthermore, because ASC is a monotonic transformation, it maintains the model's original verification performance while providing trustworthy scores.
Discussion and Conclusion
By performing calibration in the angular space where features are learned, the authors successfully bridge the gap between high-performance deep learning and reliable probabilistic output. The study also demonstrated that after calibration, decision thresholds across different models tend to converge, suggesting a more equitable distribution of similarity scores.
While the framework is highly effective, future work remains in improving cross-dataset generalization and reducing the minor computational overhead introduced by the uncertainty estimation branch. Nonetheless, this work provides a robust foundation for deploying more transparent and trustworthy AI in sensitive verification tasks.
Summary updated