Face recognition systems are certified for fairness the way many new materials are certified for strength. Someone runs a short test on fresh samples and reports a single number. For materials, that test might be a 28-day compressive strength or a 15-day accelerated aging run. For face recognition, it is the spread of verification accuracy across racial groups on a static benchmark. Materials engineers have learned that early performance can be misleading, because degradation builds up slowly and nonlinearly. They therefore report retention, meaning the fraction of the original property that remains after exposure, and they check whether an accelerated test actually predicts what happens in the field. Face recognition has no equivalent practice. A deployed system matches a person's live face against a photo enrolled years earlier. If faces from different demographic groups drift away from their enrolled templates at different rates, then a model that is fair on the day it is certified can become unfair over its working life. Nobody would see this from current benchmarks.
We propose to measure demographic fairness as a function of the time elapsed between enrollment and probe, and to report it as a retention curve. We then ask two questions. First, does a model's fairness on a static benchmark predict its fairness at multi-year gaps? Second, can cheap "accelerated" tests forecast how each group degrades on real longitudinal data? Those cheap tests are synthetic age progression and simulated degradation of the camera sensor. Our main hypothesis is that the score a model gives to two photos of the same person declines at group-dependent rates as the gap grows. If so, existing bias-mitigation methods that were tuned and evaluated on static images would lose part of their fairness advantage over time. The second hypothesis is that whether a cheap test can stand in for real elapsed time is an empirical question that has never been checked. Our study tests it directly.
Where this sits in prior work
The bias-mitigation literature motivates the question, and it evaluates only at a single moment. Racial Faces in the Wild built the Racial Faces in the Wild (RFW) benchmark, with balanced Caucasian, Asian, Indian and African test subsets. It showed that African error rates are roughly double Caucasian ones, and it reduced the gap by treating races as domains for unsupervised adaptation. Skewness-aware reinforcement learning released the BUPT-Balancedface training set and learned per-group margins with a reinforcement-learning agent. This lowered the cross-race accuracy standard deviation on RFW from 1.11 to 0.93. Group Adaptive Classifier used demographic-conditioned convolution kernels and attention masks and brought the same standard deviation to 0.60. All three papers measure fairness on single photographs, with no elapsed time between enrollment and probe. Our proposal keeps their models and training data. It changes the evaluation axis to elapsed time.
The age-invariance literature supplies one method we reuse, and it has the opposite blind spot. Decorrelated Adversarial Learning splits a face embedding into an identity part and an age part, and uses an adversarial game to make the two uncorrelated. It reports 98.97% rank-1 identification on MORPH Album 2 and 99.4% verification on the Cross-Age Celebrity Dataset verification subset (CACD-VS). However, it treats MORPH as one undivided population and never checks whether the separation works equally well across groups. Longitudinal Face Aging in the Wild surveys generative aging models, which can be used to "pre-age" a gallery. It reports no fairness metrics. That gap is exactly what our accelerated-test validation addresses.
Two papers constrain the design. Review of Demographic Bias in Face Recognition argues that lower optical contrast on darker skin is a physical driver of bias. It also describes the "threshold trap": groups have shifted score distributions, so a single decision threshold produces unequal error rates. This suggests a concrete mechanism. Age-related texture changes are contrast phenomena, and they may be captured less reliably for some groups. The review also treats age only as a static age-group disparity, not as drift over time. Fairness Under Cover is the closest precedent for stress-testing fairness. It showed that synthetic masks and sunglasses raise demographic-parity bias by 60–300% and equal-opportunity bias by 57–275% on RFW. It also warned that ratio metrics such as the Skewed Error Ratio can appear to improve under stress simply because everyone's errors rise. We adopt that warning as a design rule. Their stress is instantaneous and applied to static images. Ours is elapsed biological and acquisition time, validated against real repeat photographs.
Core Claim
Fairness should be reported as a curve over elapsed time, not as a point. The informative quantity is each group's rate of degradation. For group $g$ and elapsed-time bin $t$, define retention as
$$R_g(t) = \frac{\mathrm{TAR}_g(t)}{\mathrm{TAR}_g(t_0)},$$
where $\mathrm{TAR}_g(t)$ is the true-accept rate for same-person pairs in group $g$ whose photos are $t$ years apart, $t_0$ is the shortest gap bin (under one year), and the operating point is a fixed global false-match rate. Unfairness that grows with time then shows up as a widening gap between groups' retention curves. We will also fit, per group, the slope of the mean same-person score against log elapsed time. The log form is an assumption chosen because aging changes are typically fastest early; we will compare it against a linear fit rather than impose it. Retention-curve fitting is our only borrowed kinetics, and we use it as a summary rather than as a physical law.
Research Program
We first build a longitudinal fairness benchmark from MORPH Album 2, which contains repeat photographs of the same people with capture dates. We bin same-person pairs by elapsed time and stratify them by the demographic labels that accompany the dataset.
We expect the usable comparison to be mainly between African-descent and White subjects, because that is where MORPH has depth. We state this limit up front rather than claim four-group coverage. FG-NET and CACD-VS are used only as secondary checks. FG-NET is small and childhood-dominated. CACD-VS lacks demographic labels, and we will not infer race automatically for it, because any classifier we used would itself be biased.
The main threat to validity is confounding: in MORPH, elapsed time is entangled with age at enrollment, sex, and changes in camera and image quality across the years of collection. We therefore match pairs on enrollment age, sex, acquisition year of the enrollment image, and an image-quality score. We report both matched and unmatched curves.
Second, we train comparable models under one recipe. All are ResNet-50 networks trained on BUPT-Balancedface with the same schedule and compute. The models are:
- An ArcFace baseline.
- A Group Adaptive Classifier.
- An ArcFace model with the decorrelation module of Decorrelated Adversarial Learning added, using an age-labeled auxiliary set. This tests whether explicitly removing age information from the identity embedding equalizes retention across groups.
- A combination of the group-adaptive kernels with age decorrelation.
The fourth model is the only new modeling component. We include it to test whether the two mechanisms complement each other or interfere.
Third, we build two accelerated proxies and test whether they are valid. The first applies a publicly available generative age-progression model to RFW enrollment images, aging them by fixed amounts. The second applies a schedule of blur, noise and contrast reduction to simulate sensor degradation, motivated by the reflectance argument in the review. Validity has a precise meaning here. A proxy is valid if, for every model, the per-group degradation it induces ranks groups and models in the same order as the real MORPH slopes, and if its predicted retention at matched severity falls within the real curves' bootstrap intervals.
Baselines
The naive reference point is to do nothing new. Under that reference, one reports each model's static RFW fairness and assumes it holds at all elapsed times. Under this assumption, the Group Adaptive Classifier is the fairest model, with a reported cross-race accuracy standard deviation of 0.60 against 1.11 for ArcFace (Group Adaptive Classifier). The reinforcement-learning margin method is next at 0.93, with a Skewed Error Ratio of 1.42 against 1.65 (Skewness-aware reinforcement learning). We retrain these models ourselves rather than reuse checkpoints, so that all models share the same data, schedule and tuning budget. We report our reproduced RFW numbers next to the published ones. "Beating" this reference is not about accuracy. It means showing that the static ranking of models by fairness fails to predict the ranking at gaps of five or more years, with non-overlapping bootstrap intervals.
The strongest existing method for the longitudinal half of the question is Decorrelated Adversarial Learning, at 98.97% rank-1 on MORPH Album 2. No reported result was found on Lacuna for per-race or per-elapsed-time performance of this or any model on MORPH. That missing number is the measurement our study supplies. We count the combined model as an improvement only if two conditions hold. First, it must reduce the between-group difference in retention at the longest gap bin compared with both the Group Adaptive Classifier alone and decorrelation alone. Second, its static RFW standard deviation must not exceed the Group Adaptive Classifier's by more than the bootstrap interval.
The key ablation separates threshold drift from representation loss. We compare two settings. In the first, each group's threshold is set once, on short-gap pairs. In the second, the threshold is re-estimated within each elapsed-time bin. If per-bin recalibration closes the widening gap, the disparity sits at the decision boundary and can be managed operationally. If a gap remains after recalibration, the embedding itself is losing identity signal faster for some groups.
Why not just…?
Why not just split the published MORPH results by race? This is the right first step, and it is the core of our first stage. We do not dress it up as something more. A naive split, however, mixes elapsed time with enrollment age and with camera changes across the years of collection. A widening gap could therefore simply reflect older images of one group being worse photographs. Our matched analysis is what turns that split into evidence. If the matched curves show no group-dependent decline, we report a clean null. That would itself be useful, because it would justify the field's static evaluation practice.
Why not just recalibrate thresholds per group? This is a real competitor, and the ablation above settles it. The review notes that per-group thresholds require demographic labels at inference time, which is its own source of error, so even a recalibration that works carries a cost. If per-bin recalibration does fully close the gap, the conclusion changes. The recommendation becomes a time-aware recalibration or re-enrollment schedule rather than new training objectives, and we would say so.
Why not just use synthetic aging as the stress test and skip real longitudinal data? Because no one has shown that generative aging models age every group realistically. Longitudinal Face Aging in the Wild reports no fairness evaluation of them. A proxy that ages underrepresented faces badly would produce made-up disparities. Validating the proxy against MORPH is therefore the point of our third stage, not an optional extra. If validation succeeds, the field gains a cheap accelerated test. If it fails, we have shown that synthetic stress tests of fairness can mislead.
Why not just collect a balanced longitudinal dataset? This would require repeat photographs of thousands of people across decades, balanced across groups. As the aging survey documents, existing longitudinal sets are small mugshot collections, and web data skews toward Western celebrities. We treat such collection as the eventual answer. Our curves on existing data can tell a future collection effort which gap lengths and groups need the most samples.
Decisive Checks
The study succeeds or fails on three measurements. The first is the matched MORPH retention curves. Either the between-group difference in true-accept rate at fixed false-match rate grows with elapsed time, or it does not. We report absolute gaps and equal-opportunity differences alongside ratio metrics, following Fairness Under Cover. The second is the rank correlation between models' static RFW fairness and their fairness at long gaps. A low or negative correlation would show that static certification is insufficient. The third is the agreement between each accelerated proxy's per-group degradation slopes and the real ones. Agreement would mean fairness durability can be estimated without waiting years. Disagreement would mean such proxies should not be trusted. The threshold-recalibration ablation determines which kind of fix any observed decline calls for.
References
- Racial Faces in the Wild: Reducing Racial Bias by Information Maximization Adaptation Network
- Mitigating Bias in Face Recognition Using Skewness-Aware Reinforcement Learning
- Mitigating Face Recognition Bias via Group Adaptive Classifier
- Decorrelated Adversarial Learning for Age-Invariant Face Recognition.
- Longitudinal Face Aging in the Wild - Recent Deep Learning Approaches
- Review of Demographic Bias in Face Recognition
- Fairness Under Cover: Evaluating the Impact of Occlusions on Demographic Bias in Facial Recognition