Canary Impostors: Tuning Self-Updating Biometric Galleries Without Labels

A face or fingerprint login system that never updates its stored reference samples, called templates, slowly starts rejecting its own users as they age, change lighting, or change how they press the sensor. Systems that do update, by adding samples from successful logins to each user's stored set (the gallery), adapt well until an impostor gets in once. After that, the impostor's sample stretches the user's profile toward the attacker and makes the next intrusion easier. This failure is called template poisoning. The best defenses keep a small, fixed number $p$ of templates per user, chosen near the center of that user's feature cluster. They work, but they depend on settings (the acceptance threshold, $p$, and the rule for choosing templates) that someone has to pick in advance. In deployment there are no labels saying which logins were genuine, so these settings cannot be validated.

We propose to tune those settings per user and without labels. The tuning signal comes from canary impostors: samples we know are not this user, because they belong to other enrolled identities. Some are pushed in embedding space toward the target user to mimic a gradual drift attack. We inject them into short replays of the user's own unlabeled login stream, measure how often each candidate setting lets a canary into the gallery, and pick the setting that adapts best while staying under an intrusion budget the operator states. Running many update policies over long login histories is costly, so we treat the choice the way efficient architecture search treats candidate networks. The question becomes whether a cheap, short replay ranks policies the same way the full long-term outcome does, and we measure that with rank correlation.

Core Claim

Cross-identity canaries are a better pseudo-anomaly than generic noise for selecting the hyperparameters of a self-updating gallery. Because they are injected per user, they also detect vulnerable users without any labels: the "Lambs" whose features are easy to imitate. Per-user policies chosen this way should cut impostor inclusion for Lambs well below the rates of uniform policies, while keeping the error-rate gains of updating.

The closest work is AutoUAD. It tunes unsupervised anomaly detectors by measuring how well a model separates real data from isotropic Gaussian pseudo-anomalies. It reports a mean AUC of about 84% for a one-class support vector machine across 38 tabular datasets, against about 78% with default settings. The authors themselves note that the Gaussian assumption breaks on complex high-dimensional data. Biometric impostors are exactly that case: they are other real faces or fingers lying close to the genuine manifold, not isotropic noise. Our method differs in three ways. It replaces the noise model with the structured impostors a biometric system already holds. It tunes a stateful update policy rather than a static detector. And it scores policies on what they do over time, since poisoning compounds.

Research Program

The candidate policies come from existing work, so the contribution is how we choose among them, not new selection rules. The fixed-size classification-selection approach supplies the main mechanism. It keeps at most $p$ templates per user, chosen either as the $p$ nearest the cluster centroid (K-Means) or as the $p$ with the smallest average pairwise distance (MDIST). It reports near 0% impostor inclusion on Multimodal-DIEE, FRGC, and LFW with $p=6$, and its authors list automatic choice of $p$ as future work. We add a third rule from Moderate Coreset, which keeps samples at the median distance from the class center. That paper shows this choice resists label noise in static dataset pruning. In a gallery it may keep more genuine variation than strict centroid rules without reaching the impostor fringe. The search space $\Theta$ is the product of selection rule, $p$, acceptance threshold, and update frequency.

For each user $u$ and candidate setting $\theta \in \Theta$, we replay the user's recent unlabeled sessions twice.

In the canary-acceptance replay, we interleave canaries. Some are raw enrollment samples from other users, simulating zero-effort attacks as in the zero-effort attack protocol. Others are those samples moved stepwise along the line toward the user's gallery centroid across replay sessions, as an embedding-space stand-in for drift attacks. We record $A_u(\theta)$, the fraction of canaries that end up in the gallery.

In the coverage replay, we measure $C_u(\theta)$: the median match score between the updated gallery and the trimmed core of the next unlabeled batch. The trimmed core is the points nearest that batch's own robust centroid. Trimming means a minority of real impostors in the stream barely moves the estimate.

The chosen setting solves

$$\theta_u^* = \arg\max_{\theta \in \Theta} C_u(\theta) \quad \text{subject to} \quad A_u(\theta) \le \alpha,$$

where $\alpha$ is the canary intrusion rate the operator is willing to tolerate. We use a constraint rather than a weighted sum because operators already think in terms of a security budget, and a weight would just be one more untunable hyperparameter. When no $\theta$ meets the budget, the user is flagged as Lamb-like and falls back to the most conservative policy, or to no updating. This yields a label-free vulnerability detector, which we check against Doddington's Zoo labels.

The architecture-search lens sets the cost question. Budgeted Performance Estimation showed that a cheap training setup is only useful if it ranks candidates the same way full training does, and that only some fidelity knobs matter. Its pruning step needs fully evaluated ground truth, which we do not have at deployment. We therefore run the ground-truth study offline, once, on labeled longitudinal datasets. That study tells us how short the replay horizon and how few canaries can be while still preserving ranking. Following EZNAS, we report both global rank correlation and top-decile regret, since that paper found proxies that rank well globally often fail among the best candidates. The top decile is the regime that matters when picking one policy.

Baselines

The naive reference is a static gallery that is never updated after enrollment. The dual-stage approach reports that a graph-based labeling and risk-based selection system reduces equal error rate (EER, the operating point where false accepts equal false rejects) by 67% relative to this static baseline on the DIEE fingerprint data.

The strongest existing method is the fixed-size K-Means or MDIST gallery of the classification-selection approach at its published default of $p=6$, applied uniformly to all users. That paper reports near 0% impostor inclusion but gives no dataset-specific EER numbers in our file. We also run the dual-stage graph and risk system, with its reported 4.1% residual impostor intrusion on DIEE, as the strongest adaptive competitor.

The closest unsupervised tuning competitor is AutoUAD's Gaussian pseudo-anomaly score, used in place of our canary term with the same search space and the same Bayesian optimizer.

An oracle tuned per user on true labels gives the upper bound.

For fairness, every method uses the same frozen feature extractors (FaceNet, ResNet50, SeNet50 for face; the original matchers for fingerprint and keystroke), the same login streams and impostor schedules, and the same number of policy evaluations per user. Beating the strongest baseline means two things. Lambs' impostor inclusion must fall clearly below the uniform-policy rate. Overall EER must be equal or better than K-Means at $p=6$, on at least three of the four dataset families.

Two ablations isolate the key ingredient. The first swaps canaries for Gaussian noise while keeping everything else, which tests whether structured impostors are what matters. The second drops the drifted canaries and keeps only raw cross-user ones, which tests whether simulating the walk-toward attack adds anything.

Why not just…?

Why not just freeze the templates? The evidence says this trades poisoning for steady decay. On DSL2009 keystroke data, the no-update system's error rose across sessions. Updating systems with permissive thresholds still beat it when impostors made up 80% of login attempts (zero-effort protocol). The static condition stays in every experiment as the floor.

Why not just use a conservative global threshold with K-Means at $p=6$? This is the real threat. The published near-0% inclusion suggests the uniform default may already be enough. Against it stands the zoo study (Doddington's Zoo analysis): with uniform update rules on Equinox, 43.2% of samples added to Lambs' galleries were impostors, while Goats got only 23.3% of their genuine variation accepted. A single setting therefore over-admits some users and starves others. If per-user canary tuning does not beat the uniform default on the Lamb and Goat subgroups, the honest conclusion is that centroid selection already absorbs the problem, and we will report that.

Why not just tune once on a labeled development population and ship those settings? This is also a serious alternative, because labeled longitudinal data exist for some modalities. Our answer is empirical. We tune on one dataset (for example FRGC) and deploy on another (for example LFW or Multimodal-DIEE), and compare against canary tuning done in place. If transferred settings match per-user canary tuning, our method's value shrinks to detecting Lambs, and we will say so.

Why not just use AutoUAD's Gaussian pseudo-anomalies? That paper concedes the Gaussian prior fails on complex manifolds. The direct test is the Gaussian-for-canary ablation, scored by rank correlation against long-horizon labeled outcomes.

Decisive Experiments

We use DIEE fingerprint, Equinox face, DSL2009 and GREYC keystroke, and Multimodal-DIEE, FRGC, and LFW face, all as described in the cited papers. For every user and every $\theta$ in a grid over rule, $p \in \{2,\dots,12\}$, threshold, and update frequency, we run the full labeled stream offline. This gives ground-truth long-horizon EER and true impostor inclusion.

The first check is proxy validity. Measured by Kendall tau and top-decile regret against those outcomes, canary-based rankings should clearly beat Gaussian-based rankings. No reported result exists on Lacuna for ranking biometric update policies. The only tau reference point is architecture search, where EZNAS-A reaches 0.65 on NASBench-201, and it is not directly comparable.

The second check is cost. We plot rank correlation against replay horizon and canary count. A good result is that a replay a small fraction of the full stream length keeps top-decile regret near the oracle's.

The third check is subgroups. We compute per-user impostor inclusion and acceptance of genuine variation for Lambs, Wolves, Goats, and Sheep on Equinox. Users whose canary budget cannot be met should overlap strongly with labeled Lambs, which is the label-free vulnerability claim.

The fourth check is adversarial. We run an embedding-space drift attacker that was not used to generate canaries, with a different step schedule and different source identities, to test whether tuning against canaries generalizes or simply overfits to its own simulated attacker.

The proposal fails if any of the following holds: canaries rank no better than Gaussian noise, uniform K-Means at $p=6$ matches per-user tuning on Lambs, or the held-out attacker defeats tuned galleries as easily as default ones.