Shortcut-Probe Gating for Test-Time Adaptation of Speech Deepfake Detectors

A speech deepfake detector trained on one set of attacks and recording conditions degrades badly once deployed. On the crowdsourced ASVspoof 5 data, the standard detector's pooled error rises from 17.83% on development to 29.12% on evaluation, and one legacy attack reaches 78.9% ASVspoof 5. A natural remedy is test-time adaptation: adjust the deployed model on the unlabeled audio it receives, with no access to the original training data. The usual safeguard is to adapt only on inputs the model is confident about and to abstain on the rest. In anti-spoofing, that safeguard fails exactly where it matters most. The most dangerous errors are fakes the detector confidently calls real. A confidence gate lets those errors through, and self-training then teaches the model that the attack is genuine speech.

We propose to gate both adaptation and abstention with a signal that does not come from the model's own confidence. The signal is whether the prediction survives label-preserving nuisance edits: re-encoding through a held-out codec, trimming leading and trailing silence, and changing gain. These edits do not alter whether an utterance is real or fake. A prediction that flips under them is relying on the kind of shortcut the anti-spoofing literature keeps finding, such as trailing-silence patterns Ensemble Models for Spoofing Detection or the fingerprint of one particular vocoder CRIM system, ASVspoof 2015. Such a prediction should neither become a pseudo-label nor be emitted to the user. Our central claim is testable: per unit of coverage, this "shortcut-probe" gate admits far fewer confident spoof-accepted-as-real pseudo-labels than confidence or evidential-uncertainty gates. That reduction in leakage is what separates adaptation that helps from adaptation that collapses.

Core Claim

Selective prediction splits a classifier into a predictor and a selection function that decides whether to answer or abstain. Its quality is measured by risk (error among answered inputs) against coverage (fraction answered) Selective Classification for Deep Neural Networks. Pseudo-labeling is the self-training loop in which a model's confident outputs on unlabeled data become its training targets. Confirmation bias is the failure mode in which confident errors are reinforced. The cleanest statement of why selection matters in source-free adaptation comes from the bias-variance analysis of Shen et al. and its companion. They show that label-mismatch bias grows without bound if noisy pseudo-labels are admitted indiscriminately. They also show a 10–13 point gap between their practical confidence-plus-prototype selection rule and an oracle selector: 77.1% versus 90.0% on Office-Home, and 48.4% versus 58.1% on DomainNet. This work motivates ours by showing that the gate is the bottleneck.

Our claim is that in anti-spoofing this gap is structural. Any gate built on the adapting model's own confidence, including Dirichlet-based evidential uncertainty, inherits the model's blind spots. FADEL FADEL shows that evidential uncertainty correlates with error averaged across unseen attacks. Correlation on average does not mean low leakage in the tail. That tail consists of the confidently wrong A10-style or A12-style samples, and it is the tail that poisons self-training. A gate that tests prediction invariance under known-nuisance edits measures something the model cannot fake by being confident. It asks whether the decision depends on cues that should be irrelevant.

Research Program

The base system is AASIST AASIST, a raw-waveform detector built on graph attention over spectral and temporal features. We train it on ASVspoof 2019 LA, where it reaches 0.83% EER. At deployment, a stream of unlabeled target audio arrives in mini-batches.

For each utterance $x$ we compute $K$ edited versions $\tilde{x}_1,\dots,\tilde{x}_K$ and define the probe score as

$$s(x) = \min_{k} \; \mathbb{1}[\hat{y}(\tilde{x}_k) = \hat{y}(x)] \cdot \big(1 - |p(\tilde{x}_k) - p(x)|\big),$$

where $\hat{y}$ is the predicted label and $p$ the predicted spoof probability. The score is high only if every edit leaves the decision and its probability nearly unchanged.

The selection function is $g(x)=\mathbb{1}[s(x)\ge\tau]$. The same gate does two jobs. Inputs with $g(x)=1$ are answered, and those that also agree with a frozen copy of the source model become pseudo-labels for updating only the normalization-layer affine parameters, as in Tent Tent. Inputs with $g(x)=0$ are deferred and excluded from all adaptation losses.

We will also test a class-asymmetric variant. Because the costly error is accepting a spoof as real, it requires a stricter threshold before admitting a "bona fide" pseudo-label than a "spoof" pseudo-label.

One design rule is essential for honesty. The codecs used inside the probe must be disjoint from the codecs present in the target data. ASVspoof 5 lists eleven codec conditions, including Encodec, AMR, G.711, MP3 and Opus. We will partition them so that the probe never uses a codec that appears in the target data, and we will report results under several partitions. Otherwise the probe would simply be data augmentation toward the test condition.

The analysis instrument we will build alongside the method is gate leakage. It is the fraction of admitted pseudo-labels that are spoofs labeled bona fide, broken down per attack and tracked over the adaptation stream. Ground-truth labels are used only for this offline measurement, never by the method.

Baselines

The naive reference is the unadapted AASIST trained on ASVspoof 2019 LA and evaluated frozen on each target stream with softmax-response abstention. Softmax response uses the maximum class probability as the confidence score; Geifman and El-Yaniv found it competitive on clean data. This baseline tells us whether adapting at all is worth the risk. The ASVspoof 5 organizers report a pooled evaluation EER of 29.12% for their AASIST baseline ASVspoof 5. That model was trained on ASVspoof 5's own training data, so it is a reference point rather than a matched comparison, and we will rerun our own source model.

The strongest static competitor is AASIST-FADEL, the evidential head of FADEL, which reports 5.60% EER when trained on ASVspoof 2019 LA and tested on ASVspoof 2021 LA, against 6.10% for ASAM. Beating it on 2021 LA means a lower EER on the same protocol. In the selective setting, it means a lower spoof-acceptance rate at matched coverage.

The adaptive baselines run on identical streams, batch sizes, and updated parameter sets, with the same hyperparameter search budget: twenty configurations each, tuned on a held-out target-like split that is never reused for reporting. The first is label-free normalization re-estimation, UBNA UBNA. It replaces batch-norm running statistics with target batch statistics and uses no labels, so it has no confirmation loop. The second is Tent Tent. The third is fixed-threshold pseudo-label self-training. The fourth is a Shen-style confidence-plus-prototype selector Shen et al.. The fifth is a port to audio of SDA-FAS source-anchored self-training SDA-FAS, which reports 3.08% HTER versus 17.96% for SHOT on face protocol O&M&I→C. Porting it to audio is our own reimplementation.

The most important competitor is the obvious combination: FADEL-gated Tent and self-training, where evidential uncertainty replaces our probe score and everything else is identical. The ablations that isolate the key ingredient swap only the gate signal. We compare the probe score with softmax response, evidential uncertainty, frozen-versus-adapted disagreement alone, and the probe with in-distribution codecs (the leaky version). We also remove the class asymmetry.

The proposal beats the field if three things hold on ASVspoof 5. It must achieve lower area under the risk-coverage curve than FADEL-gated adaptation and than UBNA plus softmax response. It must have lower gate leakage on the worst-case attacks. And it must not show the late-stream EER increase that signals collapse.

Why not just…?

Why not just threshold softmax confidence, or FADEL's uncertainty, and skip the probe? This is the real competitor, and the evidence against it is suggestive rather than conclusive. Zara et al. report that confidence calibration breaks under domain shift. The 78.9% EER on A12 ASVspoof 5 implies many confident wrong answers. FADEL's correlation evidence is aggregate, across attacks A07–A19. The gate-swap ablation settles the question directly. If evidential gating matches the probe on leakage and risk-coverage, the probe is unnecessary, and we will report that.

Why not just re-estimate normalization statistics, which uses no pseudo-labels and so cannot suffer confirmation bias? This is a serious threat. Much of the ASVspoof 5 degradation may be codec and channel shift that UBNA alone corrects. UBNA cannot fix shortcut reliance on attack cues, however, and it offers no abstention. We therefore pair it with every gate. If UBNA plus a good gate matches the full method, the honest conclusion is that the probe's value lies in selection, not in steering adaptation. The experiment distinguishes the two by reporting adaptation-on and adaptation-off variants for each gate.

Why not just use Tent? Tent minimizes entropy on all inputs. Its own authors note that it assumes entropy tracks error Tent. Low-entropy spoof-as-real predictions are exactly where that assumption fails. We expect Tent to improve 2021 LA, where the shift is mostly channel shift, and to degrade on ASVspoof 5 attacks over the course of the stream. The per-attack trajectory plots will show whether this happens.

Why not just train a better domain-generalized detector and keep it frozen? Frozen generalization still leaves large gaps. In face anti-spoofing, adaptation improved HTER by 19.17% relative over domain-generalization methods OTA. The probe gate also applies to any frozen model as a pure abstention rule. We will report that frozen-plus-probe configuration, so a reader can see how much of the gain comes without any weight updates.

Proposed Experiments

The source model is AASIST trained on ASVspoof 2019 LA. The targets are ASVspoof 2021 LA, which tests channel shift, and the ASVspoof 5 evaluation set, which tests codecs, unseen attacks, and the adversarially filtered A09. Streams are presented in random and attack-sorted orders. The attack-sorted order stresses confirmation bias because a long run of a single unseen attack arrives together.

We report four groups of metrics: EER and min t-DCF on all inputs; risk-coverage curves with their area; spoof-acceptance rate at 80% and 90% coverage; and gate leakage per attack over time. We will also check whether a threshold calibrated with the guaranteed-risk procedure of Geifman and El-Yaniv on source development data still meets its target risk after adaptation. We expect it will not, and quantifying the violation for each gate is itself a useful result for online risk control. As a secondary generality check, we will run the same gate on face protocol O&M&I→C with label-preserving image edits (JPEG re-compression, resizing) against SDA-FAS's 3.08% HTER.

The decisive check is a single comparison: gate leakage on A12 and A10-like attacks at matched coverage, probe versus evidential uncertainty, using held-out probe codecs. If the probe does not cut leakage substantially, the core claim is false regardless of aggregate EER. If it does but downstream EER does not improve, the bottleneck lies in the adaptation objective rather than the gate, and the work becomes a selection result rather than an adaptation result.