A detector for spoofed speech, meaning audio produced by speech synthesis or voice conversion to fool a speaker-verification system, is only useful at a fixed operating point. Someone has to pick a score threshold and trust that it flags roughly the intended fraction of genuine callers. That trust breaks when the transmission channel changes. A threshold tuned on clean studio recordings can wrongly reject many genuine callers once audio passes through a phone line or an MP3 codec, or it can let most spoofs through. The field mostly reports the equal error rate (EER, the error rate at the threshold where false alarms equal misses, chosen after seeing the test labels). That number hides the problem, because no deployed system gets to choose its threshold using test labels.
We propose to make the threshold itself depend on the channel. A small gating network splits incoming utterances among a few scoring sub-networks, called experts. It makes this choice from the utterance's i-vector, a compact fixed-length summary of a recording that deliberately keeps channel and microphone information instead of removing it. Each expert keeps its own estimate of the extreme upper tail of genuine-speech scores, using extreme value theory. The final decision is made on a combined tail probability, so the false-alarm rate on genuine speech should stay near a user-set target $q$ within each channel condition, not only on average. The question is whether conditioning calibration on a learned partition of channel space keeps realized false-alarm rates on target under the codec and channel shifts of ASVspoof 2021, where fixed thresholds and global adaptive thresholds fail.
Core Claim
The claim is about calibration, not ranking. Mixture-of-experts anomaly detectors such as ADMoE route inputs to specialized experts and improve ROC-AUC by about 14% on average across eight benchmarks. They stop at a continuous score and never choose an operating point. Streaming thresholders such as SPOT and DSPOT (Anomaly Detection in Streams with Extreme Value Theory) do choose one with a stated risk level. They fit a Generalized Pareto Distribution, a standard model for values beyond a high threshold, to the top of the score distribution. However, they fit one univariate tail over the whole stream. When the stream mixes channel conditions whose genuine-score tails differ, a single tail is right on average and wrong in every individual condition.
Our proposal fits one tail per routed expert and mixes the tails with the gate's weights. For an utterance $x$, let $g_k(x)$ be the gate weight on expert $k$ and $s_k(x)$ that expert's spoof score. Let $\hat P_k$ be expert $k$'s fitted tail probability for genuine speech. The utterance is flagged when
$$p(x) = \sum_{k=1}^{K} g_k(x)\,\hat P_k\big(S > s_k(x)\big) < q .$$
If the gate sends genuine utterances from a given channel to experts whose tails were fit on similar genuine audio, then $\Pr[p(x)<q]\approx q$ within that channel. This holds as long as each $\hat P_k$ is accurate below $q$, which is exactly the regime extreme value theory covers.
The contribution is therefore the coupling between how inputs are routed and where calibration happens. We are not claiming better EER. We are not claiming better i-vectors.
Why i-vectors, and why routing rather than invariance
The i-vector model writes an utterance's adapted Gaussian-mixture mean supervector as $M = m + Tx$. Here $m$ is the background-model supervector, $T$ is a learned low-rank "total variability" matrix, and $x$ (typically 100 to 600 dimensions) is the i-vector (Saeidi et al.; Bansé et al.). Because $T$ captures channel and speaker variation together, $x$ is a cheap, unsupervised signal for deciding which channel regime an utterance came from. That is exactly what the gate needs.
This choice cuts against the dominant approach to channel mismatch. The UR Channel-Robust Synthetic Speech Detection System for ASVspoof 2021 uses adversarial training with a gradient-reversal layer to strip codec and channel information from deep embeddings. It still reaches 20.33% EER on the Deepfake track. Meanwhile, Anti-spoofing system: an investigation of measures to detect synthetic and human speech found that phase features, which are channel-sensitive, beat MFCCs for i-vector spoof detection (19.39% versus 36.06% EER under PLDA). Removing channel information can remove evidence of spoofing.
We keep channel information and let routing and per-expert calibration absorb it. This borrows the domain-gated expert design of MDFEND, which weights text experts by a learned domain embedding but needs explicit domain labels. In our design, the i-vector itself replaces the domain label, so no channel annotation is required at test time.
Research Program
Front end and experts. The front end extracts two i-vector streams per utterance, following the anti-spoofing paper above. One uses MFCCs (standard spectral features) and the other uses modified group delay features, which are phase-based. We apply LDA and length normalization, then concatenate the streams.
The gate is a small network producing a softmax over $K$ experts, with $K$ between 2 and 8. Each expert is a two-layer network trained with a one-class softmax loss, which pulls genuine speech into a tight region and pushes spoofs outward, as in the UR system. We add a standard load-balancing term so the gate does not send everything to one expert.
Tail-homogeneity term. Beyond the load-balancing term, we add one ingredient: a regularizer that rewards the gate for grouping genuine utterances whose score tails agree. For each expert, we penalize the spread of the upper quantile of genuine scores across the codec and channel augmentations assigned to it. The intent is that experts are cut along lines that matter for calibration, not only for classification. We will ablate this term explicitly.
Calibration. After training, each expert fits a peaks-over-threshold tail on held-out genuine scores. Every genuine utterance contributes to every expert's fit, weighted by its gate weight for that expert. Tail parameters are estimated with the Grimshaw procedure from SPOT.
At test time, two variants are run:
- A frozen variant, which uses the tails from development data.
- A streaming variant, which updates each expert's tail with incoming utterances that fall below threshold, as SPOT does, again weighted by the gate.
A known risk. Splitting genuine data across experts leaves fewer samples per tail fit, so estimates at small $q$ get noisier. For this reason we target $q$ between $10^{-2}$ and $10^{-3}$ rather than $10^{-4}$. We will report tail-fit sample counts per expert, and we treat the trade-off between $K$ and calibration error as a primary result, not a nuisance.
Baselines
Naive reference. A single one-class network on the same i-vectors, with one static threshold chosen on ASVspoof 2019 development data to hit the target false-alarm rate. This is how operating points are set in practice. Any method that does not beat it on per-condition false-alarm error has no reason to exist.
Strongest existing detector. The UR-AIR system (UR Channel-Robust Synthetic Speech Detection System for ASVspoof 2021). It reports 5.46% EER with min-tDCF 0.3094 on the ASVspoof 2021 Logical Access track, and 20.33% EER on the Deepfake track, against 22.38% for RawNet2 and 23.48% for LFCC-LCNN. It is the strongest competitor because it attacks the same channel and codec mismatch through invariance, and its EER is what our system will be compared against.
We will re-train an ECAPA-TDNN with one-class softmax and adversarial augmentation on the same ASVspoof 2019 LA training data, with the same tuning budget in trials. We then threshold it the same ways we threshold our model: static, global SPOT, and per-expert. Beating it means one of two things:
- lower worst-condition false-alarm deviation from $q$ at comparable EER; or
- lower actual (not minimum) tandem detection cost, t-DCF.
We found no reported result on Lacuna for realized false-alarm rates at a fixed target on ASVspoof 2021. That number must be produced here for every system.
Strongest adaptive thresholder. The single-expert network with a global SPOT/DSPOT threshold on its score stream. It isolates whether routing adds anything beyond adaptive tail modeling. SPOT's reported operating point elsewhere, 86% true-positive rate at under 4% false-positive rate on MAWI SYN floods, shows the method works on univariate streams. It does not predict speech performance.
Test-time normalization. Gauss-D dynamic scoring from An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series, which re-standardizes scores with local test statistics. That study found it mattered more than the choice of network architecture.
Ablations. We compare our gate against unsupervised k-means clusters on the i-vectors, feeding per-cluster tails of a single detector. We remove the tail-homogeneity term. We also give the gate oracle channel labels, as MDFEND assumes, to measure the gap to perfect routing.
Why not just…?
Why not just fit one adaptive EVT threshold to the whole score stream? It is the right tool when the genuine-score distribution is one population. ASVspoof 2021 deliberately mixes codecs and channels, so a global tail is a mixture of tails. Its quantile over-flags genuine audio in harsh channels and under-flags in clean ones. The global-SPOT baseline tests this directly. If its worst-condition false-alarm deviation matches ours, routing is unnecessary and we will say so.
Why not just cluster i-vectors and fit a threshold per cluster, without experts? This is the most serious threat, since it keeps the conditioning idea and drops the learned gate. The k-means ablation is designed to settle it. Clusters formed by i-vector geometry may not align with where genuine-score tails differ, and that mismatch is what the tail-homogeneity term targets. If k-means clusters calibrate as well, the finding narrows to "condition calibration on unsupervised channel clusters." That is still useful but simpler, and we would report it as the result.
Why not just use deep embeddings, which beat i-vectors on EER? Nothing in the method requires i-vectors. We will run the same gate and per-expert tails on ECAPA-TDNN embeddings, with and without gradient reversal. Our prediction is that gradient-reversal embeddings route worse, because the adversarial training strips the channel cues the gate relies on. If deep embeddings without adversarial training route and calibrate better than i-vectors, the representation claim fails, and the calibration claim stands on its own.
Why not just renormalize scores at test time (Gauss-D)? Local standardization adapts the center and scale of scores but not tail shape. Spoof decisions at $q\le 10^{-2}$ depend on tail shape. It also presumes that recent test traffic is mostly genuine. We include it as a full baseline, and a mixed-codec stream with a high spoof fraction will show whether its assumption breaks.
Proposed Experiments
Protocol. Train on ASVspoof 2019 LA (The ASVspoof 2019 database) and evaluate on the ASVspoof 2021 LA and Deepfake tracks. Within the 2021 evaluation, hold out entire codec and channel conditions from any calibration data, so that at least one condition is never seen. As a sanity check against published i-vector numbers, first reproduce the MFCC plus phase-feature fusion on ASVspoof 2015, which reported 8.55% EER on training and development data only.
Metrics. The primary metric is the absolute deviation of realized genuine false-alarm rate from $q$, reported per condition, as the worst case across conditions, and averaged, for $q \in \{10^{-2}, 5\cdot10^{-3}, 10^{-3}\}$. Secondary metrics are miss rate at the realized threshold, actual versus minimum t-DCF, and EER to confirm ranking is not sacrificed.
Streaming test. Utterances are ordered in blocks by condition, with abrupt codec switches, to test how quickly each thresholder recovers after a switch.
Decisive checks. Three outcomes decide the claim:
- Per-expert tails must beat global SPOT on worst-condition false-alarm deviation, including on held-out codecs.
- The learned gate with the tail-homogeneity term must beat k-means clusters. If it does not, we drop the gate.
- The gap to the oracle-label gate must be small, showing that i-vectors carry enough channel information to route without annotation.
References
- ADMoE: Anomaly Detection with Mixture-of-Experts from Noisy Labels
- Anomaly Detection in Streams with Extreme Value Theory
- Speaker recognition for speech under face cover
- Analysis of the second phase of the 2013-2014 i-vector machine learning challenge
- UR Channel-Robust Synthetic Speech Detection System for ASVspoof 2021
- Anti-spoofing system: an investigation of measures to detect synthetic and human speech
- MDFEND: Multi-domain Fake News Detection
- An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series
- The ASVspoof 2019 database