Detectors that tell AI-generated speech from real recordings are trained mostly on English and fail badly elsewhere. A detector trained on English benchmarks scores an average precision of 2.42% on Arabic–English code-switched deepfakes, and Urdu detectors perform at coin-flip level. One likely reason is that the detector has partly learned "what English real speech sounds like" rather than "what synthesis leaves behind." This proposal borrows a tool from protein-folding interpretability to test that explanation directly and to fix it. The tool is to find, inside the network, the specific internal directions that carry a given kind of information, then check by intervention whether the network's output actually depends on them.
The proposal has two steps. First, we locate the internal directions in a speech detector's backbone that encode language identity and that causally move the real-versus-fake score. Second, we project those directions out at test time. The projection is estimated from unlabeled real speech in the target language, so the method needs no deepfakes in that language. If it works, it offers a cheap, auditable way to carry a detector into languages where fake training data does not exist. That problem has the same shape as structural biology's reliance on scarce experimental structures, and the answer is also the same kind: substitute an abundant, cheaper signal for the scarce one.
Core Claim
In self-supervised speech backbones used for deepfake detection, some of the evidence the classifier uses is carried by a low-dimensional subspace that also linearly encodes language. The subspace is found at identifiable layers. Removing only the part of it that is causally used by the detector, as shown by activation patching, should narrow the cross-lingual gap more than two alternatives: blanket adversarial removal of language information, and simple layer selection. It should also preserve in-language accuracy.
Three lines of prior work motivate this claim.
- Comparative Analysis of ASR Methods for Speech Deepfake Detection benchmarks frozen Whisper and wav2vec 2.0 encoders with linear heads. It finds that the 128-language XLS-R detects English fakes worse than monolingual backbones, and that Whisper Large is beaten by Whisper Small. This suggests that linguistic content competes with forensic evidence inside the representation.
- HierCon learns attention over time and layers of XLS-R. It reports that discrimination peaks in intermediate layers, where acoustic features turn into phonetic ones. That is also where language information should begin to appear.
- Turing's Echo shows that changing one word in ten can drop the AASIST2 detector's fake probability from 91.2% to 6.4%. So detectors respond to linguistic content, not only to acoustics.
None of these papers localizes where that sensitivity lives or tests whether it causes the decision.
The method comes from Mechanisms of AI Protein Folding in ESMFold. That work combines linear probes, activation patching (overwriting part of one input's internal activations with another's and watching the output change) and steering. With these it shows that folding trunks encode local biochemistry early and pairwise geometry late, and that both codes are causally used. We reuse that procedure on speech detectors. We do not reuse any protein-specific representation.
We deliberately do not pursue the more literal transfer suggested by the file, which is recasting speech frames as pairwise distance maps in the style of SPROF. A frame-to-frame distance map computed from the same backbone features inherits whatever language information those features carry. Nothing about the representation removes it.
Research Program
Detectors and layer-wise probing
We start from two standard detectors: a frozen XLS-R backbone with a light head, and a fine-tuned XLS-R detector in the style of HierCon. Both are trained on ASVspoof 2019 LA.
At every layer we fit linear probes for three things:
- language identity, using bona fide speech from IndicFake's languages, ArEnAV's Modern Standard Arabic and dialect audio, BRSpeech-DF, and the Urdu set;
- real versus fake, within each language;
- generator family.
The probes give a layer-by-layer map of where language information and artifact information each concentrate. That map is the speech analogue of ESMFold's two-stage picture.
Causal test with matched pairs
We construct matched pairs using a multilingual voice-cloning system such as XTTS-v2, which ArEnAV also uses. Within a pair, the speaker voice, the generator and the real/fake label are fixed, and only the spoken language differs. Real bona fide pairs come from code-switched ArEnAV speakers.
At layer $l$ we compute the language subspace $U_l$ with a probe. We then patch the component of the activation that lies in $U_l$ from the language-B member of a pair into the language-A member, and measure how much the detector score moves.
The target is directions that satisfy two conditions: patching them moves the score while the label is held fixed, and patching them between a real and a fake utterance in the same language does not move the score. We retain only those directions. The result is a causally filtered subspace $C_l$ with orthonormal columns.
Erasure at test time
The fix is a fixed projection applied to the backbone features:
$$\tilde h_l = h_l - C_l C_l^\top (h_l - \mu_l),$$
where $h_l$ is the layer-$l$ feature and $\mu_l$ is the mean feature of bona fide speech pooled across the calibration languages.
For a new low-resource language, we re-estimate $C_l$ using only unlabeled real speech in that language together with the existing languages. No fakes are needed, because the subspace is defined by language variation among real recordings. In a second variant, we fine-tune the detector head with the projection in place.
Baselines
Naive reference. The unmodified English-trained detector, meaning ASVspoof 2019 LA training with the XLS-R backbone, evaluated zero-shot on each target language. This quantifies the gap we want to close. On ArEnAV, the corresponding reported figure for English-trained models is 2.42% [email protected] (Tell me Habibi, is it Real or Fake?). That number comes from audio-visual detectors, so we will re-measure an audio-only reference ourselves.
Strongest in-English passive detector. Balance, Multiple Augmentation, and Re-synthesis (the "Triad" strategy) trains with supervised contrastive learning on an XLS-R backbone. Its hard negatives are vocoder re-synthesized copies of real utterances, which hold the words constant, and it also uses RawBoost augmentation and balanced mini-batches. It reports 2.17% EER on ASVspoof 2021 DF and 4.51% EER on In-the-Wild, down from 10.48%. HierCon reports 1.93% EER on DF and 6.87% EER on In-the-Wild. We re-train both on identical ASVspoof 2019 LA data with the same backbone and the same hyperparameter search budget. We then add our projection on top of each. Beating them means two things: lower EER on the non-English sets, and In-the-Wild EER no more than 0.5 points worse than the Triad strategy's 4.51%.
Strongest multilingual system. IndicFake Meets SAFARI-LLM fuses Whisper and m-HuBERT streams through a gating module and classifies with a LoRA-tuned LLaMA-7B. It reports 94.21% accuracy for English-to-Japanese transfer on WaveFake, 84.48% for English-to-Chinese transfer on DECRO, and 0.941% intra-family EER on IndicFake. Our detector is roughly 20 times smaller, so we do not expect to match its in-family numbers. The comparison that matters is cross-lingual transfer on WaveFake and DECRO. Matching its 84.48% on DECRO with a 300M-parameter model and no Chinese fakes would count as a meaningful result.
Ablations that isolate the key ingredient.
- Erasing the full probe subspace $U_l$ instead of the causally filtered $C_l$.
- Erasing a random subspace of the same dimension.
- Erasing $C_l$ at the wrong layers.
If the unfiltered $U_l$ works as well as $C_l$, the causal filtering step is unnecessary and the contribution shrinks to plain concept erasure.
Why not just…?
Why not just fine-tune on target-language fakes, as IndicFake does? Where such data exists, fine-tuning is the upper bound, and we report it as one. The regime we target is languages with no fakes available. Even with target data, fine-tuning on ArEnAV reached only 79.97% AUC (Tell me Habibi), and a code-switched test is exactly where a language shortcut would bite. We also test fine-tuning combined with erasure, to see whether the two are complementary.
Why not just re-synthesize target-language real speech with vocoders? This is the most serious threat. Real target-language speech is abundant, and the Triad recipe could be applied in any language to make cheap, word-matched fakes. The limitation, which the Triad authors themselves acknowledge, is that such fakes cover vocoder artifacts and miss neural-codec and concatenative generators. We will run target-language Triad as a baseline. If it matches erasure on codec-generated test sets such as ArEnAV's XTTS-v2 and OpenVoice-v2 fakes, then erasure is not needed. In that case we would report the combination of the two, not erasure alone.
Why not just train with an adversarial language classifier? Gradient reversal removes all linearly decodable language information. It can therefore also delete acoustic cues that happen to correlate with language, such as recording channels typical of a region. It is run with the same budget as our method, and the full-$U_l$ ablation tests this concern in its linear form.
Why not just pick the best layer, following HierCon's observation about intermediate layers? Layer selection is a baseline in our study. If one layer free of language information carries all the artifact evidence, layer selection will tie with erasure, and the probing map will show that directly.
Proposed Experiments and Decisive Checks
All models are trained on ASVspoof 2019 LA. Evaluation uses EER (the error rate at the threshold where false accepts equal false rejects) and AUC on the following sets:
- ASVspoof 2021 DF and In-the-Wild, as an English sanity check;
- zero-shot transfer to IndicFake language families, ArEnAV audio, BRSpeech-DF, the Urdu set, WaveFake Japanese and DECRO.
Check 1: causal dependence. Language-swap patching in matched XTTS-v2 pairs should move the detector score by a measurable amount before erasure, and by near zero after. If patching moves the score only negligibly even before erasure, the language-shortcut hypothesis is wrong for these backbones and the project stops there.
Check 2: code-switching. On ArEnAV, the false-positive rate on real utterances should be broken down by whether the utterance contains a code-switch. Erasure should remove the excess false alarms at switch points.
Check 3: lexical stability. Using the Turing's Echo protocol, we substitute a single word in cloned speech and measure how much the score changes before and after erasure.
Check 4: generator family. We separate held-out-language gains by generator family: vocoder, codec-based autoregressive, and flow-matching. This distinguishes genuine removal of language information from a side effect of one synthesis type.
References
- Comparative Analysis of ASR Methods for Speech Deepfake Detection
- HierCon: Hierarchical Contrastive Attention for Audio Deepfake Detection
- Turing's Echo: Investigating Linguistic Sensitivity of Deepfake Voice Detection via Gamification
- Mechanisms of AI Protein Folding in ESMFold
- To Improve Protein Sequence Profile Prediction through Image Captioning on Pairwise Residue Distance Map
- Tell me Habibi, is it Real or Fake?
- Balance, Multiple Augmentation, and Re-synthesis: A Triad Training Strategy for Enhanced Audio Deepfake Detection
- IndicFake Meets SAFARI-LLM: Unifying Semantic and Acoustic Intelligence for Multilingual Deepfake Detection