In the study of adversarial machine learning, one phenomenon consistently challenges our assumptions about model security: adversarial transferability. An adversarial example optimized in a "white-box" setting to fool a specific surrogate model will frequently fool an entirely different, unseen "black-box" model.
In the audio domain—spanning Automatic Speech Recognition (ASR), speaker verification, and synthetic speech detection—this cross-model generalization is remarkably robust. An attacker does not need to know the exact architecture, parameters, or cloud platform of a target system to compromise it.
But why does an optimization process tailored to the loss surface of Model A yield a key that unlocks Model B? The transferability of audio adversarial examples is not a mystical coincidence; it is the direct mathematical consequence of shared signal-processing front-ends, convergent self-supervised representations, and the intrinsic physics of human speech.
The Shared Front-End: Signal Processing Vulnerabilities
Before an audio waveform ever reaches the hidden layers of a neural network, it undergoes a deterministic mathematical transformation. Raw audio is highly redundant, so almost all modern speech and speaker recognition systems rely on standardized feature extraction techniques, such as Mel-Frequency Cepstral Coefficients (MFCCs) or Mel-spectrograms, to map time-domain signals into the frequency domain.
As demonstrated by Abdullah et al., this reliance on universal signal-processing pipelines creates a shared vulnerability. When an attacker crafts a perturbation that distorts these standardized acoustic features, the malicious signal remains effective across diverse proprietary and open-source architectures. The attack targets the mathematical physics of the feature extractor rather than the specific weights of the classifier.
Raw Audio ---> [ Standardized Front-End (MFCC/Mel) ] ---> [ Model A (Surrogate) ] ---> Error
---> [ Model B (Target) ] ---> Error
This vulnerability is deeper than a single choice of feature engineering. Kreuk et al. explored "cross-feature attacking," where perturbations are generated on a surrogate model using one acoustic feature set and evaluated on a target model using another. They discovered that noise generated to exploit one representation successfully transfers to models using entirely different features. This indicates that adversarial weaknesses are fundamentally inherent to how neural networks process speech signals globally, transcending individual front-end configurations.
The Convergence of Self-Supervised Representations
Historically, speech models were trained via supervised learning to map audio directly to specific text or labels. These models often developed highly specialized decision boundaries, making them somewhat resistant to transferred attacks. However, modern state-of-the-art speech systems rely heavily on Self-Supervised Learning (SSL) pretraining (e.g., Wav2Vec2, HuBERT).
Olivier et al. demonstrated that this shift in training paradigm is a primary driver of modern adversarial transferability. When models are pretrained on massive, unlabeled audio corpora, they are forced to learn exhaustive, overlapping feature representations of the underlying speech distribution.
Because different SSL models optimize for the same fundamental structure of human language, their internal representations converge. Olivier et al. showed that while supervised models remain resistant to transferred perturbations, models sharing similar self-supervised pretraining objectives are highly susceptible. An adversarial example optimized to shift a representation in one SSL space will effortlessly shift it in another.
Cross-Scale and Cross-Architecture Generalization
Intuition might suggest that a small, simple neural network would be an inadequate surrogate for attacking a massive, state-of-the-art production system. In practice, the opposite is true.
Gao et al. showed that adversarial examples generated on smaller surrogate models can successfully compromise significantly larger and more complex architectures. This cross-scale transferability occurs because diverse ASR systems share fundamental vulnerabilities in how they extract low-level acoustic features. They also noted a key distinction in attack design: untargeted attacks (which simply aim to maximize transcription errors or cause a system to output nothing) transfer far more effectively across models than targeted attacks (which force a specific malicious phrase).
This cross-architecture alignment is so pronounced that it bridges entirely different modeling paradigms. As evaluated by Jati et al. and Xingyu Zhang et al., perturbations optimized against deep neural networks successfully transfer to traditional statistical or factor-analysis models (like i-vectors). The underlying susceptibility to signal manipulation is a property of the task and the data distribution, not just the neural network topology.
Exploiting and Defending Transferability
The reality of cross-model transferability has forced a re-evaluation of both audio privacy tools and defensive security strategies.
Proactive Defenses and Privacy Camouflage
If adversarial perturbations transfer reliably to unknown models, we can use them defensively to protect user privacy. For instance, Chiquier et al. developed real-time neural voice camouflage systems that play subtle over-the-air sounds to disrupt unauthorized automatic speech recognition. Similarly, Enkidu leverages transferability to inject frequency-domain patches into a user's speech, preventing unauthorized text-to-speech models from cloning their vocal identity. Because the biometric features used for speaker identification are highly conserved across architectures, a model-agnostic defense remains highly effective.
The Intrinsic Fingerprints of Attacks
Because adversarial perturbations rely on exploiting universal properties of audio processing, the attacks themselves leave distinct, model-independent traces. Villalba et al. demonstrated that detectors trained to identify adversarial noise on one type of system (such as speaker identification) generalize remarkably well to entirely different tasks (like ASR). The adversarial noise possesses an intrinsic "fingerprint" that can be isolated via representation learning, regardless of the victim model.
Exploiting the Gaps
While transferability is powerful, it is rarely 100% perfect. Defensive frameworks can turn this imperfection into a detection mechanism. Fu et al. introduced MVP-EARS, a system inspired by multiversion programming. By passing an audio clip through multiple distinct ASR architectures simultaneously, the system checks for consensus. Because adversarial examples exploit subtle, model-specific alignment gaps during transfer, a malicious clip will often cause wildly divergent transcriptions across different models, exposing its adversarial nature.
---> [ ASR Version 1 ] ---> "Hello World"
|
Audio Clip ----+---> [ ASR Version 2 ] ---> "Hello World" ===> Clean Audio (Consensus)
|
---> [ ASR Version 3 ] ---> "Hello World"
-----------------------------------------------------------------------------------------
---> [ ASR Version 1 ] ---> "Go to malicious.com"
|
Adversarial ---+---> [ ASR Version 2 ] ---> "Open window" ===> Attack Detected
| (Divergence)
---> [ ASR Version 3 ] ---> [Gibberish / Muted]
Ultimately, the cross-model transferability of audio attacks reveals that deep learning models do not learn in a vacuum. By operating on the same physical signals and converging toward the same mathematical representations of human speech, they inherently inherit the same vulnerabilities.
Key Foundational Work
Fooling End-To-End Speaker Verification With Adversarial Examples (2018) — contributed 2 concept elements to this direction.
Adversarial Optimization for Dictionary Attacks on Speaker Verification (2019) — contributed 2 concept elements to this direction.
Adversarial Black-Box Attacks on Automatic Speech Recognition Systems Using Multi-Objective Evolutionary Optimization (2019) — contributed 1 concept elements to this direction.