Abstract
Adversarial attacks on Automatic Speech Recognition (ASR) systems have historically been computationally expensive and fragile under real-world conditions. This paper presents a novel approach using Weighted Perturbation Technology (WPT) and Sampling Perturbation Technology (SPT) to generate robust audio adversarial examples efficiently. By focusing perturbations on key audio segments identified by an Audio Sequence Location (ASL) model and sampling only a portion of the audio vector, the proposed method reduces generation time from hours to minutes while improving signal-to-noise ratio (SNR) and environmental robustness.
Introduction
The rise of smart assistants has made ASR systems ubiquitous, yet they remain vulnerable to adversarial inputs—carefully crafted audio that sounds like one thing to humans but is interpreted as a malicious command by machines. Existing attacks, such as the Carlini & Wagner (C&W) method, suffer from two main drawbacks: extreme slowness (often taking over an hour per clip) and "brittleness" (failing when exposed to minor environmental noise). This research addresses these gaps by optimizing where and how perturbations are applied to the audio signal.
Methods
The researchers introduce three primary innovations to the standard gradient-descent-based attack:
- Weighted Perturbation Technology (WPT): The authors observed that the "last mile" of generating an attack (reducing the Levenshtein distance from 3 to 0) accounts for over 55% of the total computation time. WPT uses an Audio Sequence Location (ASL) model to identify "key points" in the audio that correspond to the characters that still need to change. It then assigns higher weights to these points during optimization, forcing the model to converge faster.
- Sampling Perturbation Technology (SPT): Instead of modifying every sample point, SPT perturbs only a subset (e.g., 75%). This concentration of adversarial signal makes the attack more robust against random noise, as the original audio's characteristics are better preserved.
- Total Variation Denoising (TVD): A TVD-based loss function is used to smooth perturbations, making them sound more like natural background hiss rather than artificial artifacts.
Results
The proposed method was evaluated using the Mozilla Common Voice dataset (MCVD). The results demonstrate significant improvements over baseline attacks like C&W and CommanderSong.
Key findings include:
- Efficiency: Generation time was slashed from ~3,600 seconds to approximately 251 seconds, a 14x speedup.
- Stealthiness: The attack achieved a higher SNR of 31.9 dB, indicating the added noise is much quieter and less perceptible than previous methods (which hovered around 17-18 dB).
- Robustness: In tests with simulated random noise, the SPT-based approach maintained a significantly higher success rate than the baseline. For example, at a noise level of $\Delta = 15$, the baseline robustness was 0.04, while the SPT-EOT combined approach maintained a robustness of 0.30.
Conclusion
This work demonstrates that audio adversarial attacks are now practical at a "minute-level" time scale. By leveraging the temporal nature of speech and focusing perturbations on critical segments, the proposed weighted-sampling approach achieves both efficiency and robustness. These findings highlight the increasing vulnerability of ASR systems and the urgent need for more robust defense mechanisms in commercial speech recognition technology.
Summary updated