Abstract
In this paper, we propose PhantomSound, a query-efficient black-box attack toward voice assistants. Existing black-box adversarial attacks on voice assistants either apply substitution models or leverage the intermediate model output to estimate the gradients for crafting adversarial audio samples. However, these attack approaches require a significant amount of queries with a lengthy training stage. PhantomSound leverages the decision-based attack to produce effective adversarial audios, and reduces the number of queries by optimizing the gradient estimation. In the experiments, we perform our attack against 4 different speech-to-text APIs under 3 real-world scenarios to demonstrate the real-time attack impact. The results show that PhantomSound is practical and robust in attacking 5 popular commercial voice controllable devices over the air, and is able to bypass 3 liveness detection mechanisms with >95% success rate. The benchmark result shows that PhantomSound can generate adversarial examples and launch the attack in a few minutes. We significantly enhance the query efficiency and reduce the cost of a successful untargeted and targeted adversarial attack by 93.1% and 65.5% compared with the state-of-the-art black-box attacks, using merely ~300 queries (~5 minutes) and ~1,500 queries (~25 minutes), respectively.
Introduction
Adversarial attacks on Automatic Speech Recognition (ASR) systems—like those powering Alexa, Siri, and Google Assistant—usually fall into two categories: white-box attacks, which require full access to the model, and black-box attacks, which do not. Existing black-box attacks are notoriously difficult because they require thousands of queries to the target API to estimate the model's gradients. This process is slow and expensive, as commercial ASR APIs charge per second of audio.
Furthermore, ASR systems behave differently than image classifiers. While an image model will almost always return a label, an ASR system will often reject "garbage" audio and return no result at all. This creates a "non-contiguous decision boundary" that makes traditional boundary-searching attacks fail.
Navigating the "Void" in ASR Systems
A key discovery in this research is that ASR systems have "exclusive boundaries" where anything between valid words is rejected. Unlike image classification, where the space is densely populated with labels, ASR decision spaces are like islands in a sea of silence or rejection.
To overcome this, PhantomSound uses the user’s voice as a "carrier." By injecting split-second phonemes into legitimate commands, the attack finds a phonetic path through valid-sounding intermediate words (e.g., from "stop" to "stopwhat" to "backward").
Methodology
PhantomSound introduces three main innovations to achieve query efficiency and robustness:
- Phoneme-level Boundary Searching: Instead of adding random noise, PhantomSound breaks a target malicious command into individual phonemes and iteratively injects them into a benign carrier command. This allows the attack to navigate through the ASR's valid decision spaces.
- Query-Efficient Gradient Estimation: The attack uses a modified Sign-Opt algorithm. To save queries, PhantomSound applies the same estimated gradient multiple times until the attack goal is no longer met, drastically cutting down API calls.
- Weak Synchronization: To handle the timing difficulties of playing a perturbation while a human is speaking, the authors designed a scheme that minimizes average loss over a mismatch interval, ensuring robustness even with a 100ms playback offset.
Experimental Evaluation
The researchers evaluated PhantomSound using the Google Speech Commands dataset, self-recorded long commands, and a custom phoneme library consisting of 453 audio clips from 200 speakers. The attack was tested against 4 major APIs (Google Cloud, MS Azure, Amazon Transcribe, and IBM Watson) and 5 commercial devices.
Key Results
- Efficiency: PhantomSound requires only ~300 queries (5 minutes) for untargeted attacks and ~1,500 queries (25 minutes) for targeted attacks, representing a 93.1% and 65.5% reduction in cost compared to previous state-of-the-art methods.
- Effectiveness: The attack achieved a >95% success rate against three popular liveness detection mechanisms (CQCC, STC, and Void). These defenses typically look for electronic signatures; because the human voice is present, the system is fooled into thinking the signal is legitimate.
- Transferability: Perturbations crafted for one speaker showed robust transferability, with success rates of 98% for male speakers and 74% for female speakers.
Conclusion and Future Work
PhantomSound demonstrates that phoneme-level injection is a stealthy and efficient method for attacking commercial voice assistants. While highly effective, the attack remains sensitive to ambient noise and struggles with very long sentences. Future work is needed to develop more robust ASR architectures that can counter these subtle phonetic manipulations.
Summary updated