TL;DR
TTSlow is a novel adversarial attack framework designed to compromise the efficiency of Text-to-Speech (TTS) systems by inducing "endless" speech generation. By applying nearly imperceptible perturbations to input text or speaker embeddings, the authors demonstrate that both autoregressive and non-autoregressive models can be tricked into massive inference delays, effectively creating a computational Denial-of-Service (DoS) attack.
What Problem Does This Paper Address?
While the research community has spent years making TTS models faster and more natural, their efficiency robustness—how well they maintain performance under adversarial conditions—has been largely ignored. Most adversarial research in audio focuses on "accuracy" (e.g., making a model say something different).
However, in real-world deployments like banking verification, autonomous vehicle alerts, or emergency notifications, a delay is just as dangerous as a wrong word. If an attacker can force a TTS system to take 10 seconds to generate a 2-second alert, they can exploit the resulting security gap. This paper identifies this "slow-down" vulnerability across diverse TTS architectures.
Methods
The authors propose two primary attack strategies under the TTSlow umbrella:
- TTSlow-text: This targets the input text. Since text is discrete and non-differentiable, the authors use a differentiable objective approximation. They combine:
- Character Swaps: Replacing a character with another from the vocabulary.
- Homoglyph Replacement: Using visually identical but different Unicode characters (e.g., replacing a Latin 'a' with a Cyrillic 'а').
- The goal is to find the specific character changes that most aggressively increase the gradient of the loss function associated with speech length.
- TTSlow-spk: This targets the speaker embedding (the vector representing the voice's characteristics). Using Projected Gradient Descent (PGD), the attack adds a tiny, human-unnoticeable noise vector to the embedding.
The "Secret Sauce" - Efficiency Loss Functions: The attack targets the specific mechanism each model uses to decide when to stop talking:
- For Autoregressive models (like SpeechT5): The loss function encourages the model not to predict the "stop token," effectively telling it to keep generating frames indefinitely.
- For Non-Autoregressive models (like VITS): The loss function targets the "stochastic duration predictor," forcing it to maximize the predicted duration for every input phoneme.
Datasets
The researchers evaluated TTSlow using three widely-used public datasets to ensure the results weren't voice- or script-specific:
- LJ Speech: A single-speaker English dataset.
- LibriSpeech: A large-scale corpus of read English speech.
- English Dialects: A dataset featuring various English accents (e.g., Scottish female subset) to test cross-accent robustness.
Key Insights and Ideas
- Length as a Vulnerability: The most striking insight is how fragile the "stop" mechanism is in modern TTS. A perturbation so small it's invisible to a human can cause a model to generate 4x more audio than requested.
- The "Endless Loop" Phenomenon: Qualitative analysis showed that these attacks often cause models to enter "word repetition" loops or generate long stretches of silence/noise, which consumes massive GPU energy and time.
- Intelligibility Trade-off: Using Gemini 1.5 Pro for evaluation, the authors found that as the speech gets longer (the "slow down"), the actual intelligibility scores plummet. The attack doesn't just delay the message; it effectively destroys it.
- Architecture Agnostic: The paper proves that this isn't a bug in one specific model but a fundamental challenge for both autoregressive (sequential) and non-autoregressive (parallel) architectures.
Key Results
- Massive Delays: On the LJ Speech dataset, TTSlow-spk (l2) increased the mean number of frames by 300% compared to clean inputs.
- High Success Rate: The framework achieved an average Attack Success Rate (ASR) of 90.39% across all tested models and datasets.
- Resource Exhaustion: The attacks significantly increased both inference time and GPU energy consumption, proving the feasibility of a computational DoS attack.
- Human Imperceptibility: The text attacks (homoglyphs) are virtually impossible for a human reader to spot, yet they successfully crippled the model's efficiency.
Limitations and Future Work
The primary limitation is that TTSlow currently operates in a white-box setting, meaning the attacker needs access to the model's internal parameters and gradients to calculate the perturbations.
The authors suggest that future research should focus on:
- Black-box attacks: Developing methods that don't require internal model access.
- Defense Mechanisms: Creating "efficiency-robust" training methods or detection systems that can flag adversarial inputs before they hit the synthesis engine.
- Accuracy-oriented extensions: Combining slow-down attacks with content-manipulation attacks for even more sophisticated audio security threats.
Summary updated