Abstract
Text-to-speech (TTS) has been extensively studied for generating high-quality speech with textual inputs, playing a crucial role in various real-time applications. For real-world deployment, ensuring stable and timely generation in TTS models against minor input perturbations is of paramount importance. Therefore, evaluating the robustness of TTS models against such perturbations, commonly known as adversarial attacks, is highly desirable. In this paper, we propose TTSlow, a novel adversarial approach specifically tailored to slow down the speech generation process in TTS systems. To induce long TTS waiting time, we design novel efficiency-oriented adversarial loss to encourage endless generation process. TTSlow encompasses two attack strategies targeting both text inputs and speaker embedding. Specifically, we propose TTSlow-text, which utilizes a combination of homoglyphs-based and swap-based perturbations, along with TTSlow-spk, which employs a gradient optimization attack approach for speaker embedding. TTSlow serves as the first attack approach targeting a wide range of TTS models, including autoregressive and non-autoregressive TTS ones, thereby advancing exploration in audio security. Extensive experiments are conducted to evaluate the inference efficiency of TTS models, and in-depth analysis of generated speech intelligibility is performed using Gemini. The results demonstrate that TTSlow can effectively slow down two TTS models across three publicly available datasets.
Introduction
While research in Text-to-Speech (TTS) has primarily focused on improving naturalness and inference speed, the efficiency robustness of these systems—how they maintain performance under adversarial conditions—has been largely overlooked. In critical real-world deployments such as emergency notifications or autonomous vehicle alerts, a significant delay in speech generation can be as dangerous as an incorrect message.
This paper identifies a "slow-down" vulnerability across diverse TTS architectures. We introduce TTSlow, an adversarial framework that exploits the mechanisms TTS models use to determine speech duration, forcing them into massive inference delays and effectively creating a computational Denial-of-Service (DoS) attack.
Methodology
TTSlow employs two primary attack strategies to compromise TTS efficiency by targeting different input modalities.
TTSlow-text
This strategy targets the input text. Since text is discrete, the framework utilizes a differentiable objective approximation to apply nearly imperceptible perturbations. These include Character Swaps (replacing characters within the vocabulary) and Homoglyph Replacements (using visually identical Unicode characters). The goal is to find perturbations that maximize the gradient of the efficiency-oriented loss function.
TTSlow-spk
This approach targets the speaker embedding, the vector representing voice characteristics. Using Projected Gradient Descent (PGD), the attack adds a tiny, human-unnoticeable noise vector to the embedding to disrupt the model's timing logic.
Efficiency Loss Functions
The core of the attack lies in targeting the specific "stop" mechanisms of different architectures:
- Autoregressive models: The loss function encourages the model to avoid predicting the "stop token," leading to indefinite generation.
- Non-Autoregressive models: The loss targets the stochastic duration predictor, forcing it to maximize the predicted duration for every input phoneme.
Experiments and Results
The framework was evaluated using three public datasets: LJ Speech, LibriSpeech, and English Dialects.
Inference Efficiency and Success Rate
TTSlow demonstrated high effectiveness across all tested models. On the LJ Speech dataset, TTSlow-spk increased the mean number of frames by 300%. The framework achieved an average Attack Success Rate (ASR) of 90.39%. These increases in frame counts directly correlate with higher GPU energy consumption and significant inference delays.
Impact on Intelligibility
Qualitative analysis using Gemini 1.5 Pro revealed that the "slow-down" effect often results in repetitive loops or long stretches of noise/silence. As the generated speech length increases, intelligibility scores plummet. Thus, the attack does not merely delay the message; it effectively destroys its content while exhausting computational resources.
Conclusion
TTSlow highlights a fundamental security gap in modern TTS architectures, proving that both sequential and parallel models are vulnerable to efficiency-based adversarial attacks. While currently demonstrated in a white-box setting, these findings underscore the urgent need for efficiency-robust training methods and detection systems to secure TTS deployments against resource exhaustion threats.
Summary updated