Phantom in the opera: adversarial music attack for robot dialogue system

Sheng Li, Jiyi Li, Yang Cao

Abstract

As Android robots become increasingly integrated into domestic and professional environments, their reliance on Automatic Speech Recognition (ASR) modules introduces significant security vulnerabilities. This paper explores a stealthy adversarial attack, "Phantom in the Opera," which hides malicious voice commands within ordinary music clips. By utilizing psychoacoustic masking and forced alignment, the attack generates perturbations that are imperceptible to humans but highly effective at triggering unauthorized actions in industry-standard Time-Delay Neural Network (TDNN) models. Our results demonstrate that certain musical genres, such as Pop and Big Band, provide superior camouflage, achieving high command recognition rates with minimal audible distortion.

Introduction

Modern robot dialogue systems depend on Deep Neural Networks (DNNs) to process and interpret human speech. While these systems have reached high levels of accuracy, they remain susceptible to adversarial attacks—small, intentional modifications to input data that lead to incorrect model outputs. In the audio domain, the primary challenge is maintaining imperceptibility; human hearing is highly sensitive to the "hissing" or robotic artifacts often produced by mathematical optimization.

This research addresses the gap in creating effective yet stealthy targeted attacks. By using music as a "natural camouflage," the proposed method hides instructions like "OK, open the front door" within background tracks, effectively creating "ghost commands" that bypass human detection while hijacking the robot's control system.

Methodology

The "Phantom in the Opera" attack targets the Time-Delay Neural Network (TDNN) architecture, a staple in industrial ASR. The attack pipeline involves a targeted white-box approach where the raw audio waveform is iteratively perturbed to minimize the loss between the model's prediction and the target malicious command.

The 'Phantom in the Opera' attack pipeline, illustrating the iterative optimization process using frequency masking and forced alignment.

Key components of the methodology include:

  1. Fast Gradient Signed Method (FGSM): A modified version of FGSM is used to optimize the 256-dimension raw waveform directly, providing finer control than feature-based methods.
  2. Forced Alignment: This technique identifies optimal "mask regions" within the carrier music where spectral characteristics most closely align with the target speech, increasing attack efficiency.
  3. Psychoacoustic Masking: The attack employs frequency and temporal masking to keep perturbations below the human auditory threshold. By restricting changes to the low-frequency domain (< 8,000 Hz) and hiding them behind the music’s natural overtones, the audio quality remains high.

Experimental Setup

The TDNN model was trained on 460 hours of Librispeech data. To evaluate the attack, the researchers selected 22 common or dangerous commands (4 to 9 words in length) from Google Home datasets. These commands were embedded into seven diverse music genres: Anime, Ballade, Big Band, Heavy Metal, House, Pop, and Rock.

Results and Discussion

The effectiveness of the attack varied significantly across musical genres. Contrary to expectations, high-energy genres like Heavy Metal were less effective carriers. The most successful camouflage was found in Pop and Big Band music.

Spectral analysis of a pop ballad, showing the fundamental frequency ($f_0$) and overtones used to camouflage adversarial perturbations.

As shown in Figure 5, the complex structure of pop music—characterized by clear fundamental frequencies ($f_0$) and rich overtones—provides an ideal environment for masking adversarial noise. These genres share energy distributions similar to human speech, making it easier for the ASR model to "extract" the hidden command.

Key findings include:

  • Command Length: Longer commands (7–9 words) were surprisingly easier to hide and achieved higher success rates. The additional context likely allows the optimizer more flexibility in aligning perturbations with the model's temporal expectations.
  • Performance Metrics: In Big Band music, the attack achieved a Word Error Rate (WER) of just 17.08%, indicating high reliability. Subjective Mean Opinion Scores (MOS) and objective metrics like PESQ confirmed that the adversarial music was nearly indistinguishable from the original tracks.

Conclusion

The "Phantom in the Opera" attack demonstrates a critical vulnerability in current robot dialogue systems. By leveraging the structural similarities between music and speech, attackers can deliver imperceptible commands with high precision. Future work will focus on extending these attacks to black-box scenarios and testing robustness in real-world physical environments. Ultimately, this research highlights the urgent need for defensive mechanisms, such as specialized audio filtering and liveness detection, to secure the next generation of human-robot interaction.

Summary updated

Abstract

p>This study explores the vulnerability of robot dialogue systems' automatic speech recognition (ASR) module to adversarial music attacks. Specifically, we explore music as a natural camouflage for such attacks. We propose a novel method to hide ghost speech commands in a music clip by slightly perturbing its raw waveform. We apply our attack on an industry-popular ASR model, namely the time-delay neural network (TDNN), widely used for speech and speaker recognition. Our experiment demonstrates that adversarial music crafted by our attack can easily mislead industry-level TDNN models into picking up ghost commands with high success rates. However, it sounds no different from the original music to the human ear. This reveals a serious threat by adversarial music to robot dialogue systems, calling for effective defenses against such stealthy attacks.</p>
BibTeX
@article{li2024phantom,
  author = {Sheng Li and Jiyi Li and Yang Cao},
  title = {{Phantom in the opera: adversarial music attack for robot dialogue system}},
  journal = {Frontiers of Computer Science},
  year = {2024},
  doi = {10.3389/fcomp.2024.1355975},
}
Figure