ADAGIO: Interactive Experimentation with Adversarial Attack and Defense for Audio

Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Li Chen, Michael E. Kounavis, Duen Horng ChauECML/PKDD 2018 · arXiv · OpenReview

Abstract

Adversarial machine learning research has recently demonstrated the feasibility to confuse automatic speech recognition (ASR) models by introducing acoustically imperceptible perturbations to audio samples. To help researchers and practitioners gain better understanding of the impact of such attacks, and to provide them with tools to help them more easily evaluate and craft strong defenses for their models, we present ADAGIO, the first tool designed to allow interactive experimentation with adversarial attacks and defenses on an ASR model in real time, both visually and aurally. ADAGIO incorporates AMR and MP3 audio compression techniques as defenses, which users can interactively apply to attacked audio samples. We show that these techniques, which are based on psychoacoustic principles, effectively eliminate targeted attacks, reducing the attack success rate from 92.5% to 0%. We will demonstrate ADAGIO and invite the audience to try it on the Mozilla Common Voice dataset.

Introduction

Deep Neural Networks (DNNs) used in Automatic Speech Recognition (ASR) are notoriously vulnerable to adversarial attacks. An attacker can add a tiny amount of "noise" to an audio clip—noise that is imperceptible to human ears—which causes the AI to transcribe the audio as a completely different, malicious phrase. For example, a command like "Hey Siri, play music" could be surreptitiously altered to "Hey Siri, open the front door."

While these attacks are well-documented in research, they often remain abstract. There is a lack of accessible tools for researchers to experiment with these attacks in real-time. Furthermore, finding a defense that is both effective and computationally efficient remains a significant challenge in AI security. ADAGIO (Adversarial Defense for Audio in a Gadget with Interactive Operations) addresses this by providing an interactive sandbox for visualizing and hearing how adversarial attacks "trick" models and how defenses can neutralize them.

The ADAGIO System

ADAGIO is a web-based system composed of four main components:

  1. Interactive UI: A visual interface where users can upload audio, run attacks, and apply defenses.
  2. Speech Recognition Module: Utilizes Mozilla’s implementation of DeepSpeech, a state-of-the-art RNN-based model.
  3. Attack Generator: Implements the Carlini & Wagner iterative attack, which crafts perturbations over 100 iterations to reach a target transcription.
  4. Defense Module: Employs AMR (Adaptive Multi-Rate) and MP3 compression.
The ADAGIO interface allows users to interactively apply attacks and defenses to audio samples.

The logic behind using compression is rooted in psychoacoustics. These codecs are designed to discard audio information that the human ear cannot perceive. Since adversarial perturbations are designed to be "imperceptible" to humans, the authors hypothesized that standard compression would strip away the malicious noise while leaving the actual speech intact.

Experimental Evaluation

The researchers evaluated ADAGIO using the Mozilla Common Voice dataset. They processed the first 100 test samples and generated five adversarial instances for each, targeting random phrases from the dataset. This resulted in a benchmark of 500 adversarial audio samples.

Key Results and Insights

The quantitative results demonstrate that compression is an incredibly effective defense against non-adaptive attacks. Both AMR and MP3 compression reduced the targeted attack success rate from 92.45% to 0.00%.

Quantitative evaluation of AMR and MP3 defenses against adversarial attacks.

Under attack, the Word Error Rate (WER) was extremely high (1.287). Applying the AMR defense cut that error rate nearly in half (0.666), significantly recovering the original message. However, there is a slight "utility tax": applying compression to clean (non-attacked) audio slightly increases the WER (e.g., from 0.369 to 0.488 for AMR) because some useful signal is lost during compression.

The study highlights several key insights:

  • Adversarial Fragility: Sophisticated mathematical optimizations that perfectly fool a raw waveform are often fragile and can be broken by standard audio processing.
  • Leveraging Human Perception: By using compression algorithms designed for human hearing, the AI's perception is aligned more closely with human perception, effectively filtering out "imperceptible" adversarial noise.
  • Interactivity: ADAGIO moves research into a sandbox, allowing researchers to build better intuition by hearing the audio while seeing transcriptions change in real-time.

Conclusion and Future Work

The primary limitation of this study is the focus on non-adaptive attacks, where the attacker is unaware of the compression defense. If an attacker knows the specific algorithm, they might craft an "adaptive attack" to survive compression. However, the authors note that creating such attacks for audio is currently prohibitively slow and often fails to converge, suggesting that compression remains a formidable practical hurdle. Future work will involve testing ADAGIO against more sophisticated, defense-aware adversaries.

Summary updated

Abstract

Adversarial machine learning research has recently demonstrated the feasibility to confuse automatic speech recognition (ASR) models by introducing acoustically imperceptible perturbations to audio samples. To help researchers and practitioners gain better understanding of the impact of such attacks, and to provide them with tools to help them more easily evaluate and craft strong defenses for their models, we present Adagio, the first tool designed to allow interactive experimentation with adversarial attacks and defenses on an ASR model in real time, both visually and aurally. Adagio incorporates AMR and MP3 audio compression techniques as defenses, which users can interactively apply to attacked audio samples. We show that these techniques, which are based on psychoacoustic principles, effectively eliminate targeted attacks, reducing the attack success rate from 92.5% to 0%. We will demonstrate Adagio and invite the audience to try it on the Mozilla Common Voice dataset. Code related to this paper is available at: https://github.com/nilakshdas/ADAGIO.
BibTeX
@inproceedings{das2018adagio,
  author = {Nilaksh Das and Madhuri Shanbhogue and Shang-Tse Chen and Li Chen and Michael E. Kounavis and Duen Horng Chau},
  title = {{ADAGIO: Interactive Experimentation with Adversarial Attack and Defense for Audio}},
  booktitle = {ECML/PKDD},
  year = {2018},
  url = {https://openreview.net/forum?id=dJaUiQ97e3},
}
Figure