On the human evaluation of audio adversarial examples

Jon Vadillo, Roberto SantanaarXiv 2020 · OpenReview

Abstract

Human-machine interaction is increasingly dependent on speech communication. Machine Learning models are usually applied to interpret human speech commands. However, these models can be fooled by adversarial examples, which are inputs intentionally perturbed to produce a wrong prediction without being noticed. While much research has been focused on developing new techniques to generate adversarial perturbations, less attention has been given to aspects that determine whether and how the perturbations are noticed by humans. This question is relevant since high fooling rates of proposed adversarial perturbation strategies are only valuable if the perturbations are not detectable. In this paper we investigate to which extent the distortion metrics proposed in the literature for audio adversarial examples, and which are commonly applied to evaluate the effectiveness of methods for generating these attacks, are a reliable measure of the human perception of the perturbations. Using an analytical framework, and an experiment in which 18 subjects evaluate audio adversarial examples, we demonstrate that the metrics employed by convention are not a reliable measure of the perceptual similarity of adversarial examples in the audio domain.

Introduction

In the field of adversarial machine learning, an attack is considered truly successful only if it remains "imperceptible" to human observers while causing a model to fail. In the audio domain, researchers typically rely on mathematical metrics such as Signal-to-Noise Ratio (SNR) or Decibel (dB) levels to quantify this stealthiness. However, a significant gap exists between these mathematical thresholds and actual human psychoacoustics. A perturbation may have a low decibel level relative to the entire file, yet remain highly audible if concentrated in silent gaps between words.

Conceptual overview of audio adversarial attacks: a small perturbation causes a DNN to misclassify while a human still perceives the original command.

This study investigates whether standard metrics used in the literature actually correlate with human perception. The core problem is that "mathematically stealthy" attacks are often glaringly obvious to the human ear, suggesting that current evaluation frameworks are insufficient.

Methodology

The researchers employed a dual approach involving analytical modeling and empirical human testing using the Google Speech Command Dataset (covering 12 classes like "Yes," "No," "Go," etc.).

  1. Attack Generation: The study used "single-class universal perturbations" (UAP-HC based on the Deepfool algorithm). These are static noise patterns that, when added to any audio of a specific class, trigger misclassification by a Deep Neural Network (DNN).
  2. Signal-Part Analysis: To better model perception, the authors proposed splitting signals into a vocal part (speech energy) and a background part (silence/ambient noise). This distinguishes between noise masked by speech and noise exposed in silence.
  3. Human Experiment: 18 subjects participated in two tests:
  4. Naturalness Assessment: Rating clips on a 1–5 scale from "clearly artificial" to "clearly natural."
    • ABX Testing: A psychoacoustic test where subjects must identify which of two samples (clean or perturbed) matches a third random sample, measuring if a difference is detectable.

Results and Discussion

The attacks were highly effective against the DNN, often achieving fooling rates exceeding 70%. While humans could still understand the intended command with approximately 94% accuracy, they almost always detected the presence of tampering.

Metric Failure and Human Detection

Standard metrics are often "uninformed," averaging distortion over the entire file. The study found that perturbations frequently exceed audibility thresholds in the background portions of a clip, even if they are quiet relative to vocal peaks. In low and medium-intensity audio, humans detected perturbations with 97.2% and 91.7% accuracy, respectively. Only in high-intensity audio, where the signal masks the noise, did detection drop to near-random levels (55.6%).

Distribution of the $dB_{x,max}(v)$ metric across different command classes, highlighting the variability in distortion levels that humans often find detectable.

Commonly cited thresholds for imperceptibility (e.g., $dB_{x,max} < -32dB$) proved insufficient. Humans consistently flagged samples as containing "artificial noise" even when they met these mathematical criteria. Clean samples were rated as natural 65.5% of the time, compared to only 33% for adversarial samples.

Conclusion

The findings demonstrate that current audio adversarial attacks are only truly stealthy when the underlying signal is very loud. Standard mathematical metrics fail to capture the nuances of human hearing, particularly the lack of masking in silent intervals. Future work should focus on developing psychoacoustically-informed metrics that better model the human auditory system and investigating whether individual perturbations (tailored to specific files) can achieve higher levels of imperceptibility than universal ones.

Summary updated

Abstract

Human-machine interaction is increasingly dependent on speech communication. Machine Learning models are usually applied to interpret human speech commands. However, these models can be fooled by adversarial examples, which are inputs intentionally perturbed to produce a wrong prediction without being noticed. While much research has been focused on developing new techniques to generate adversarial perturbations, less attention has been given to aspects that determine whether and how the perturbations are noticed by humans. This question is relevant since high fooling rates of proposed adversarial perturbation strategies are only valuable if the perturbations are not detectable. In this paper we investigate to which extent the distortion metrics proposed in the literature for audio adversarial examples, and which are commonly applied to evaluate the effectiveness of methods for generating these attacks, are a reliable measure of the human perception of the perturbations. Using an analytical framework, and an experiment in which 18 subjects evaluate audio adversarial examples, we demonstrate that the metrics employed by convention are not a reliable measure of the perceptual similarity of adversarial examples in the audio domain.
BibTeX
@misc{vadillo2020human,
  author = {Jon Vadillo and Roberto Santana},
  title = {{On the human evaluation of audio adversarial examples}},
  year = {2020},
  eprint = {2001.08444},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2001.08444},
}
Figure