Attentive Filtering Networks for Audio Replay Attack Detection

Cheng-I Lai, Alberto Abad, Korin Richmond, Junichi Yamagishi, Najim Dehak, Simon King

TL;DR

This paper introduces the Attentive Filtering Network (AFN), a novel deep learning system for detecting audio replay attacks in speaker verification systems. By combining an attention-based filtering mechanism with a Dilated Residual Network (DRN) classifier, AFN effectively enhances discriminative features in both time and frequency domains, achieving a competitive 8.99% Equal Error Rate (EER) on the ASVspoof 2017 dataset.

What Problem Does This Paper Address?

Automatic Speaker Verification (ASV) systems, designed to authenticate individuals based on their voice, are increasingly vulnerable to "presentation attacks," commonly known as spoofing. Replay attacks, a specific type of spoofing, involve an impostor playing back a recording of a genuine user's voice to fool the ASV system. The ASVspoof 2017 Challenge aimed to address this by focusing on replay attack detection and developing robust countermeasures. Existing methods often struggle with the subtle and sometimes time-varying nature of spoofing cues, and generalizing to unseen conditions due to limited training data. This paper tackles the challenge of building a more robust and interpretable system for detecting replay attacks by intelligently focusing on the most informative parts of the audio features.

Methods

The core of the proposed system is the Attentive Filtering Network (AFN), which comprises two main components:

  • Attentive Filtering (AF): This mechanism acts as a pre-processor to the main classifier. It learns an "attention heatmap" that selectively enhances or suppresses features in the input spectrogram. The motivation is that spoofing cues might be embedded in specific time-frequency regions, and the attention mechanism helps the network focus on these crucial areas. The attention heatmap is learned using a U-net-like structure, inspired by image segmentation networks, and is applied to the input feature map.
  • Dilated Residual Network (DRN): This is the classifier component. It's a ResNet-based architecture enhanced with dilated convolutions. Dilated convolutions allow the network to have a larger receptive field without increasing the number of parameters or losing resolution, which is crucial for capturing broader context in the audio features and mitigating overfitting on small datasets. The DRN is composed of five Dilated Residual Modules (DRMs), each containing a residual unit, max-pooling, and a dilated convolution layer.

The system is trained end-to-end, with the AF learning to guide the DRN's focus. The authors also explored different non-linear activation functions (Sigmoid, Tanh, Softmax) for the attention mechanism, finding that Sigmoid yielded the best performance for single systems.

Datasets

The experiments were conducted on the ASVspoof 2017 Version 2.0 dataset. This dataset includes:

  • Training set: 1507 replay and 1507 bona fide files.
  • Development (dev) set: 950 replay and 760 bona fide files.
  • Evaluation (eval) set: 12008 replay and 1298 bona fide files.

The authors also compared their results with prior work on the ASVspoof 2017 Version 1.0 dataset.

Key Insights and Ideas

  • Interpretability through Attention: Visualizing the learned attention heatmaps provides valuable insights into why the network makes certain decisions. The authors observed that the attention mechanism tends to focus on high-frequency components of speech, which aligns with findings in related literature, suggesting the network is learning meaningful acoustic cues for spoofing detection.
  • The Power of Selective Feature Enhancement: By allowing the network to "attend" to specific parts of the feature map, the AF mechanism helps the DRN classifier focus on discriminative information, rather than being overwhelmed by less relevant parts of the audio. This is particularly useful when dealing with limited and unbalanced datasets.
  • Dilated Convolutions for Robustness: The use of dilated convolutions in the DRN is a key strategy to combat overfitting. By expanding the receptive field without downsampling, the network can learn from a wider context, leading to better generalization on unseen conditions, a common challenge in ASVspoof datasets.
  • Non-linearities Matter: The choice of non-linear activation function for the attention mechanism significantly impacts performance. The authors found that Sigmoid, which scales features independently, was more effective than Softmax-based activations for this task, suggesting that preserving the relative scale of different features is important for replay attack detection.

Key Results

  • Single System Performance: The Attentive Filtering Network (AFN) achieved a competitive 8.99% Equal Error Rate (EER) on the ASVspoof 2017 Version 2.0 evaluation set using the Sigmoid activation for AF and a DRN with ReLU.
  • System Fusion: By fusing multiple AF systems with different non-linearities, the authors achieved an even better 8.54% EER on the evaluation set. This represents a 30% relative improvement over the ASVspoof 2017 enhanced baseline system.
  • Generalization: The small differences between dev and eval EERs for their proposed systems indicate good generalization capabilities, even with the relatively small and unbalanced dataset.
  • Comparison to Baselines: The proposed AFN system significantly outperforms several baseline methods, including CQCC-GMM, i-vectors, and a re-implemented LCNN, particularly on the Version 2.0 dataset.

Limitations and Future Work

  • Dataset Size: While the use of dilated convolutions helps, the authors acknowledge that the ASVspoof 2017 dataset is relatively small, and their approach might underfit if not carefully managed.
  • Fusion Complexity: While system fusion improved performance, it adds complexity. Future work could explore more efficient fusion strategies or end-to-end training of fused components.
  • Interpretability of Fusion: While single AFN heatmaps are interpretable, understanding the combined attention from fused systems might require further investigation.
  • Exploring Other Attack Types: The current work focuses on replay attacks. Future research could extend the AFN to detect other types of spoofing attacks (e.g., speech synthesis, voice conversion).

Summary updated

Abstract

An attacker may use a variety of techniques to fool an automatic speaker verification system into accepting them as a genuine user. Anti-spoofing methods meanwhile aim to make the system robust against such attacks. The ASVspoof 2017 Challenge focused specifically on replay attacks, with the intention of measuring the limits of replay attack detection as well as developing countermeasures against them. In this work, we propose our replay attacks detection system - Attentive Filtering Network, which is composed of an attention-based filtering mechanism that enhances feature representations in both the frequency and time domains, and a ResNet-based classifier. We show that the network enables us to visualize the automatically acquired feature representations that are helpful for spoofing detection. Attentive Filtering Network attains an evaluation EER of 8.99% on the ASVspoof 2017 Version 2.0 dataset. With system fusion, our best system further obtains a 30% relative improvement over the ASVspoof 2017 enhanced baseline system.
BibTeX
@inproceedings{lai2019attentive,
  author = {Cheng-I Lai and Alberto Abad and Korin Richmond and Junichi Yamagishi and Najim Dehak and Simon King},
  title = {{Attentive Filtering Networks for Audio Replay Attack Detection}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=brSHZgr2pIt},
}
Figure