Speech Replay Detection with x-Vector Attack Embeddings and Spectral Features

Jennifer Williams, Joanna RownickaInterspeech 2019

TL;DR

This paper tackles the challenge of detecting replayed speech, a common vulnerability in speaker verification systems. The authors propose a novel approach that combines traditional spectral features with "x-vector attack embeddings" derived from acoustic environments and attack types, significantly outperforming existing baselines.

What Problem Does This Paper Address?

Automatic Speaker Verification (ASV) systems are designed to authenticate individuals based on their voice. However, these systems can be tricked by "spoofing" attacks, where an attacker presents a recording of a legitimate user's voice. One prevalent type of spoofing is "replay attack," where a genuine speech segment is intercepted and played back to the ASV system. Detecting these replayed segments is crucial for robust security. The challenge lies in the fact that replay attacks can occur in diverse acoustic environments and be executed using various playback and recording devices, making it difficult for a system to generalize across these variations. This paper aims to develop a countermeasure that can reliably distinguish between live, authentic speech and replayed speech, even under these challenging and varied conditions.

Methods

The core of the proposed system lies in a novel feature combination strategy. The authors leverage two main types of features:

  • Spectral Features: They extract a range of established spectral features, including Mel-Frequency Cepstral Coefficients (MFCCs), Inverted MFCCs (IMFCCs), Rectangular Filter Cepstral Coefficients (RFCCs), Linear Frequency Cepstral Coefficients (LFCCs), Sub-band Spectral Centroid Magnitude Coefficients (SCMCs), and Constant Q Cepstral Coefficients (CQCCs). For their final submission, they found SCMC features to be particularly effective. These features are processed to have a fixed duration (10 frames) and scaled to a range of -1 to +1.
  • x-Vector Attack Embeddings: This is the novel contribution. Instead of using x-vectors for speaker identification, they are trained to capture the characteristics of different acoustic environments and attack types. They combine labels for 27 acoustic environments and 10 attack types to create 270 "env+attack" classes. A Time-Delay Neural Network (TDNN) is trained to differentiate these classes, producing 512-dimensional x-vectors. These vectors are then dimensionality-reduced to 10 dimensions using Linear Discriminant Analysis (LDA) and further scaled by a factor of 0.1. These scaled x-vectors are referred to as xEAs.

These two feature sets (SCMCs and xEAs) are concatenated and fed into a 3-layer Convolutional Neural Network (CNN) for the final classification task. The CNN is trained as a regression model, outputting a value between -1 (spoofed) and +1 (bona fide), using a hyperbolic tangent activation function. To enhance robustness, an additive Gaussian noise layer is included during training as a data augmentation technique.

Datasets

The research utilizes the ASVspoof 2019 Challenge dataset, specifically for the Physical Access (PA) task. This dataset includes labeled data for bona fide (live) speech and various types of spoofing attacks, including replay attacks, across different simulated acoustic environments and attack configurations. The authors report using 54,000 training instances and 29,700 development instances for their experiments.

Key Insights and Ideas

  • x-vectors for Environment/Attack, not Speaker: The most significant insight is the successful repurposing of x-vectors. By training them to discriminate between acoustic environments and attack types rather than speakers, they capture crucial contextual information about the spoofing scenario.
  • Complementary Features: Spectral features (like SCMC) capture the acoustic characteristics of the speech signal itself, while x-vector embeddings capture the "context" of the recording. Combining them provides a more comprehensive representation for detecting replay attacks.
  • Noise Augmentation for Robustness: The inclusion of an additive Gaussian noise layer during training proved beneficial, improving performance and making the system more robust to unseen conditions.
  • Replay Device Quality is Key: The analysis of x-vector embeddings revealed that they are better at distinguishing replay device quality than attack-to-talker distance. This suggests that the quality of the playback device is a significant factor in replay attacks.
  • Tandem Detection Cost Function (tDCF): The use of tDCF as a primary evaluation metric, in conjunction with an ASV system, provides a more realistic assessment of the countermeasure's effectiveness in a real-world scenario.

Key Results

The proposed system, combining SCMC features with scaled x-vector attack embeddings (xEAs), achieved superior performance compared to the challenge baselines.

  • Development Set: The SCMC+xEAs combination achieved a t-DCF of 0.194 and an EER of 7.74%.
  • Evaluation Set: The system achieved a t-DCF of 0.235 and an EER of 9.15%.

These results significantly outperformed the provided baselines (LFCC-GMM and CQCC-GMM), demonstrating the effectiveness of the proposed feature combination and CNN architecture.

Limitations and Future Work

  • Limitations of x-vector Discrimination: While useful, the x-vector embeddings did not perfectly discriminate between all attack and environment classes. Specifically, attack-to-talker distance was not well-captured.
  • Output Range of Regression: The use of the tanh activation function for regression output can lead to scores clustered near -1 and +1, potentially making it harder to interpret intermediate "degrees" of spoofing.
  • Future Work: The authors suggest exploring different input features for x-vector extraction, investigating optimal frequency ranges, and experimenting with various normalization techniques. They also propose further analysis of how the output scores are distributed within the -1 to +1 range for different attack types and environmental conditions.

Summary updated

BibTeX
@inproceedings{williams2019speech,
  author = {Jennifer Williams and Joanna Rownicka},
  title = {{Speech Replay Detection with x-Vector Attack Embeddings and Spectral Features}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure