Transfer-Representation Learning for Detecting Spoofing Attacks with Converted and Synthesized Speech in Automatic Speaker Verification System

Su-Yu Chang, Kai-Cheng Wu, Chia-Ping ChenInterspeech 2019 · DOI

Introduction

Automatic Speaker Verification (ASV) systems—the technology enabling voice-based authentication for mobile devices and banking—face an escalating threat from AI-generated spoofing. Modern Text-to-Speech (TTS) and Voice Conversion (VC) technologies have reached a level of sophistication that can easily deceive standard ASV systems.

The primary challenge in developing countermeasures is generalization. While a detector may successfully identify specific synthetic voices encountered during training, it often fails when confronted with "logical access" attacks generated by unseen algorithms. This research proposes a robust "Transfer-Representation" learning framework designed to distinguish genuine human speech (bona fide) from a wide variety of synthesized or converted speech attacks.

Proposed Methodology

The researchers developed a hybrid approach that leverages both Representation Learning and Transfer Learning to build a more resilient detector.

1. Feature Extraction

The system utilizes two types of raw acoustic features: Constant-Q Cepstral Coefficients (CQCCs) and Linear-Frequency Cepstral Coefficients (LFCCs). These provide a high-dimensional spectral view of the audio, capturing artifacts that might be invisible to standard processing.

2. Multi-Stage Learning Framework

The core of the innovation lies in a two-stage training process:

  • Stage 1 (Representation Learning): Instead of a simple binary classification, the model is first trained to perform a 7-class classification. It learns to identify specific spoofing methods (A1–A6) alongside bona fide speech. This multi-class task forces the neural network (utilizing architectures like TDNN, Inception-ResNet-v1, and Light CNN) to learn rich, discriminative "embeddings."
  • Stage 2 (Transfer and Fine-tuning): The top layers of the pre-trained networks are replaced with a 3-layer Multilayer Perceptron (MLP). The system is then fine-tuned for the final binary task: Bona fide vs. Spoof.
The Transfer-Representation learning framework, illustrating the transition from multi-class representation learning to binary fine-tuning and back-end classification.

3. Backend Classification and Fusion

Finally, 512-dimensional embeddings are extracted and passed to a Support Vector Machine (SVM) with a linear kernel to produce a final spoofing score. The "Primary System" averages scores across different architectures to improve overall reliability.

Experimental Results and Insights

The study utilized the ASVspoof 2019 Logical Access (LA) dataset, which includes both known attacks in the training set and "unseen" attacks in the evaluation set.

Key Insights

  • Categorization Improves Detection: Visualizing the learned embeddings via t-SNE reveals that training the model to recognize specific types of attacks first leads to more distinct clustering. When transitioned to the binary task, these clusters merge into clearly separable "Human" and "Machine" regions, significantly simplifying the classification task.
  • Architecture Diversity: Combining Time-Delayed Neural Networks (TDNNs) for temporal context with CNNs (Inception-ResNet) for position-invariant features yielded superior performance compared to single-architecture systems.
t-SNE visualization demonstrating the effective separation between bona fide speech and various spoofing attacks in the embedding space.

Performance Metrics

On the rigorous evaluation set containing unseen attacks, the primary system achieved:

  • min t-DCF: 0.1791
  • Equal Error Rate (EER): 9.08%

The system was particularly effective against mixed-method spoofing attacks (A13–A15), suggesting that the representation learning successfully captured general "machine-like" artifacts rather than just memorizing specific training examples.

Conclusion and Future Work

While the Transfer-Representation framework showed strong generalization, it encountered challenges with specific Voice Conversion attacks (A17 and A18) involving waveform filtering and vocoders, where error rates spiked. Future research will focus on addressing these "blind spots" and investigating methods to reduce overfitting, such as splitting training data more strictly between the representation and fine-tuning stages. Despite these challenges, the multi-stage approach represents a significant step forward in creating adaptive defenses for voice biometrics.

Summary updated

BibTeX
@inproceedings{chang2019transfer,
  author = {Su-Yu Chang and Kai-Cheng Wu and Chia-Ping Chen},
  title = {{Transfer-Representation Learning for Detecting Spoofing Attacks with Converted and Synthesized Speech in Automatic Speaker Verification System}},
  booktitle = {Interspeech},
  year = {2019},
  doi = {10.21437/interspeech.2019-2014},
}
Figure