Fully Automated End-to-End Fake Audio Detection

Chenglong Wang, Jiangyan Yi, Jianhua Tao, Haiyang Sun, Xun Chen, Zhengkun Tian, Haoxin Ma, Cunhang Fan, Ruibo Fu

Abstract

As speech synthesis and voice conversion technologies advance, the threat of deepfake audio to security and misinformation grows. Traditional detection systems rely heavily on manual feature engineering and expert-tuned neural architectures, which are often suboptimal and lack generalization. This paper presents a fully automated, end-to-end framework for fake audio detection. By integrating self-supervised feature extraction via wav2vec with a novel light-DARTS (Differentiable Architecture Search) framework, the system removes the human-in-the-loop. The proposed approach achieves state-of-the-art performance, including an Equal Error Rate (EER) of 1.08% on the ASVspoof 2019 benchmark, demonstrating superior robustness across diverse datasets and attack types.

Introduction

The rapid evolution of AI-generated audio has made it increasingly difficult to distinguish "bona-fide" human speech from synthetic counterparts. This poses significant risks to biometric authentication systems and the integrity of digital media. Current state-of-the-art detection models typically require researchers to manually select acoustic features—such as Mel-frequency cepstral coefficients (MFCCs)—and painstakingly design network architectures. This reliance on "expert experience" is a bottleneck, as manual configurations often fail to adapt to the evolving landscape of audio spoofing attacks.

To address these limitations, this research proposes a "hands-off" approach that automates both feature representation and classifier design. The goal is to create a pipeline that can autonomously learn to identify the subtle artifacts of synthetic speech without requiring domain-specific manual tuning.

Methodology

The proposed system architecture is built upon two pillars of automation: self-supervised feature learning and differentiable architecture search.

1. Self-Supervised Feature Extraction

Rather than using traditional spectral features, the system leverages pre-trained wav2vec and wav2vec 2.0 models. These models are trained on vast quantities of unlabeled raw audio using self-supervised learning objectives. This allows the models to capture high-level, robust speech representations that encapsulate the underlying structure of human phonetics. These representations are significantly more informative for detecting synthetic anomalies than hand-crafted features, which often discard phase information or fine-grained temporal details.

2. Neural Architecture Search with light-DARTS

To design the classifier, the authors utilize Differentiable Architecture Search (DARTS). DARTS transforms the discrete search for a network structure into a continuous optimization problem, allowing the system to "learn" the optimal combination of operations (such as convolutions and pooling) through gradient descent.

To optimize this for audio detection, the authors introduced light-DARTS. This framework incorporates a Max Feature Map (MFM) module directly into the search space. Inspired by Light-CNN architectures, MFM uses element-wise maximum operations to suppress low-energy noise and retain only the most competitive features. This serves as an automated internal feature selection mechanism, enabling the search process to converge on architectures that are specifically tuned to separate informative signals from synthetic artifacts.

Experimental Evaluation

The robustness of the automated system was evaluated across three major benchmarks, each representing different challenges in the deepfake landscape:

  • ASVspoof 2019 LA (Logical Access): The primary benchmark for synthetic speech and voice conversion.
  • ASVspoof 2021 DF (Deepfake): A dataset designed to test detection in real-world scenarios, including audio compression and various internet media coding artifacts.
  • ADD 2022 (Audio Deepfake Detection): A challenging set focused on low-quality audio and Mandarin speech, featuring significant background noise.

Results and Discussion

The system's performance was measured using the Equal Error Rate (EER), where a lower percentage indicates higher accuracy in balancing false alarms and missed detections.

Performance Benchmarks

  • ASVspoof 2019 LA: The system achieved an EER of 1.08%, surpassing established models like W2V-Siamese (1.15%) and RawNet2 (1.12%).
  • ASVspoof 2021 DF: The system demonstrated exceptional generalization, achieving an EER of 7.86%. This result nearly halved the error rate of the top-performing team in the original challenge (15.64%).
  • ADD 2022: The system recorded an EER of 20.11%, again outperforming the first-place competition result of 21.71%.

Key Insights

The transition from hand-crafted features (like LFCC) to wav2vec representations provided the most significant performance gain, suggesting that self-supervised models are better at capturing the "hidden" signatures of deepfakes. Furthermore, the DARTS-generated architectures consistently outperformed fixed, human-designed structures like the Light Convolutional Neural Network (LCNN), proving that automated search can find more efficient pathways for feature processing than human intuition.

Conclusion

This research demonstrates that a fully automated end-to-end pipeline can outperform manually tuned systems in the critical task of fake audio detection. By combining the representational power of wav2vec with the architectural flexibility of light-DARTS, the authors have established a new state-of-the-art that generalizes across languages and audio qualities. While the initial search phase is computationally intensive, the resulting models offer a robust defense against the growing sophistication of AI-generated speech. Future work will focus on reducing the computational cost of the architecture search and improving performance in extremely noisy environments.

Summary updated

BibTeX
@inproceedings{wang2022fully,
  author = {Chenglong Wang and Jiangyan Yi and Jianhua Tao and Haiyang Sun and Xun Chen and Zhengkun Tian and Haoxin Ma and Cunhang Fan and Ruibo Fu},
  title = {{Fully Automated End-to-End Fake Audio Detection}},
  booktitle = {DDAM@MM},
  year = {2022},
  url = {https://openreview.net/forum?id=zpl0KHdBi0},
}
Figure