AASIST3: KAN-Enhanced AASIST Speech Deepfake Detection using SSL Features and Additional Regularization for the ASVspoof 2024 Challenge

Kirill Borodin, Vasiliy Kudryavtsev, Dmitrii Korzh, Alexey Efimenko, Grach Mkrtchian, Gorodnichev Mikhail Genadievich, Oleg Y. Rogov

TL;DR

This paper introduces AASIST3, a novel deepfake speech detection system that significantly improves upon the existing AASIST framework. By integrating Kolmogorov-Arnold Networks (KANs) and leveraging self-supervised learning (SSL) features from Wav2Vec2, AASIST3 achieves a more than twofold performance improvement in detecting synthetic voices, enhancing the security of Automatic Speaker Verification (ASV) systems.

What Problem Does This Paper Address?

Automatic Speaker Verification (ASV) systems are crucial for security applications like user authentication and fraud detection. However, the rapid advancement of deep learning, particularly in Text-to-Speech (TTS) and Voice Conversion (VC) technologies, has made it increasingly easy to generate highly realistic synthetic audio. This poses a significant threat to ASV systems, as they can be fooled by these deepfakes. The paper addresses the critical need for robust anti-spoofing systems that can reliably distinguish between genuine and synthetic speech, thereby safeguarding ASV systems from malicious attacks.

Methods

The core contribution of this paper is the development of AASIST3, an enhanced version of the AASIST architecture. Key modifications include:

  • Kolmogorov-Arnold Networks (KANs): The paper integrates KANs into various attention layers (GAT, GraphPool, HS-GAL). KANs, based on the Kolmogorov-Arnold theorem, represent functions as compositions of one-dimensional functions parameterized by B-splines. This allows for more flexible and potentially more expressive feature extraction compared to traditional activation functions.
  • Wav2Vec2 SSL Features: For the open-set condition, the authors leverage Wav2Vec2, a powerful self-supervised learning model, as a feature extractor. This allows the system to learn rich audio representations from unlabeled data, which is beneficial for generalization.
  • Pre-emphasis: A pre-emphasis technique is applied to the input signal to boost high-frequency components, hypothesized to aid in distinguishing between real and synthetic speech.
  • Architectural Enhancements: The AASIST3 model incorporates additional layers, encoders, and specific graph-based operations (KAN-GAL, KAN-GraphPool, KAN-HS-GAL) to process and integrate features effectively. The architecture involves complex graph formation and attention mechanisms to capture intricate relationships within the audio data.

Datasets

The paper primarily evaluates its models on the ASVspoof 2024 Challenge dataset. Specific details about the training and validation splits are mentioned, but no new datasets are introduced. The evaluation is conducted under both "closed condition" (where training and testing data are from the same distribution) and "open condition" (where testing data may include unseen spoofing methods).

Key Insights and Ideas

  • KANs for Feature Extraction: The integration of KANs into attention mechanisms is a novel approach that demonstrates promising results in extracting more relevant features for deepfake detection. The B-spline parameterization within KANs offers a flexible way to learn complex functions.
  • SSL Features for Robustness: Utilizing Wav2Vec2 as a frontend for the open-set condition highlights the effectiveness of self-supervised learning in capturing robust audio representations, leading to better generalization against unseen spoofing attacks.
  • Pre-emphasis as a Simple Yet Effective Tool: The consistent application of pre-emphasis, even without other augmentations, proved to be an effective strategy for improving performance, underscoring the importance of high-frequency information.
  • Scaling Width vs. Depth: Experiments suggest that scaling the model's width (number of parameters in layers) is more beneficial than scaling its depth (number of layers) for improving performance in this context.

Key Results

AASIST3 demonstrates significant improvements over the baseline AASIST model:

  • Performance Boost: The paper reports a more than twofold improvement in performance.
  • minDCF Scores:
    • Closed Condition: AASIST3 achieved a minDCF of 0.5357.
  • Open Condition: AASIST3 achieved a minDCF of 0.1414.
  • These results indicate a substantial enhancement in the detection of synthetic voices and overall ASV security.

Limitations and Future Work

  • Limited Augmentation Effectiveness: While various augmentation techniques were explored, many did not lead to performance improvements and sometimes even degraded results. The paper notes that pre-emphasis was the most effective augmentation.
  • Model Complexity: The proposed AASIST3 architecture is quite complex, involving multiple KAN-based graph operations. Further research could focus on simplifying the architecture while retaining performance.
  • Generalization to Unseen Spoofs: While the open-set results are promising, continuous research is needed to ensure robust detection against novel and sophisticated deepfake generation techniques.
  • Computational Cost: The use of KANs and complex graph operations might introduce higher computational costs, which could be a consideration for real-time applications.

The paper's success with AASIST3 opens avenues for further exploration into KAN-based architectures and the strategic integration of SSL features for enhanced audio security applications.

Summary updated

Abstract

Automatic Speaker Verification (ASV) systems, which identify speakers based on their voice characteristics, have numerous applications, such as user authentication in financial transactions, exclusive access control in smart devices, and forensic fraud detection. However, the advancement of deep learning algorithms has enabled the generation of synthetic audio through Text-to-Speech (TTS) and Voice Conversion (VC) systems, exposing ASV systems to potential vulnerabilities. To counteract this, we propose a novel architecture named AASIST3. By enhancing the existing AASIST framework with Kolmogorov-Arnold networks, additional layers, encoders, and pre-emphasis techniques, AASIST3 achieves a more than twofold improvement in performance. It demonstrates minDCF results of 0.5357 in the closed condition and 0.1414 in the open condition, significantly enhancing the detection of synthetic voices and improving ASV security.
BibTeX
@misc{borodin2024aasist,
  author = {Kirill Borodin and Vasiliy Kudryavtsev and Dmitrii Korzh and Alexey Efimenko and Grach Mkrtchian and Gorodnichev Mikhail Genadievich and Oleg Y. Rogov},
  title = {{AASIST3: KAN-Enhanced AASIST Speech Deepfake Detection using SSL Features and Additional Regularization for the ASVspoof 2024 Challenge}},
  year = {2024},
  eprint = {2408.17352},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.17352},
}
Figure