TL;DR
MOS-FAD introduces a novel framework that repurposes automatic Mean Opinion Score (MOS) prediction—traditionally used to evaluate synthetic speech quality—to dramatically improve Fake Audio Detection (FAD). By leveraging MOS for both targeted training data selection and as a gating mechanism in model fusion, the system achieves a 13.6% relative reduction in Equal Error Rate (EER) over the previous state-of-the-art on challenging deepfake benchmarks.
What Problem Does This Paper Address?
Generative speech technologies like Text-to-Speech (TTS) and Voice Conversion (VC) have advanced to the point where synthetic voices sound remarkably human. While beneficial for many applications, these technologies also enable sophisticated spoofing attacks and deepfakes.
Existing Fake Audio Detection (FAD) systems typically treat this as a binary classification problem, relying on acoustic front-ends to distinguish real from fake. However, they face two major hurdles:
- Data Imbalance: Standard FAD training datasets are heavily skewed, often containing up to ten times more fake samples than real ones.
- The "Hard Case" Problem: As synthetic speech gets better, the boundary between real and fake blurs. Standard models waste capacity learning from obvious, low-quality fakes rather than focusing on the highly realistic fakes that pose the actual security threat.
Methods
The authors introduce MOS-FAD, a framework that injects external knowledge from an automatic MOS predictor into the FAD pipeline at two critical stages: data filtering and model fusion.
1. The Core Architecture
The framework utilizes 7 distinct self-supervised learning (SSL) speech models (including variants of Wav2Vec 2.0, HuBERT, and WavLM). These models are duplicated into two parallel subsystems:
- SSL-FADs: Fine-tuned to output binary FAD scores ($y_i$).
- SSL-MOSs: Fine-tuned to predict continuous speech quality scores ($z_i$) on a 1-to-5 scale, which are then combined via an MOS Fusion layer into a final quality score ($z_f$).
2. MOS-Based Data Filtering
Instead of training on the entire, imbalanced dataset, the authors use the predicted MOS scores to filter the training data. They select only the samples falling into the "uncertainty zone" (MOS between 3.0 and 4.0), effectively balancing the dataset and forcing the FAD models to train exclusively on highly realistic fakes and natural human speech.
3. Advanced Model Fusion
To combine the predictions of the 7 FAD models and the MOS predictor, the authors explored multiple fusion strategies:
- Gated MLP: Uses a linear decoder and a Sigmoid function on the MOS score to act as a dynamic "gate," controlling how much information from the FAD scores passes into the final Multi-Layer Perceptron.
- Fusion with Thresholding: A hybrid approach where samples with extremely low MOS scores ($<2.5$) are automatically flagged as fake (0), and those with extremely high scores ($>4.0$) are flagged as real (1). The Gated MLP is only invoked for the ambiguous middle ground.
Datasets
- VoiceMOS Challenge 2022 (Main Track): Used exclusively to train the automatic MOS predictor.
- ASVspoof2019 LA Track: Used for training and validating the FAD models, as well as evaluating the MOS-based data filter.
- ASVspoof2021 DF Track (Evaluation Set): A massive test set comprising 611,829 samples processed with various commercial audio codecs, used to evaluate the robustness of the final system against real-world compression and coding artifacts.
Key Insights and Ideas
- The Uncertainty Zone: By plotting the MOS distributions of real vs. fake audio, the authors discovered that fake audio rarely scores above 4.0, and real audio rarely scores below 2.5. The overlap between 3.0 and 4.0 represents the true battleground for FAD systems.
- Quality as a Proxy for Authenticity: A binary label (real/fake) throws away granular information. A continuous MOS score provides a fine-grained metric of "naturalness," which serves as an excellent prior for determining if an audio clip is a high-quality spoof.
- MOS as a Gatekeeper: In model fusion, treating the MOS score as a gating mechanism (rather than just another feature input) yielded significantly better results, proving that quality metrics can effectively modulate how acoustic classification features are interpreted.
Key Results
- Data Filtering Breakthrough: Training the base SSL models on the MOS-filtered dataset yielded massive performance gains across the board. For example, the Equal Error Rate (EER) for the standard
Wav2Vec Largemodel plummeted from 0.4794 (without the filter) to 0.1891 (with the filter). - New State-of-the-Art: The proposed Gated MLP + thresholding fusion method achieved an EER of 0.1351 on the ASVspoof2021 DF evaluation set. This comfortably beat the previous SOTA benchmark of 0.1564, representing a 13.6% relative reduction in error.
Limitations and Future Work
The authors do not explicitly dedicate a section to limitations, but a few factors are apparent from the text:
- Dependency on MOS Accuracy: The framework's success heavily relies on the robustness of the external automatic MOS predictor. If the MOS predictor fails to generalize to a new type of synthetic speech, the data filter and thresholding mechanisms could miscategorize samples.
- Future Directions: The authors note that because the "fusion with thresholding" approach is model-agnostic, it can be applied to any existing FAD architecture. Future work will likely explore integrating this framework with alternative, non-SSL-based front-ends and testing it against emerging, even more advanced generative speech algorithms.
Summary updated