Abstract
The proliferation of high-fidelity synthetic speech necessitates robust forensic tools for both detection and attribution. This paper introduces Syn-Att, a framework designed to attribute synthetic audio to its generating algorithm. By employing a multi-class ensemble of Convolutional Neural Networks (CNNs) and a novel semi-supervised strategy for handling "Unknown" generation sources, Syn-Att addresses the open-set problem in audio forensics. Experimental results demonstrate that Syn-Att significantly outperforms state-of-the-art competitors, particularly under heavy audio distortion.
Introduction
While binary detection (real vs. fake) is a common focus in audio forensics, attribution—identifying the specific source algorithm—is crucial for tracking malicious content. However, traditional classifiers often fail when encountering "open-set" scenarios where the test audio comes from an unseen generator. Syn-Att aims to bridge this gap by creating a system capable of identifying five known algorithms while accurately categorizing unseen sources into an "Unknown" class.
Methodology
The Syn-Att pipeline is a multi-stage process that transforms raw audio into actionable forensic intelligence.
Feature Extraction and Data Augmentation
Raw audio is converted into log-mel spectrograms. To handle varying noise levels, two configurations were used: a standard resolution for clean audio and a higher-resolution version for noisy environments. Data augmentation techniques like MixUp, CutMix, and SpecAugment were applied to ensure the model focuses on algorithmic artifacts rather than speaker-specific characteristics.
The "Unknown" Class Strategy
To mitigate "hallucination" where the model incorrectly assigns unseen algorithms to known classes, the authors explicitly trained an "Unknown" category. This class was populated using a diverse mix of natural speech (from LJSpeech, LibriSpeech, and VCTK) and synthetic audio from external text-to-speech models.
Ensemble of CNNs and Semi-Supervised Learning
Syn-Att utilizes an ensemble of diverse CNN backbones, including ResNet, EfficientNet, and ConvNeXt. The final prediction is an average of the probability outputs, enhancing resilience against perturbations. Furthermore, a semi-supervised approach using pseudo-labeling allows the model to adapt to the specific distribution of the test data without requiring ground-truth labels.
Experimental Results
The framework was evaluated against the IEEE SP Cup 2022 benchmarks, which include weakly perturbed (Eval 1) and strongly perturbed (Eval 2) datasets.
As shown in the comparative analysis, Syn-Att achieves a significant performance lead. On the challenging Eval 2 dataset, it outperformed the next best competitors by 12-13% in accuracy. On the cleaner Eval 1 dataset, it maintained a high accuracy of 98%. The results highlight the effectiveness of the ensemble approach and the "Unknown" class strategy in handling complex, real-world audio distortions.
Conclusion
Syn-Att provides a robust solution for the forensic attribution of synthetic speech. By combining diverse CNN architectures with a strategic approach to open-set classification and semi-supervised adaptation, it sets a new benchmark for accuracy and reliability in the field. Future work will explore reducing computational overhead and integrating transformer-based architectures for further performance gains.
Summary updated