Abstract
As an important identity authentication method, speaker verification (SV) has been widely used in many domains, such as mobile financials. However, existing SV systems are vulnerable to replay spoofing attacks. To develop a more secure and stable SV system, this article proposes a replay attack detection system based on deformable convolutional neural networks (DCNNs) and a time–frequency double-channel attention model. In the DCNN, the positions of elements in the convolutional kernel are modified by trainable variables, enabling the model to extract more useful local information from input spectrograms. Simultaneously, a time–frequency domino double-channel attention model is adopted to extract effective distinctive features for distinguishing genuine and replayed speech. Experimental results on the ASVspoof 2019 dataset demonstrate that the proposed model detects replay attacks with high accuracy.
Introduction
Speaker Verification (SV) systems are increasingly integrated into daily life, from smartphone security to banking authorizations. Despite their convenience, these systems face significant security threats from spoofing, particularly replay attacks. In a replay attack, an adversary records a target user’s voice and plays it back to the microphone.
While replayed speech is perceptually similar to natural speech, it contains subtle "artifacts" introduced by the recording device, the playback hardware, and the acoustic environment. Standard SV systems often struggle to distinguish these subtle differences. This research proposes a robust "front-end" detector designed to capture these irregular acoustic fingerprints before they reach the main verification system.
Proposed Methodology
The proposed system is built upon a modified ResNet-34 architecture, incorporating two primary innovations: Deformable Convolutional Neural Networks and a Temporal-Frequency Attention Model.
Deformable Convolutional Neural Networks (DCNN)
Standard convolutional neural networks use fixed, rectangular grids (kernels) to scan spectrograms. However, the acoustic artifacts of a replay attack are often irregular and not perfectly aligned with a rigid grid. DCNNs address this by adding learnable offsets ($dx, dy$) to the kernel positions. This allows the network to "stretch" and "warp" its filters to better capture irregular local information in the input spectrograms.
Temporal-Frequency Attention Model (TF-AM)
While many audio attention mechanisms focus solely on the time dimension, this paper argues that the frequency dimension is equally vital for spotting replay artifacts. The TF-AM uses a "double-channel" approach:
- Frequency Attention: Pools data to create a weight template ($W_f$) that highlights specific frequency bands critical for artifact detection.
- Time Attention: Creates a weight template ($W_t$) to highlight specific moments in the audio signal.
By applying frequency attention followed by time attention in a "domino" fashion, the model progressively filters out irrelevant noise, focusing strictly on the most discriminative parts of the signal.
Experimental Results
The model was evaluated using the ASVspoof 2019 dataset, which includes natural and replayed speech recorded across nine different environments with varying equipment quality. The test set contains over 150,000 audio samples.
The proposed model (ResNet + DCNN + TF-AM) significantly outperformed baseline models on the ASVspoof 2019 evaluation set, achieving a state-of-the-art Equal Error Rate (EER) of 1.03%.
| Model | t-DCF (Lower is better) | EER (%) (Lower is better) |
|---|---|---|
| GMM + LFCC (Baseline) | 0.302 | 13.54% |
| Standard ResNet | 0.068 | 1.97% |
| LCNN | 0.079 | 2.03% |
| Proposed (DCNN + TF-AM) | 0.025 | 1.03% |
Conclusion
The integration of deformable kernels and dual-channel attention provides a nearly 50% improvement over standard ResNet architectures. This suggests that spectrograms, while treated as images, benefit from the fluid, shifting nature of deformable filters which adapt to physical signal characteristics. Future work will focus on automating hyperparameter tuning through meshed optimization and testing the system in more diverse real-world scenarios, such as public security and judicial applications.
Summary updated