Abstract
Speaker Verification (SV) systems are highly sensitive to environmental noise. This paper introduces an Adversarial Network Bottleneck (AN-BN) feature extraction framework designed to produce noise-invariant representations. By training an encoding network to deceive a noise-type discriminator, the system learns to suppress noise-specific patterns. Experimental results on the RSR2015 dataset demonstrate that AN-BN features significantly outperform traditional and deep-learning-based speech enhancement methods, even improving performance in clean-speech scenarios.
Introduction
Confirming a person's identity via voice—Speaker Verification—is a critical task for secure mobile and remote applications. However, real-world performance is often degraded by additive noise such as babble, traffic, or electronic interference. Traditional front-end solutions like Wiener filters or spectral subtraction often introduce artifacts that hinder downstream verification. This research proposes moving beyond signal reconstruction toward learning features that are inherently robust to environmental variations.
Methodology: The Adversarial Framework
The proposed system utilizes an Adversarial Network Bottleneck (AN-BN) architecture. It consists of two primary components: an Encoding Network (EN) and a Discriminative Network (DN).
The EN takes 11 frames of Mel-frequency cepstral coefficients (MFCCs) and maps them to a 128-dimensional bottleneck layer. The DN is trained to classify the noise type (e.g., White, Babble, Market) present in these features. In an adversarial setup, the EN is trained with a "deceptive" objective: it must produce features that the DN classifies as "clean," regardless of the actual input noise. This competition forces the EN to strip away noise-specific information, retaining only the underlying speech characteristics.
Experimental Setup
The framework was evaluated using the RSR2015 database (male speakers). The Universal Background Model (UBM) was trained on the TIMIT dataset. Five noise types were considered: Babble, Cantine, Market, Airplane, and White Gaussian noise. To simulate realistic conditions, training and testing were conducted using different mobile devices (Samsung Nexus, Samsung Galaxy S, and HTC Desire) to account for microphone variability.
Results and Discussion
The AN-BN features were compared against several baselines: standard MFCCs (No Enh.), statistical enhancement (STSA-MMSE), and deep neural network-based speech enhancement (DNN-SE).
As shown in the results table, the AN-BN approach (labeled NS-AN and NG-AN) consistently achieves the lowest Equal Error Rates (EER). Key observations include:
- Robustness to High Noise: In low SNR environments (0dB to 5dB), the proposed features reduced EER by approximately 25% for White and Babble noise and up to 40% for other types compared to DNN-SE.
- Clean Condition Gains: A notable finding was the improvement in clean-speech performance. By training the EN on clean speech from multiple sessions with a single "clean" label, the network learned to reduce session-to-session variability, a common challenge in SV.
- Generalization: The system demonstrated strong performance on SNRs not explicitly seen during the training phase, indicating high generalization capability.
Conclusion
The Adversarial Network Bottleneck framework offers a powerful method for extracting noise-invariant features for speaker verification. By focusing on invariance rather than signal reconstruction, the system achieves state-of-the-art robustness across diverse environmental conditions. Future research will explore the applicability of these features to Automatic Speech Recognition (ASR) and investigate dimensionality reduction techniques to optimize computational efficiency.
Summary updated