Le comportement des systèmes de reconnaissance du locuteur de l'état de l'art face aux variabilités acoustiques

Mohammad Mohammadamini, Driss Matrouf, Sandipana Dowerah, Romain Serizel, Denis Jouvet, Jean-François Bonastre2022 · DOI

TL;DR

This paper investigates the robustness of state-of-the-art speaker recognition systems (TDNN and ResNet) against acoustic variabilities like noise and reverberation. It finds that ResNet is inherently more robust than TDNN, and surprisingly, noise compensation techniques that benefit TDNN offer little to no improvement for ResNet, which already performs significantly better.

What Problem Does This Paper Address?

Speaker recognition systems aim to identify individuals based on their voice. A critical challenge in this field is ensuring these systems perform reliably in real-world scenarios, which are often plagued by acoustic variabilities such as background noise and reverberation. These distortions can significantly degrade the quality of speech signals, making it difficult for systems to extract accurate speaker-specific information. While various deep learning architectures like TDNNs (Time Delay Neural Networks) and ResNets (Residual Networks) have shown promise, their comparative robustness against these common acoustic challenges, and the effectiveness of compensation techniques, remains an important area of investigation. This paper specifically addresses the question of how well current state-of-the-art systems, particularly TDNNs and ResNets, handle additive noise and reverberation, and whether existing compensation methods can bridge the performance gap.

Methods

The researchers compare two prominent deep learning architectures for speaker recognition: TDNN and ResNet.

  • TDNN: This architecture, based on previous work (Snyder et al., 2018), uses time-delayed layers to capture temporal context in speech. It also incorporates statistical pooling and segment-level layers.
  • ResNet: A variant of the ResNet architecture (Zeinali, 2019) is employed. This network is known for its ability to train very deep models by using residual blocks, which allow gradients to flow more easily. For speaker recognition, it processes acoustic features and aims to optimize a speaker classification cross-entropy loss.

The study evaluates the performance of these systems under various acoustic conditions. They also investigate the effectiveness of noise compensation techniques, specifically DAE (Denoising Auto-Encoder) and i-MAP (a statistical approach), applied at the embedding level. To understand the underlying behavior, they utilize t-SNE (t-Distributed Stochastic Neighbor Embedding) for visualizing the extracted speaker embeddings.

Datasets

The paper uses the following datasets:

  • Voxceleb 1 & 2: Used for training the deep learning models (embeddings extraction). Data augmentation with noise and reverberation was applied to these datasets, resulting in approximately 5 million recordings for training.
  • MUSAN: Used for data augmentation, specifically for adding noise to the training data.
  • RIR (Room Impulse Response): Used for simulating reverberation in the training data.
  • Fabiol: Used for evaluating system robustness against simulated noise and reverberation. It includes 130 clients, with 30 used for testing (6870 test files).
  • Voices: Used for evaluating robustness against real-world noise and reverberation. This dataset has separate training and testing partitions derived from Librispeech. The test set consists of 1320 clean files from 100 speakers, and the training set has 2583 files from 200 speakers. For experiments with this corpus, recordings from a distant microphone ("mic 05") and rooms with higher reverberation ("room2", "room3") were specifically chosen.

Key Insights and Ideas

  • ResNet's Intrinsic Robustness: The most striking finding is that ResNet, even without any explicit noise compensation, is significantly more robust to acoustic variabilities (noise and reverberation) than TDNN. This suggests that the architectural design of ResNet inherently captures more resilient speaker representations.
  • Compensation Ineffectiveness for ResNet: Counter-intuitively, the noise compensation techniques (DAE and i-MAP) that proved effective for TDNN offered minimal to no improvement for ResNet. This implies that the compensation methods might rely on assumptions that are not met by ResNet's embeddings, or that ResNet's embeddings are already so robust that further compensation yields negligible gains.
  • TDNN Benefits from Compensation: For TDNN, compensation techniques significantly reduce the Equal Error Rate (EER), bringing its performance closer to that of clean speech. This highlights the importance of these techniques for TDNN-based systems.
  • Visualizing Embeddings Reveals Differences: t-SNE visualizations clearly show that for TDNN, clean and noisy embeddings form distinct clusters, suggesting a clear separation that compensation can help bridge. In contrast, ResNet's clean and noisy embeddings are highly intertwined, visually supporting its inherent robustness.
  • ResNet Outperforms TDNN Even with Compensation: Even when TDNN is enhanced with compensation techniques, ResNet (without compensation) consistently achieves superior performance. This reinforces the idea that ResNet's architecture is a key factor in its robustness.

Key Results

  • Baseline Performance: In clean conditions, ResNet achieved an EER of 6.27% on Fabiol and 0.89% on Voices, while TDNN achieved 15.21% and 1.25%, respectively.
  • Impact of Noise: With additive noise (e.g., SNR 0-5 on Fabiol), ResNet's EER was 8.31%, while TDNN's was 18.15%.
  • Impact of Reverberation: On Fabiol with reverberation, ResNet's EER was 9.75%, compared to TDNN's significantly higher EER (not explicitly stated but implied to be much worse than ResNet).
  • Combined Noise and Reverberation: In the worst-case scenario (Fabiol, SNR 0-5 with reverberation), ResNet's EER was 12.48%, while TDNN's was 21.47%.
  • Compensation Results: For TDNN, compensation reduced the EER from 21.47% to 18.00% in the combined noise/reverberation scenario on Fabiol. For ResNet, the EER only slightly improved from 12.48% to 12.18% under the same conditions.

Limitations and Future Work

  • Understanding ResNet's Robustness: A key limitation acknowledged is the lack of a clear explanation for why ResNet is so robust. The authors suggest future work should focus on understanding this behavior.
  • Compensation Assumptions: They hypothesize that the assumptions underlying current compensation techniques might not hold for ResNet-extracted x-vectors. Future research could explore whether imposing these assumptions during the training of ResNet's x-vector extractor could yield better results.
  • Explaining Compensation Ineffectiveness: Further investigation is needed to understand why compensation methods do not benefit ResNet, and whether small improvements are possible if specific hypotheses are met.

Summary updated

BibTeX
@misc{mohammadamini2022le,
  author = {Mohammad Mohammadamini and Driss Matrouf and Sandipana Dowerah and Romain Serizel and Denis Jouvet and Jean-François Bonastre},
  title = {{Le comportement des systèmes de reconnaissance du locuteur de l'état de l'art face aux variabilités acoustiques}},
  year = {2022},
  doi = {10.21437/jep.2022-26},
}
Figure