Unsupervised Adaptation with Adversarial Dropout Regularization for Robust Speech Recognition

Pengcheng Guo, Sining Sun, Lei XieInterspeech 2019 · DOI

Introduction

Automatic Speech Recognition (ASR) systems frequently encounter performance degradation when deployed in environments that differ from their training conditions—a phenomenon known as "domain mismatch." While Domain Adversarial Training (DAT) is a common approach to align source and target data distributions, it often fails to account for class boundaries. This can lead to target-domain features residing near decision boundaries, making the system prone to misclassification. This paper proposes Adversarial Dropout Regularization (ADR), a method that leverages dropout to identify and avoid ambiguous decision regions, significantly improving robustness in noisy environments.

Methodology

The ADR framework moves beyond simple distribution matching by using the classifier's own decision logic to guide feature alignment. The architecture consists of a feature generator ($G$) and a senone classifier ($C$). The training follows a three-step adversarial process:

  1. Supervised Pre-training: $G$ and $C$ are initialized using labeled source-domain data (clean speech).
  2. Boundary Detection (Update $C$): Using unlabeled target data, the system performs two forward passes through the classifier with different dropout masks, creating two distinct "views" ($C_1$ and $C_2$). The classifier is updated to maximize the discrepancy between these views, effectively highlighting regions of uncertainty.
  3. Feature Robustness (Update $G$): The feature generator is then updated to minimize this discrepancy. This forces $G$ to produce representations that are invariant to dropout perturbations, pushing target features away from decision boundaries and into high-confidence discriminative regions.
The ADR architecture uses a feature generator G and two classifier views (C1, C2) to measure and minimize prediction discrepancy on target data.

Experimental Setup

The proposed method was evaluated using the CHiME-3 dataset, a benchmark for ASR in noisy real-world settings. The source domain consisted of clean speech from the Wall Street Journal (WSJ0) corpus, while the target domain included noisy speech recorded in four environments: bus, cafe, pedestrian area, and street. The model was tested on both simulated and real noisy utterances.

Key Results and Discussion

ADR demonstrated superior performance compared to both the baseline (no adaptation) and standard Gradient Reversal Layer (GRL) adversarial techniques.

  • Error Reduction: The method achieved a 12.9% relative reduction in Word Error Rate (WER) on simulated noisy data and a 9.3% reduction on real noisy data.
  • Comparative Performance: ADR consistently outperformed GRL-based adaptation, recording a 40.26% WER versus 40.98% for GRL on simulated data.
  • Feature Alignment: Visualization of the feature space confirms the effectiveness of the regularization. As shown in the t-SNE plots, ADR produces tighter, more distinct clusters for the target domain compared to the baseline model, indicating that features have been successfully moved away from ambiguous zones.
t-SNE visualization comparing the Baseline Model and the ADR Model, demonstrating that ADR creates more distinct and tighter class clusters.

Conclusion

Adversarial Dropout Regularization provides a robust mechanism for unsupervised domain adaptation by specifically targeting the vulnerability of features near decision boundaries. By using dropout as a probe for uncertainty, ADR ensures that adapted features are not just aligned in distribution, but are also highly discriminative. Future work will explore integrating domain-private information and extending the ADR framework to end-to-end ASR architectures.

Summary updated

BibTeX
@inproceedings{guo2019unsupervised,
  author = {Pengcheng Guo and Sining Sun and Lei Xie},
  title = {{Unsupervised Adaptation with Adversarial Dropout Regularization for Robust Speech Recognition}},
  booktitle = {Interspeech},
  year = {2019},
  doi = {10.21437/interspeech.2019-2544},
}
Figure