Introduction
Automatic Speech Recognition (ASR) systems frequently encounter "domain shift"—a significant drop in accuracy when the acoustic characteristics of test data (the target domain) diverge from the training data (the source domain). These shifts are typically driven by variations in speaker gender, accents, or environmental noise.
Traditional ASR pipelines rely on hand-crafted features like Mel-Frequency Cepstral Coefficients (MFCCs). However, these features may inadvertently preserve domain-specific information that hinders generalization. This paper proposes a method for unsupervised domain adaptation by learning robust features directly from raw audio waveforms using adversarial training, eliminating the need for expensive labeled data in the target domain.
Methodology: The Domain Adversarial Neural Network
The researchers employ a Domain Adversarial Neural Network (DANN) architecture, characterized by a tripartite "Y-shaped" structure:
- Feature Extractor ($G_f$): A multi-layer Convolutional Neural Network (CNN) that processes raw speech windows (310ms context) to extract high-level representations.
- Label (Senone) Classifier ($G_y$): Predicts the phonetic state (senone) of the speech, encouraging the extraction of descriptive features for recognition.
- Domain Classifier ($G_d$): Attempts to identify the domain of the input (e.g., speaker gender or accent).
The critical innovation is the Gradient Reversal Layer (GRL) situated between the feature extractor and the domain classifier. During backpropagation, the GRL multiplies the gradients from the domain classifier by a negative constant ($-\lambda$). This adversarial setup forces the feature extractor to minimize the label classification error while maximizing the domain classification error. Consequently, the network learns to strip away domain-specific traits, retaining only domain-invariant features essential for speech recognition.
Experimental Setup
The model was evaluated on two distinct speech corpora to test its robustness against common sources of variability:
- TIMIT (Gender Variability): The model was trained on male speakers (source) and adapted to female speakers (target) using unlabeled data.
- Voxforge (Accent Variability): Cross-domain adaptation was performed between American and British accents.
Key Results and Analysis
The adversarial approach consistently outperformed standard neural networks (NN) without adaptation across both datasets.
Gender Adaptation (TIMIT)
In the gender shift experiment, a standard model trained on male speech and tested on female speech yielded a Phone Error Rate (PER) of 37.20%. By applying DANN, the error rate dropped to 32.26%, representing a substantial ~5% absolute improvement.
Accent Adaptation (Voxforge)
Similar gains were observed for accent variability. When adapting from American (source) to British (target), the Word Error Rate (WER) decreased from 28.11% to 23.10%. Conversely, moving from British to American saw the WER drop from 23.37% to 23.16%.
Raw Speech vs. MFCCs
A key finding is that raw speech features are more amenable to adversarial training than traditional MFCCs. In the gender shift task, using DANN with MFCCs only reduced the error by 2.45%, whereas raw speech yielded nearly double that improvement. This suggests that raw waveforms provide the adversarial process with more "room" to filter out domain-specific nuances.
Conclusion and Future Work
This research demonstrates that ASR models can learn to ignore "distractions" like gender and accents by training directly on raw audio. While the model achieves impressive unsupervised adaptation, it faces an "invariance trade-off": improvements in the target domain often come at the cost of a slight performance dip in the source domain.
Future research could explore maintaining source accuracy during adaptation, investigating more complex shifts like high-noise environments, or utilizing deeper architectures such as ResNets or CLDNNs to further refine the quality of learned features.
Summary updated