Introduction
State-of-the-art Automatic Speech Recognition (ASR) systems typically require hundreds of hours of transcribed speech to achieve high performance. However, in low-resource scenarios—such as rare languages or niche domains—collecting such vast amounts of data is often unfeasible. Standard acoustic models frequently fail in these settings because they cannot adequately capture the complex variations in speech, channel noise, and speaker characteristics from limited samples. This paper explores a method to "borrow" knowledge from high-resource datasets to enhance low-resource systems using data-driven posterior features.
Methodology
The core innovation lies in the use of Multi-Layer Perceptrons (MLPs) to generate robust features that are more discriminative than standard raw acoustic signals like Perceptual Linear Prediction (PLP) features.
Monolingual Front-end
In the monolingual approach, an MLP is trained on a larger dataset from the same language but a different genre. For instance, training on the Switchboard corpus to recognize Callhome English. The MLP learns to map acoustic inputs to phoneme probabilities (posteriors). These posteriors are then used as features for the target low-resource task.
Cross-lingual and Multilingual Front-end
The authors extend this concept to cross-lingual transfer. By training MLPs on high-resource languages (e.g., 200 hours of Spanish) that share similar acoustic channel conditions (telephone speech), the system can extract fundamental acoustic-phonetic patterns that transcend specific languages.
The "Tandem" technique is employed here: phoneme posteriors from the multilingual MLPs are Gaussianized and decorrelated. These enhanced features are then combined with target-language features to create a highly robust representation that suppresses unwanted variability like speaker identity and channel noise.
Experimental Setup
The study evaluates these features on two primary tasks:
- Low-resource LVCSR: Training a recognizer on just 1 hour of Callhome English (spontaneous telephone conversations).
- Spoken Term Discovery: A zero-resource task requiring the system to find repeated words in untranscribed audio using Dynamic Time Warping (DTW).
Source data included up to 20 hours of English Switchboard for the monolingual front-end and 200 hours of Spanish Switchboard/Callhome for the cross-lingual front-end.
Key Results
The experimental results demonstrate significant improvements over baseline features:
- ASR Accuracy: Standard PLP features on 1 hour of data yielded 28.8% word accuracy. The monolingual front-end improved this to 37.2%, while the cross-lingual front-end (using 200 hours of Spanish) reached 41.5%.
- Efficiency: Using just 1 hour of data with these data-driven features performs comparably to using 5-10 hours of data with standard features.
- Zero-resource Performance: Posterior features significantly outperformed raw acoustic features in "Average Precision" for spoken term discovery, proving their superior discriminative power in environments without any transcriptions.
Conclusion
This research highlights that neural networks trained on large, task-independent datasets—even from different languages—can create robust feature representations for low-resource ASR. The key requirement is matching acoustic channel conditions rather than language or genre. Future work may involve adapting these multilingual MLPs directly to the target language to further bridge the resource gap.
Summary updated