Data-driven Posterior Features for Low Resource Speech Recognition Applications

Samuel Thomas, Sriram Ganapathy, Aren Jansen, Hynek HermanskyInterspeech 2012 · DOI

Introduction

State-of-the-art Automatic Speech Recognition (ASR) systems typically require hundreds of hours of transcribed speech to achieve high performance. However, in low-resource scenarios—such as rare languages or niche domains—collecting such vast amounts of data is often unfeasible. Standard acoustic models frequently fail in these settings because they cannot adequately capture the complex variations in speech, channel noise, and speaker characteristics from limited samples. This paper explores a method to "borrow" knowledge from high-resource datasets to enhance low-resource systems using data-driven posterior features.

Methodology

The core innovation lies in the use of Multi-Layer Perceptrons (MLPs) to generate robust features that are more discriminative than standard raw acoustic signals like Perceptual Linear Prediction (PLP) features.

Monolingual Front-end

In the monolingual approach, an MLP is trained on a larger dataset from the same language but a different genre. For instance, training on the Switchboard corpus to recognize Callhome English. The MLP learns to map acoustic inputs to phoneme probabilities (posteriors). These posteriors are then used as features for the target low-resource task.

Monolingual data-driven front-end architecture for low-resource speech recognition.

Cross-lingual and Multilingual Front-end

The authors extend this concept to cross-lingual transfer. By training MLPs on high-resource languages (e.g., 200 hours of Spanish) that share similar acoustic channel conditions (telephone speech), the system can extract fundamental acoustic-phonetic patterns that transcend specific languages.

The "Tandem" technique is employed here: phoneme posteriors from the multilingual MLPs are Gaussianized and decorrelated. These enhanced features are then combined with target-language features to create a highly robust representation that suppresses unwanted variability like speaker identity and channel noise.

Cross-lingual front-end architecture incorporating multilingual posteriors and tandem processing.

Experimental Setup

The study evaluates these features on two primary tasks:

  1. Low-resource LVCSR: Training a recognizer on just 1 hour of Callhome English (spontaneous telephone conversations).
  2. Spoken Term Discovery: A zero-resource task requiring the system to find repeated words in untranscribed audio using Dynamic Time Warping (DTW).

Source data included up to 20 hours of English Switchboard for the monolingual front-end and 200 hours of Spanish Switchboard/Callhome for the cross-lingual front-end.

Key Results

The experimental results demonstrate significant improvements over baseline features:

  • ASR Accuracy: Standard PLP features on 1 hour of data yielded 28.8% word accuracy. The monolingual front-end improved this to 37.2%, while the cross-lingual front-end (using 200 hours of Spanish) reached 41.5%.
  • Efficiency: Using just 1 hour of data with these data-driven features performs comparably to using 5-10 hours of data with standard features.
  • Zero-resource Performance: Posterior features significantly outperformed raw acoustic features in "Average Precision" for spoken term discovery, proving their superior discriminative power in environments without any transcriptions.

Conclusion

This research highlights that neural networks trained on large, task-independent datasets—even from different languages—can create robust feature representations for low-resource ASR. The key requirement is matching acoustic channel conditions rather than language or genre. Future work may involve adapting these multilingual MLPs directly to the target language to further bridge the resource gap.

Summary updated

BibTeX
@inproceedings{thomas2012data,
  author = {Samuel Thomas and Sriram Ganapathy and Aren Jansen and Hynek Hermansky},
  title = {{Data-driven Posterior Features for Low Resource Speech Recognition Applications}},
  booktitle = {Interspeech},
  year = {2012},
  doi = {10.21437/interspeech.2012-242},
}
Figure