The RWTH 2009 quaero ASR evaluation system for English and German

Markus Nußbaum-Thom, Simon Wiesler, Martin Sundermeyer, Christian Plahl, Stefan Hahn, Ralf Schlüter, Hermann NeyInterspeech 2010

TL;DR

This paper presents the RWTH Aachen University's automatic speech recognition (ASR) systems for English and German, developed for the 2009 Quaero evaluation. The systems achieved top results for German and competitive results for English by leveraging hierarchical neural network features, system combination, and domain adaptation, particularly in the challenging scenario of limited in-domain training data.

What Problem Does This Paper Address?

The paper tackles the challenge of building robust automatic speech recognition systems for large vocabulary tasks, specifically for transcribing diverse web data, European Parliament proceedings, and broadcast news. A significant hurdle in the 2009 Quaero evaluation was the scarcity of in-domain training data for both English and German, forcing the systems to generalize well from out-of-domain or limited in-domain resources. This variability in acoustic conditions and speech styles, coupled with limited training data, makes accurate transcription a difficult problem.

Methods

The RWTH systems employ a multi-pass recognition strategy for both English and German. Key techniques include:

  • Hierarchical Neural Network (HNN) based Posterior Features: These features, derived from a hierarchical neural network (HMRASTA), are used to augment standard acoustic features (MFCC or PLP). This approach aims to capture more discriminative information about speech.
  • Speaker Normalization and Adaptation: Vocal Tract Length Normalization (VTLN) is applied for initial normalization. Speaker adaptation is achieved through Constrained Maximum Likelihood Linear Regression (CMLLR) and, for German, Maximum Likelihood Linear Regression (MLLR). These techniques help compensate for variations in speaker characteristics.
  • System Combination: Multiple subsystems, differing in acoustic features, acoustic models, and language models, are combined using confusion network combination. This ensemble approach leverages the strengths of individual systems to improve overall performance.
  • Domain-Specific Modeling: Separate acoustic and language models are trained for different domains (e.g., broadcast news, European Parliament sessions) to better adapt to the specific characteristics of each data type.
  • Cross-Adaptation: For the German system, subsystems are cross-adapted to each other in a third pass, further enhancing their performance by learning from each other.

Datasets

The paper utilizes data from the Quaero 2009 evaluation campaign, which includes:

  • English Training Data:
    • Broadcast News (BN) data: HUB4, TDT4 corpora (American English).
  • European Parliament Plenary Sessions (EPPS) data: TC-STAR corpus.
  • A total of approximately 500 hours of acoustic training data was used. Notably, no in-domain training data was provided for English.
  • German Training Data:
    • In-domain data: WEB08 corpus (podcast data from comedy, seminar, report, interview) and EPPS08 corpus (European parliament speeches). A total of 19 hours of in-domain data.
  • Additional out-of-domain data: Verbmobil, WDR, Report Mainz, and Zeit corpora, contributing significantly to the acoustic modeling.
  • Evaluation Data: Dev09 and Eval09 corpora for both English and German, comprising web, BN, and EPPS domains.

Key Insights and Ideas

  • The Power of Neural Network Features: The paper strongly suggests that hierarchical neural network based posterior features are a major contributor to the performance improvements, especially in the challenging scenario with limited in-domain data.
  • Domain Adaptation is Crucial: Even with limited in-domain data, building domain-specific acoustic and language models significantly boosts performance, indicating that tailoring models to the target domain is essential.
  • System Combination Yields Gains: Combining multiple diverse systems consistently improves results, acting as a robust method to mitigate individual system weaknesses.
  • Cross-Adaptation Benefits German: The German system's cross-adaptation strategy in the third pass demonstrates a successful approach to leverage information between subsystems.
  • Limited In-Domain Data is a Real Challenge: The authors repeatedly highlight the difficulty posed by the lack of in-domain training data, making their achievements even more impressive.

Key Results

  • German System: Achieved the best results in the 2009 Quaero evaluation for German. The system showed significant improvements over a baseline, with MLP features and additional training data contributing to a reduction of 26.1% WER on dev09 and 31.1% on eval09.
  • English System: Achieved competitive results for English. The incorporation of MLP features led to improvements of 41.9% WER relative on dev09 and 34.9% on eval09 compared to a baseline. System combination further reduced WER by 6.4% on dev09 and 11.9% on eval09.
  • Overall: The RWTH systems demonstrated substantial gains over previous year's systems, attributed to the combination of advanced features, adaptation techniques, and ensemble methods.

Limitations and Future Work

  • Baseline Tuning: The authors note that the significant improvements observed might be partly due to a not-well-tuned baseline system, making it difficult to precisely isolate the gain from individual components like MLP features.
  • Small Improvement from Cross-Adaptation/Combination (German): For the German system, the final gains from cross-adaptation and system combination were relatively small, possibly due to the limited number of subsystems being combined.
  • Future Work: Not explicitly detailed, but the success of MLP features and domain adaptation suggests continued exploration in these areas. Further investigation into optimizing the interaction between different adaptation techniques and system combination strategies would be valuable.

Summary updated

BibTeX
@inproceedings{nubaumthom2010rwth,
  author = {Markus Nußbaum-Thom and Simon Wiesler and Martin Sundermeyer and Christian Plahl and Stefan Hahn and Ralf Schlüter and Hermann Ney},
  title = {{The RWTH 2009 quaero ASR evaluation system for English and German}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure