The AMIDA 2009 meeting transcription system

Thomas Hain, Lukás Burget, John Dines, Philip N. Garner, Asmaa El Hannani, Marijn Huijbregts, Martin Karafiát, Mike Lincoln, Vincent WanInterspeech 2010 · OpenReview

Introduction

The AMIDA 2009 system represents a significant milestone in the field of automatic speech recognition (ASR), specifically targeting the highly challenging domain of meeting transcription. By achieving a 6–13% relative reduction in Word Error Rate (WER) compared to previous benchmarks, the system demonstrated that substantial gains in accuracy could be paired with dramatic improvements in computational efficiency. Notably, the 2009 iteration was five times faster than its predecessor and required significantly less training data, primarily through the use of "stacked bottleneck" features and optimized decoding strategies.

The Challenge of Meeting Transcription

Transcribing meetings is often considered a "boss level" task for speech recognition. Unlike clean, dictated speech recorded in controlled environments, meetings involve a multitude of acoustic hurdles: overlapping voices, distant microphones (Multiple Distant Microphones or MDM), varying room acoustics, and highly informal, spontaneous language.

The AMIDA researchers specifically sought to address the limitations of their 2007 system, which was computationally prohibitive—running at nearly 100x real-time—and relied on massive external datasets like the Fisher corpus. The 2009 system aimed to be more robust, particularly for "multi-room" meetings where participants are connected via video conference, introducing complex echo and delay issues.

Key Methodological Innovations

Stacked Bottleneck (SBN) Features

The primary innovation of the AMIDA 2009 system is the use of Stacked Bottleneck (SBN) features. Traditional bottleneck features utilize a neural network to compress speech into a narrow hidden layer before passing it to a recognizer. The AMIDA system evolves this by "stacking" these networks: it uses the outputs of left-context and right-context networks as inputs for a subsequent "merger" Multi-Layer Perceptron (MLP).

The Stacked Bottleneck (SBN) feature extraction process, showing the contextual processing and merger stages.

This architecture allows the model to capture much richer temporal information by looking at a wider window of time (approximately 19 frames). This context is crucial for deciphering the rapid and often messy nature of conversational speech.

Optimized Decoding and Training

Beyond feature engineering, the system introduced several other upgrades:

  • Juicer (WFST Decoder): The transition from the older HTK HDecode to "Juicer," a decoder based on Weighted Finite State Transducers (WFST), was pivotal. This allowed the team to utilize higher-order language models (up to 7-grams) while drastically increasing decoding speed.
  • fMPE Training: The researchers employed Feature-space Minimum Phone Error (fMPE) training, which transforms input features to minimize phone-level errors, further boosting discriminative power.
  • Automatic Room Filtering: To handle the complexities of video conferencing, a cross-correlation algorithm was developed to identify and discard audio played through loudspeakers in remote rooms, preventing the system from transcribing the same speaker twice.

System Architecture

The resulting system is a sophisticated multi-pass architecture. As shown in the pipeline below, the process involves initial beamforming and room assignment, followed by multiple stages of feature extraction (PLP and SBN) and adaptation (CMLLR) before reaching the final decoding stages.

The multi-pass system architecture of the AMIDA 2009 transcription system, illustrating the flow from source audio to final transcription.

Experimental Results

The system was benchmarked against the NIST RT’07 and RT’09 evaluation sets, the gold standards for meeting transcription research.

  • Accuracy: On the rt09seval Individual Head Microphone (IHM) task, the final system reached a WER of 27.4% using automatic segmentation.
  • Speed: The Real-Time Factor (RTF) was reduced by a factor of five. While the 2007 system required 100 seconds to process 1 second of audio, the 2009 system's M3 model achieved roughly 9.84 RTF.
  • Data Efficiency: By focusing on the AMI and AMIDA corpora (totaling ~177 hours for IHM and ~154 hours for MDM), the team proved that smarter feature engineering and search algorithms are more effective than simply utilizing massive external datasets like the Fisher corpus.

Discussion and Future Work

Despite these advancements, several frontiers remain. The researchers noted a persistent "WER gap" between manual and automatic segmentation, particularly when there are large imbalances in gain between microphone channels. Additionally, while the system is significantly faster, its multi-pass nature remains complex for real-time edge applications. Future work continues to focus on the challenges of overlapping speech and further distilling these complex architectures for broader use.

Summary updated

BibTeX
@inproceedings{hain2010amida,
  author = {Thomas Hain and Lukás Burget and John Dines and Philip N. Garner and Asmaa El Hannani and Marijn Huijbregts and Martin Karafiát and Mike Lincoln and Vincent Wan},
  title = {{The AMIDA 2009 meeting transcription system}},
  booktitle = {Interspeech},
  year = {2010},
  url = {https://openreview.net/forum?id=YwWfOK5QF43},
}
Figure