Abstract
Automatic Speech Recognition (ASR) systems in specialized environments like national parliaments face a "moving target" problem: new members join, new topics emerge, and room acoustics change over time. To maintain high accuracy, these systems need regular updates. However, official minutes are often edited for readability, making them unsuitable for direct training. This paper presents a semi-automated framework that leverages statistical machine translation and lightly-supervised training to update ASR models using edited minutes, eliminating the need for costly manual verbatim transcription.
Introduction
In the Japanese National Diet, ASR systems are critical for efficient transcription. However, the environment is dynamic; a single general election can replace over 100 members, introducing new voices and vocabulary. Maintaining these systems typically requires verbatim transcripts, which are expensive and slow to produce. While official minutes are readily available, they are heavily edited by stenographers to remove fillers and formalize colloquialisms. This "cleaned" text creates a mismatch with the raw, spontaneous speech the ASR system must process.
Methodology
The proposed framework addresses the mismatch between spoken and written styles through three primary components:
- Speaking-Style Transformation (SST): Treating the discrepancy as a translation task, the system uses a statistical machine translation (SMT) framework to "translate" formal document-style minutes back into spoken-style N-gram statistics. This allows the language model (LM) to incorporate new vocabulary while retaining the characteristics of spontaneous speech.
- Lightly-Supervised Training (LSV): To update the Acoustic Model (AM), the system uses the SST-enhanced language model to transcribe new speech data. These "best guess" labels are then used to re-train the acoustic model, allowing it to adapt to new speakers and acoustic conditions without manual intervention.
- Accuracy Estimation: To monitor performance in production, the system performs the reverse operation: transforming ASR output into document-style text and comparing it to official minutes. A high degree of similarity indicates high ASR accuracy.
Experimental Setup
The system was initially trained on 168 million words from Diet minutes (1999–2009) and 225 hours of speech. The update phase utilized 95 hours of speech from the 173rd session. Evaluation was conducted on three committee meetings from the 174th session (2010), a particularly challenging period following a general election that introduced significant new content and speakers.
Results and Discussion
The semi-automated update yielded significant performance gains. Combining updated acoustic and language models resulted in a 5.9% relative reduction in word error and a 7.3% relative reduction in character error compared to the baseline. The update successfully integrated 310 new words into the system's vocabulary.
Crucially, the accuracy estimation method proved highly reliable. The correlation between the estimated accuracy (using minutes) and actual accuracy (using verbatim transcripts) ranged from 0.88 to 0.92, validating it as a proxy for system health monitoring. The researchers also found that full re-training using lightly supervised labels outperformed standard adaptation techniques like MLLR, which slightly degraded performance in this context.
Conclusion
This framework demonstrates that SMT-based style transformation can effectively bridge the gap between formal records and spontaneous speech. By closing the loop with automated accuracy estimation, the system can maintain high performance in dynamic environments like the Japanese Diet with minimal manual overhead. Future work may extend this logic to other domains, such as transcribing university lectures using polished course notes.
Summary updated