TL;DR
This paper proposes a novel method for selecting automatically transcribed speech data to improve acoustic models for speech recognition. By combining forward and backward-based decoders, they identify common word hypotheses, which are highly likely to be correct, leading to significant reductions in word error rates when this selected data is used for adaptation.
What Problem Does This Paper Address?
Training robust acoustic models for speech recognition requires vast amounts of manually transcribed speech data. This process is incredibly time-consuming and expensive, posing a significant bottleneck for developing and improving ASR systems. Researchers are constantly seeking ways to leverage readily available, untranscribed audio data to supplement or replace manual transcriptions. This paper tackles the challenge of effectively selecting the most reliable segments from automatically transcribed data for unsupervised training or adaptation of acoustic models.
Methods
The core innovation lies in combining the outputs of two different speech recognition systems: a forward-based Sphinx decoder and a backward-based Julius decoder. The hypothesis is that when both decoders agree on a particular word hypothesis for a speech segment, that hypothesis is very likely to be correct. This "common hypothesis" criterion is used to select segments from automatically transcribed data.
The paper compares this novel selection method against a more traditional approach using confidence measures (specifically, word posterior probabilities from the Julius decoder) to filter automatically transcribed data. They also explore using the selected data in two ways:
- Adapting existing manually trained acoustic models.
- Training acoustic models entirely from scratch.
Datasets
- ESTER2: Used for performance evaluation (word error rates). Both development and test sets were utilized, focusing on non-African radio data.
- ETAPE: Used for training acoustic models, containing spontaneous speech from radio and TV debates.
- EPAC: A large corpus of French broadcast news. The manually transcribed portions were used for baseline model training, while the untranscribed portions (approximately 1377 hours) were used for unsupervised selection and training experiments.
Key Insights and Ideas
- The Power of Agreement: The most striking finding is that segments where forward and backward decoders agree on word hypotheses are highly reliable (over 90% correct in previous work). This agreement acts as a powerful, implicit confidence measure.
- Beyond Confidence Scores: The common hypothesis selection method outperforms traditional confidence measure-based selection, especially when using the Sphinx and Julius combination. This suggests that inter-decoder agreement is a more robust indicator of transcription quality than a single decoder's confidence score.
- Adaptation is Key: Adding the selected automatically transcribed data to an existing manually transcribed training set for adaptation yields significantly better results than retraining models from scratch. This highlights the efficiency of using automatically transcribed data to fine-tune existing models.
- No Benefit in Full Retraining: Retraining acoustic models entirely from scratch using only the selected automatically transcribed data does not offer significant improvements over adaptation and is generally worse than the baseline manual training.
Key Results
- Significant WER Reduction: Using the common hypothesis selection (Sphinx forward + Julius backward) and adding the selected data to the baseline manual training set for gender adaptation resulted in a significant reduction in Word Error Rate (WER) on the ESTER2 dataset (approximately 0.4% absolute improvement compared to the baseline).
- More Data Selected: The Sphinx & Julius common hypothesis selection method yielded a much larger amount of selected data (842 hours) compared to using only Julius with a confidence threshold of 0.6 (689 hours) or even 0.4 (885 hours).
- Comparable to Baseline (Full Training): When training acoustic models from scratch using only the selected automatically transcribed data, the WER was only about 1% worse than the baseline trained on a large manually transcribed set.
Limitations and Future Work
- Model Size Impact: The paper notes that the baseline training set (300 hours) was not large enough to effectively handle increased model complexity (more senones or Gaussian components). This suggests that for larger models, more manually transcribed data or even more effectively selected automatically transcribed data might be needed.
- Future Directions: The authors suggest that this method opens doors for applying automatic transcription to new types of data, such as large bandwidth telephone speech, for later training of large bandwidth models. This could leverage vast amounts of previously underutilized speech corpora. They also imply that further exploration into combining different decoder types or refining the selection criteria could yield even better results.
Summary updated