TL;DR
Training acoustic models for spontaneous speech is hindered by the high cost of accurate transcriptions. This paper proposes a method to automatically identify and remove transcription errors from "rough" transcripts by analyzing phone duration anomalies during forced alignment, leading to improved ASR performance.
What Problem Does This Paper Address?
Automatic Speech Recognition (ASR) systems are increasingly being developed for spontaneous, conversational speech, which is far more complex than clean, read speech. Training robust ASR models requires large amounts of transcribed speech data. However, manually creating precise, word-for-word transcripts of spontaneous speech is extremely time-consuming and expensive. To reduce this burden, researchers often use "rough" transcripts that may contain minor errors, omissions of non-speech events (like laughter), or imprecisely transcribed fragments. While cheaper to produce, these rough transcripts can negatively impact the performance of acoustic models (AMs) trained on them. This paper addresses the challenge of leveraging these imperfect, but more accessible, rough transcripts for AM training by developing a method to automatically detect and mitigate the errors within them.
Methods
The core of the proposed method lies in detecting "phone duration anomalies" within rough transcripts. The authors' approach involves the following steps:
- Baseline AM Training: A baseline acoustic model is trained using a small amount of high-quality, precisely transcribed speech data.
- Forced Alignment: This baseline AM is used to perform forced alignment on the precise transcripts and their corresponding audio. This process aligns the audio signal with the phonetic transcription at a very fine-grained level.
- Phone Duration Statistics: Statistics (mean and standard deviation) for the duration of each phone are collected from the forced alignments of the precise transcripts. This establishes a model of "normal" phone durations.
- Forced Alignment with Rough Transcripts: The baseline AM is then used to perform forced alignment on the rough transcripts and their corresponding audio.
- Anomaly Detection: The durations of phones in the rough transcript alignments are compared against the established "normal" durations. If a phone's duration deviates significantly (e.g., more than N standard deviations from the mean, where N is a tunable parameter), it's flagged as a potential "phone duration anomaly." This anomaly is hypothesized to be caused by a mismatch between the rough transcript and the actual speech, often due to transcription errors or unannotated non-speech events.
- Purification: Utterances containing these detected anomalies are then "purified." The authors opt to retain the portion of the utterance and transcript up to the point of the anomaly, discarding the rest, as subsequent alignments are deemed unreliable. This results in "purified transcripts."
- AM Training with Purified Transcripts: New acoustic models are trained using these purified transcripts, either with conventional Maximum Likelihood (ML) criteria or discriminative training (boosted Minimum Mutual Information - bMMI).
Datasets
- Training Data: The paper mentions using 50 hours of conversational telephone speech with precise transcripts for training the baseline AM. Additionally, new audio data was collected and transcribed into rough transcripts for the proposed method.
- Evaluation Data: ASR experiments were conducted on real-life data from a Japanese call center, consisting of 886 utterances with a total duration of 44.9 minutes. This data was used to evaluate the performance of AMs trained with different transcriptions.
Key Insights and Ideas
- Phone Duration as an Error Indicator: The central insight is that transcription errors or unannotated speech events in rough transcripts can manifest as unnaturally long phone durations during forced alignment. This provides a quantifiable signal for error detection.
- Trade-off in Error Removal: The parameter 'N' (controlling the sensitivity of anomaly detection) creates a trade-off between precision and recall in error detection. A lower N leads to higher recall (catching more errors) but lower precision (more false positives), and vice-versa.
- Discriminative Training Benefits More: The paper suggests that discriminative training methods like bMMI are more sensitive to transcription errors and thus benefit more from the purified transcripts compared to standard ML training.
- Beyond Transcription Errors: The method also implicitly helps in handling non-speech events like laughter or breathing noises, which can cause similar duration anomalies and negatively impact AM training.
Key Results
- Error Detection Performance: A preliminary experiment showed that the proposed method could detect erroneous parts with moderately high precision and recall, particularly when N=4.
- ASR Performance Improvement (ML): Training an AM with purified transcripts (N=4) using ML criteria resulted in a Character Error Rate (CER) of 22.4%, an improvement of 0.5% over using all rough transcripts (CER 22.9%). This was achieved with a purified transcript size of 69% of the original rough transcripts.
- ASR Performance Improvement (bMMI): Using bMMI training, the AM trained with purified transcripts (N=4) achieved a CER of 20.7%, an improvement of 0.8% over using all rough transcripts (CER 21.5%). This demonstrates a more significant benefit from error purification with discriminative training.
Limitations and Future Work
- Parameter Tuning: The effectiveness of the method depends on the choice of the parameter N, which requires careful tuning to balance precision and recall for optimal AM training.
- Data Loss: The purification process involves discarding parts of utterances, which leads to a reduction in the total amount of training data. While the paper shows this is beneficial, it's a trade-off.
- Future Work: The authors express interest in contrasting and combining their proposed method with other existing techniques for lightly supervised training in the future. They also acknowledge that the method could be further refined by exploring different anomaly detection strategies or more sophisticated ways of handling the discarded portions of utterances.
Summary updated