Abstract
Introducing pronunciation models into decoding has been proven to be benefit to LVCSR. In this paper, a discriminative pronunciation modeling method is presented, within the framework of the Minimum Phone Error (MPE) training for HMM/GMM. In order to bring the pronunciation models into the MPE training, the auxiliary function is rewritten at word level and decomposes into two parts. One is for co-training the acoustic models, and the other is for discriminatively training the pronunciation models. On Mandarin conversational telephone speech recognition task, compared to the baseline using a canonical lexicon, the discriminative pronunciation models reduced the absolute Character Error Rate (CER) by 0.7% on LDC test set, and with the acoustic model co-training, 0.8% additional CER decrease had been achieved.
Introduction
In Large Vocabulary Continuous Speech Recognition (LVCSR), the system typically relies on a lexicon—a dictionary mapping words to their phonetic pronunciations. Most systems utilize "canonical" lexicons where each word is assigned one or two standard pronunciations. However, spontaneous speech often deviates from these standards due to mumbling, regional accents, or phonetic shortcuts.
While Pronunciation Models (PMs) have been developed to account for these variations, they are traditionally trained using Maximum Likelihood (ML) estimation. ML focuses on fitting the model to the data, but for speech recognition, discriminative training—which emphasizes distinguishing correct word sequences from incorrect ones—is often more effective. The Minimum Phone Error (MPE) criterion is a powerful discriminative training objective, but applying it to the lexicon level is mathematically complex. This paper introduces a method to bridge this gap.
Methodology: The MPEPM Approach
The authors propose MPEPM (Minimum Phone Error Pronunciation Modeling), a method for discriminatively training PMs within the MPE framework. The core innovation involves rewriting the standard MPE auxiliary function at the word level rather than the phone level.
The rewritten auxiliary function is decomposed into two distinct parts:
- Acoustic Model (AM) Co-training: A component dedicated to updating the AMs.
- Pronunciation Model (PM) Training: A component for discriminatively training the PMs.
These models are updated iteratively. A significant practical advantage of MPEPM is that it utilizes statistics already present in the recognition lattices generated during standard MPE training. This eliminates the need for generating expensive "N-best" lists of pronunciations, making the process computationally efficient.
Experimental Setup
The method was evaluated on a Mandarin conversational telephone speech recognition task.
- Training Data: Approximately 400 hours of audio, including 196 hours from the Linguistic Data Consortium (LDC) and 200 hours of self-collected data.
- Evaluation Sets:
- HTest04: 4 hours of data from HKUST (2005).
- GDTest: 30 minutes of self-collected data from 354 telephone conversations.
Results and Discussion
The performance was measured using Character Error Rate (CER). The results demonstrate a clear synergy between PM training and AM co-training.
On the HTest04 set, the MPEPM approach alone reduced CER from 49.7% to 49.0%. When combined with AM co-training, the CER further dropped to 48.2%, representing a 1.5% absolute reduction from the baseline. Similar trends were observed on the GDTest set, with a total reduction of 1.1%. These improvements highlight the effectiveness of moving beyond fixed canonical pronunciations toward learned, discriminative variants.
Conclusion
This work successfully integrates discriminative pronunciation modeling into the MPE framework. By co-training AMs and PMs, significant reductions in CER were achieved for Mandarin conversational speech. While originally developed for HMM/GMM systems, the underlying logic of word-level decomposition for sequence-discriminative training remains relevant for modern hybrid DNN architectures.
Summary updated