Discriminatively learning factorized finite state pronunciation models from dynamic Bayesian networks

Preethi Jyothi, Eric Fosler-Lussier, Karen Livescu

Introduction

Standard speech recognition systems typically rely on phonetic dictionaries. However, phones are often too coarse to capture the nuances of casual, conversational speech. Articulatory feature (AF) based models, which track physical speech production mechanisms like tongue position or lip opening, offer a more detailed alternative. While Dynamic Bayesian Networks (DBNs) are effective for modeling these features, they face two major challenges: integration with modern Weighted Finite State Transducer (WFST) based recognizers and the difficulty of applying discriminative training. This paper presents a method to transform DBN-based pronunciation models into factorized WFSTs, enabling discriminative optimization and significant performance gains.

Methodology

The core contribution is a two-step approach to bridge the gap between generative DBNs and discriminative WFSTs.

Factorization and Conversion

To avoid the state-space explosion associated with converting a complex DBN into a single massive WFST, the authors partition the DBN’s random variables into five distinct subsets ($S_1$ to $S_5$).

The DBN structure partitioned into five subsets (S1-S5) for factorization.

As shown in Figure 4, these subsets represent different levels of the speech production process, from the target word down to surface features. Based on this partitioning, five smaller "factor" FSTs ($F_1$ to $F_5$) are created.

The sequence of factorized FSTs (F1-F5) representing the composed pronunciation model.

When composed together ($F_1 \circ F_2 \circ F_3 \circ F_4 \circ F_5$), these transducers represent the same probability distribution as the original DBN while maintaining computational efficiency through parameter tying.

Discriminative Training

Once the DBN is represented as a factorized WFST, the arc weights can be treated as parameters of a linear model. The authors employ the averaged perceptron algorithm for discriminative training. Unlike generative Maximum Likelihood training, this approach focuses on distinguishing between similar words. During training, if the model misclassifies an utterance, the weights are updated to penalize the incorrect path and reward the path corresponding to the ground truth word.

Experimental Results

The proposed method was evaluated on a lexical access task using a subset of the Switchboard conversational speech corpus. The task involved predicting a word given its surface articulatory feature pronunciation.

Model Error Rate (ER%)
Baseform dictionary only 59.3%
Baseform + Phonological rules 56.4%
Original AF-based DBN 43.0%
Proposed Disc-trained Factorized FST 35.9%

The results demonstrate that discriminative training provides a 7.1% absolute improvement over the generative DBN baseline. This improvement is statistically significant ($p < 0.001$), confirming that the factorized FST structure successfully enables more powerful learning without losing the modeling strengths of the DBN.

Conclusion

This work demonstrates that complex articulatory models can be efficiently integrated into standard WFST frameworks through factorization. By moving beyond the limits of generative training, the discriminative factorized FST can learn subtle correlations and "correct" simplified independence assumptions inherent in the original DBN structure. Future work will focus on extending this approach to continuous speech recognition and exploring context-dependent feature substitutions.

Summary updated

BibTeX
@inproceedings{jyothi2012discriminatively,
  author = {Preethi Jyothi and Eric Fosler-Lussier and Karen Livescu},
  title = {{Discriminatively learning factorized finite state pronunciation models from dynamic Bayesian networks}},
  booktitle = {Interspeech},
  year = {2012},
  doi = {10.21437/interspeech.2012-323},
}
Figure