Bayesian Learning of N-Gram Statistical Language Modeling

Shuanhu Bai, Haizhou Li

Abstract

The n-gram language model adaptation is typically formulated using deleted interpolation under the maximum likelihood estimation framework. This paper proposes a Bayesian learning framework for n-gram statistical language model training and adaptation. By introducing a Dirichlet conjugate prior to the n-gram parameters, we formulate the deleted interpolation under maximum a posterior criterion with a Bayesian learning procedure. We study the Bayesian learning formulation for n-gram and continuous n-gram language models. The experiments on North American News Text corpus have validated the effectiveness of the proposed algorithms.
BibTeX
@inproceedings{bai2006bayesian,
  author = {Shuanhu Bai and Haizhou Li},
  title = {{Bayesian Learning of N-Gram Statistical Language Modeling}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2006},
  doi = {10.1109/icassp.2006.1660203},
}

Loading related content...

Figure