Hyperparameter estimation for speech recognition based on variational Bayesian approach

Kei Hashimoto, Heiga Zen, Yoshihiko Nankaku, Akinobu Lee, Keiichi TokudaThe Journal of the Acoustical Society of America 2006 · DOI

Abstract

A hyperparameter estimation technique for HMM-based speech recognition using variational Bayesian (VB) approach is proposed. In recent speech recognition systems, the ML and MDL criteria have been adopted for training HMMs and the context clustering, respectively. However, since the ML estimator produces a point estimate of parameters, the estimation accuracy is reduced with insufficient data. The MDL criterion is also affected by the amount of training data due to an asymptotic approximation. To avoid this problem, the VB method has been applied to HMM-based speech recognition. By using the VB approach, it is expected that the higher generalization ability is achieved and an appropriate model structure can be selected in the context clustering. In the Bayesian framework, the predictive distribution is obtained by integrating the likelihood function with respect to the prior density. Therefore, the performance of the Bayesian method depends on the setting of hyperparameters representing the prior distribution. However, the detailed investigation for hyperparameter estimation has not been conducted in the speech recognition field. This study proposes an estimation technique of hyperparameters based on the maximization of the marginal likelihood for HMM-based speech recognition. Experimental results show that the proposed technique can estimate appropriate hyperparameters from training data.
BibTeX
@article{hashimoto2006hyperparameter,
  author = {Kei Hashimoto and Heiga Zen and Yoshihiko Nankaku and Akinobu Lee and Keiichi Tokuda},
  title = {{Hyperparameter estimation for speech recognition based on variational Bayesian approach}},
  journal = {The Journal of the Acoustical Society of America},
  year = {2006},
  doi = {10.1121/1.4787217},
}

Loading related content...

Figure