Bayesian and Gaussian Process Neural Networks for Large Vocabulary Continuous Speech Recognition
Lists
Only you see this
Abstract
The hidden activation functions inside deep neural networks (DNNs) play a vital role in learning high level discriminative features and controlling the information flows to track longer history. However, the fixed model parameters used in standard DNNs can lead to over-fitting and poor generalization when given limited training data. Furthermore, the precise forms of activations used in DNNs are often manually set at a global level for all hidden nodes, thus lacking an automatic selection method. In order to address these issues, Bayesian neural networks (BNNs) acoustic models are proposed in this paper to explicitly model the uncertainty associated with DNN parameters. Gaussian Process (GP) activations based DNN and LSTM acoustic models are also used in this paper to allow the optimal forms of hidden activations to be stochastically learned for individual hidden nodes. An efficient variational inference based training algorithm is derived for BNN, GPNN and GPLSTM systems. Experiments were conducted on a LVCSR system trained on a 75 hour subset of Switchboard I data. The best BNN and GPNN systems outperformed both the baseline DNN systems constructed using fixed form activations and their combination via frame level joint decoding by 1% absolute in word error rate.
BibTeX
@inproceedings{hu2019bayesian,
author = {Shoukang Hu and Max W. Y. Lam and Xurong Xie and Shansong Liu and Jianwei Yu and Xixin Wu and Xunying Liu and Helen Meng},
title = {{Bayesian and Gaussian Process Neural Networks for Large Vocabulary Continuous Speech Recognition}},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
year = {2019},
doi = {10.1109/icassp.2019.8682487},
}Loading related content...