Investigation of deep Boltzmann machines for phone recognition

Zhao You, Xiaorui Wang, Bo XuICASSP 2013 · OpenReview

Abstract

In the past few years, deep neural networks (DNNs) achieved great successes in speech recognition. The layer-wise pre-trained deep belief network (DBN) is known as one of the critical factor to optimize the DNN. However, the DBN has one shortcoming that the pre-training procedure is in a greedy forward pass. The top-down influences on the inference process are ignored, thus the pre-trained DBN is suboptimal. In this paper, we attempt to apply deep Boltzmann machine (DBM) on acoustic modeling. DBM has the advantages that a top-down feedback is incorporated and the parameters of all layers can be jointly optimized. Experiments are conducted on the TIMIT phone recognition task to investigate the DBM-DNN acoustic model. Comparing with the DBN-DNN with same amount of parameters, phone error rate on the core test set is reduced by 3.8% relatively, and additional 5.1% by dropout fine-tuning.
BibTeX
@inproceedings{you2013investigation,
  author = {Zhao You and Xiaorui Wang and Bo Xu},
  title = {{Investigation of deep Boltzmann machines for phone recognition}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2013},
  url = {https://openreview.net/forum?id=3woAvoxm5j},
}

Loading related content...

Figure