Understanding the dropout strategy and analyzing its effectiveness on LVCSR

Jie Li, Xiaorui Wang, Bo XuICASSP 2013 · OpenReview

Abstract

The work by Hinton et al shows that the dropout strategy can greatly improve the performance of neural networks as well as reducing the influence of over-fitting. Nevertheless, there is still not a more detailed study on this strategy. In addition, the effectiveness of dropout on the task of LVCSR has not been analyzed. In this paper, we attempt to make a further discussion on the dropout strategy. The impacts on performance of different dropout probabilities for phone recognition task are experimented on TIMIT. To get an in-depth understanding of dropout, experiments of dropout testing are designed from the perspective of model averaging. The effectiveness of dropout is analyzed on a LVCSR task. Results show that the method of dropout fine-tuning combined with standard back-propagation gives significant performance improvements.
BibTeX
@inproceedings{li2013understanding,
  author = {Jie Li and Xiaorui Wang and Bo Xu},
  title = {{Understanding the dropout strategy and analyzing its effectiveness on LVCSR}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2013},
  url = {https://openreview.net/forum?id=AkAbID0dVn},
}

Loading related content...

Figure