Regularization of context-dependent deep neural networks with context-independent multi-task training

Peter Bell, Steve RenalsICASSP 2015 · OpenReview

Abstract

The use of context-dependent targets has become standard in hybrid DNN systems for automatic speech recognition. However, we argue that despite the use of state-tying, optimising to context-dependent targets can lead to over-fitting, and that discriminating between arbitrary tied context-dependent targets may not be optimal. We propose a multitask learning method where the network jointly predicts context-dependent and monophone targets. We evaluate the method on a large-vocabulary lecture recognition task and show that it yields relative improvements of 3-10% over baseline systems.
BibTeX
@inproceedings{bell2015regularization,
  author = {Peter Bell and Steve Renals},
  title = {{Regularization of context-dependent deep neural networks with context-independent multi-task training}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2015},
  url = {https://openreview.net/forum?id=5lSjcUGIeE},
}

Loading related content...

Figure