Using Highway Connections to Enable Deep Small‐footprint LSTM‐RNNs for Speech Recognition

Gaofeng Cheng, Xin Li, Yonghong YanChinese 2019 · DOI

Abstract

Long short-term memory RNNs (LSTMRNNs) have shown great success in the Automatic speech recognition (ASR) field and have become the state-ofthe- art acoustic model for time-sequence modeling tasks. However, it is still difficult to train deep LSTM-RNNs while keeping the parameter number small. We use the highway connections between memory cells in adjacent layers to train a small-footprint highway LSTM-RNNs (HLSTM-RNNs), which are deeper and thinner compared to conventional LSTM-RNNs. The experiments on the Switchboard (SWBD) indicate that we can train thinner and deeper HLSTM-RNNs with a smaller parameter number than the conventional 3-layer LSTM-RNNs and a lower Word error rate (WER) than the conventional one. Compared with the counterparts of small-footprint LSTMRNNs, the small-footprint HLSTM-RNNs show greater reduction in WER.
BibTeX
@inproceedings{cheng2019using,
  author = {Gaofeng Cheng and Xin Li and Yonghong Yan},
  title = {{Using Highway Connections to Enable Deep Small‐footprint LSTM‐RNNs for Speech Recognition}},
  booktitle = {Chinese},
  year = {2019},
  doi = {10.1049/cje.2018.11.008},
}

Loading related content...

Figure