A Prefix Tree Based $n$-Best List Re-Scoring Strategy for Recurrent Neural Network Language Model
Lists
Only you see this
Abstract
In this paper, issues of speeding up Re-current neural network language model (RNNLM) in the testing phase are explored so that RNNLMs can be used to re-rank a large <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$n$</tex>-best list in real-time systems which could obtain better performance. A new <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$n$</tex>-best list re-scoring framework, Prefix tree based <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$n$</tex>-best list re-scoring (PTNR), is proposed to hundred percent eliminate the repeated computations which makes <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$n$</tex>-best list re-scoring ineffective. At the same time, the bunch mode technique, widely-used in speeding up the training of Feed-forward neural network language model (FF-NNLM), is combined with PTNR and the speed is further improved. Experimental results show that our approach is much faster than basic <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$n$</tex>-best list re-scoring. Take 1000-best as an example, our approach is almost 11 times faster than the basic <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$n$</tex>-best list re-scoring.
BibTeX
@inproceedings{si2014prefix,
author = {Yujing Si and Li Ta and Jielin Pan and Yonghong Yan},
title = {{A Prefix Tree Based $n$-Best List Re-Scoring Strategy for Recurrent Neural Network Language Model}},
booktitle = {Chinese},
year = {2014},
doi = {10.23919/cje.2014.10848022},
}Loading related content...