Long text analysis using sliced recurrent neural networks with breaking point information enrichment
Lists
Only you see this
Abstract
Sliced recurrent neural networks (SRNNs) are the state-ofthe-art efficient solution for long text analysis tasks; however, their slicing operations inevitably result in long-term dependency loss in lower-level networks and thus limit their accuracy. Therefore, we propose a breaking point information enrichment mechanism to strengthen dependencies between sliced subsequences without hindering parallelization. Then, the resulting BPIE-SRNN model is further extended to a bidirectional model, BPIE-BiSRNN, to utilize the dependency information in not only the previous but also the following contexts. Experiments on four large public real-world datasets demonstrate that the BPIE-SRNN and BPIE-BiSRNN models always achieve a much better accuracy than SRNNs and BiSRNNs, while maintaining a superior training efficiency
BibTeX
@inproceedings{li2019long,
author = {Bo Li and Zehua Cheng and Zhenghua Xu and Wei Ye and Thomas Lukasiewicz and Shikun Zhang},
title = {{Long text analysis using sliced recurrent neural networks with breaking point information enrichment}},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
year = {2019},
url = {https://openreview.net/forum?id=ybxqliQBZ_T},
}Loading related content...