Recurrently Controlling a Recurrent Network with Recurrent Networks Controlled by More Recurrent Networks

Yi Tay, Yikang Shen, Alvin Chan, Aston Zhang, Shuai ZhangOpenReview 2020 · OpenReview

Abstract

This paper explores an intriguing idea of recursively parameterizing recurrent nets. Simply speaking, this refers to recurrently controlling a recurrent network with recurrent networks controlled by recurrent networks. The proposed architecture recursively parameterizes its gating functions whereby gating mechanisms of X-RNN are controlled by instances of itself, which are repeatedly called in a recursive fashion. We postulate that our proposed inductive bias provides modeling benefits pertaining to learning with inherently hierarchically-structured sequence data. To this end, we conduct extensive experiments on recursive logic tasks (sorting, tree traversal, logical inference), sequential pixel-by-pixel classification, semantic parsing, code generation, machine translation and polyphonic music modeling, demonstrating the widespread utility of the proposed approach, i.e., achieving optimistic and competitive results on all tasks.
BibTeX
@misc{tay2020recurrently,
  author = {Yi Tay and Yikang Shen and Alvin Chan and Aston Zhang and Shuai Zhang},
  title = {{Recurrently Controlling a Recurrent Network with Recurrent Networks Controlled by More Recurrent Networks}},
  year = {2020},
  url = {https://openreview.net/forum?id=gnAGreyEzP2},
  howpublished = {OpenReview},
}

Loading related content...

Figure