Recurrently Controlling a Recurrent Network with Recurrent Networks Controlled by More Recurrent Networks
Lists
Only you see this
Abstract
This paper explores an intriguing idea of recursively parameterizing recurrent nets. Simply speaking, this refers to recurrently controlling a recurrent network with recurrent networks controlled by recurrent networks. The proposed architecture recursively parameterizes its gating functions whereby gating mechanisms of X-RNN are controlled by instances of itself, which are repeatedly called in a recursive fashion. We postulate that our proposed inductive bias provides modeling benefits pertaining to learning with inherently hierarchically-structured sequence data. To this end, we conduct extensive experiments on recursive logic tasks (sorting, tree traversal, logical inference), sequential pixel-by-pixel classification, semantic parsing, code generation, machine translation and polyphonic music modeling, demonstrating the widespread utility of the proposed approach, i.e., achieving optimistic and competitive results on all tasks.
BibTeX
@misc{tay2020recurrently,
author = {Yi Tay and Yikang Shen and Alvin Chan and Aston Zhang and Shuai Zhang},
title = {{Recurrently Controlling a Recurrent Network with Recurrent Networks Controlled by More Recurrent Networks}},
year = {2020},
url = {https://openreview.net/forum?id=gnAGreyEzP2},
howpublished = {OpenReview},
}Loading related content...