A Deep Network with Visual Text Composition Behavior

Hongyu Guo

Abstract

While natural languages are compositional, how state-of-the-art neural models achieve compositionality is still unclear. We propose a deep network, which not only achieves competitive accuracy for text classification, but also exhibits compositional behavior. That is, while creating hierarchical representations of a piece of text, such as a sentence, the lower layers of the network distribute their layer-specific attention weights to individual words. In contrast, the higher layers compose meaningful phrases and clauses, whose lengths increase as the networks get deeper until fully composing the sentence.
BibTeX
@inproceedings{guo2017deep,
  author = {Hongyu Guo},
  title = {{A Deep Network with Visual Text Composition Behavior}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2017},
  doi = {10.18653/v1/p17-2059},
}

Loading related content...

Figure