Area Attention

Yang Li, Lukasz Kaiser, Samy Bengio, Si Si

Abstract

Existing attention mechanisms are trained to attend to individual items in a collection (the memory) with a predefined, fixed granularity, e.g., a word token or an image grid. We propose area atten...
BibTeX
@inproceedings{li2019area,
  author = {Yang Li and Lukasz Kaiser and Samy Bengio and Si Si},
  title = {{Area Attention}},
  booktitle = {International Conference on Machine Learning},
  year = {2019},
}
Figure