LA-layer: General local attention layer for full attention networks

Hui Lu, Ronald Poppe, Albert Ali Salah

Abstract

Attention layers have contributed to state-of-the-art results on vision tasks. Still, they leave room for improvement because position information is used in a fixed manner, and the computation cost is typically high. To mitigate both issues, we propose a convolution-style local attention layer (LA-layer) as a replacement for traditional attention layers. LA-layers not only encode the position information of pixels in a convolutional manner, but also produce position offsets following a novel constrained rule so that keys will deform and result in larger receptive fields. Query and keys are processed by a novel aggregation function that outputs attention weights for the values. In our experiments with different types of ResNets, we replace convolutional layers with LA-layers and address image recognition, object detection and instance segmentation tasks. We consistently demonstrate performance gains, despite having fewer FLOPs and training parameters. Our code is available at: https://github.com/hotfinda/LA-layer.
BibTeX
@inproceedings{lu2023la,
  author = {Hui Lu and Ronald Poppe and Albert Ali Salah},
  title = {{LA-layer: General local attention layer for full attention networks}},
  booktitle = {ICME},
  year = {2023},
  doi = {10.1109/icme55011.2023.00352},
}

Loading related content...

Figure