rwightman/pytorch-image-models: v0.1-attn-weights

Ross Wightman, Aman Arora, Chris Ha, Alexander Soare, Mike Mike, contrastive, Mohammed Rizin, Csaba Kertész, Dushyant Mehta, Guillem Cucurull, Kushajveer Singh, Andrew Lavin, Juntang Zhuang, Matthijs Hollemans, Sepehr Sameni, Vyacheslav Shults, Yusuke Uchida, Zhun Zhong, Comar, Taehoon Kim, Alexey Chernov, Greg Dongyoon Han, Eli Uriegas, Jasha, Marcin Kardas, Michael Monashev, Min‐Qin Chen, Mohamed Rashad, Minqin Chen, Rashad, MohamedZenodo 2021 · DOI

Abstract

A collection of weights I've trained comparing various types of SE-like (SE, ECA, GC, etc), self-attention (bottleneck, halo, lambda) blocks, and related non-attn baselines.
BibTeX
@inproceedings{wightman2021rwightman,
  author = {Ross Wightman and Aman Arora and Chris Ha and Alexander Soare and Mike Mike and contrastive and Mohammed Rizin and Csaba Kertész and Dushyant Mehta and Guillem Cucurull and Kushajveer Singh and Andrew Lavin and Juntang Zhuang and Matthijs Hollemans and Sepehr Sameni and Vyacheslav Shults and Yusuke Uchida and Zhun Zhong and Comar and Taehoon Kim and Alexey Chernov and Greg Dongyoon Han and Eli Uriegas and Jasha and Marcin Kardas and Michael Monashev and Min‐Qin Chen and Mohamed Rashad and Minqin Chen and Rashad, Mohamed},
  title = {{rwightman/pytorch-image-models: v0.1-attn-weights}},
  booktitle = {Zenodo},
  year = {2021},
  doi = {10.5281/zenodo.5430409},
}

Loading related content...

Figure