An efficient Transformer with neighborhood contrastive tokenization for hyperspectral images classification

Miaomiao Liang, Xianhao Zhang, Xiangchun Yu, Lingjuan Yu, Zhe Meng, Xiaohong Zhang, Licheng Jiao

Abstract

Highlights•Efficient Transformers require compact, semantical, and context-sensitive tokens.•A global-to-local predictor promotes neighboring context-dependent tokenization.•Weighted token-level contrastive loss enriches tokens with instance discriminability.•NeiCoT is effective for both self-supervised and supervised Transformer backbones.
BibTeX
@article{liang2024efficient,
  author = {Miaomiao Liang and Xianhao Zhang and Xiangchun Yu and Lingjuan Yu and Zhe Meng and Xiaohong Zhang and Licheng Jiao},
  title = {{An efficient Transformer with neighborhood contrastive tokenization for hyperspectral images classification}},
  journal = {Int. J. Appl. Earth Obs. Geoinformation},
  year = {2024},
  doi = {10.1016/j.jag.2024.103979},
}

Loading related content...

Figure