SSETPAN: Spatial-Spectral Enhanced Transformer based network for pansharpening

Huanting Zhang, Mengting Ma, Xinyu Wang, Jiawei Yang, Xiangdong Li, Wei ZhangICME 2024 · OpenReview

Abstract

Pansharpening aims for effective spatial-spectral fusion of low-resolution multispectral (LR-MS) and panchromatic (PAN) images, yielding high-resolution multispectral (HR-MS) images. PAN images contain rich spatial details and LR-MS images contain abundant spectral features. However, most of the learning-based methods ignore their distinct attributes, and employ weaker fusion strategies. Besides, Transformer has recently gained considerable popularity in target feature extraction. Therefore, our paper develops a novel Transformer-based network for pansharpening, dubbed Spatial-Spectral Enhanced Transformer based network (SSETPAN), proficient in fine-grained spatial-spectral feature extraction and interaction. SSET-PAN comprises three main modules: channel-wise transformer (CTM), spatial-wise transformer (STM), and adaptive spatial-spectral feature fusion (ASSFM). CTM extracts LR-MS explicit spectral features, while STM obtains high-quality spatial features. ASSFM achieves adaptive spatial-spectral feature fusion via kernel-varied convolution combination. Extensive experiments on GaoFen-2 and WorldView-3 datasets demonstrate that SSETPAN achieves favorable performance against existing pansharpening methods.
BibTeX
@inproceedings{zhang2024ssetpan,
  author = {Huanting Zhang and Mengting Ma and Xinyu Wang and Jiawei Yang and Xiangdong Li and Wei Zhang},
  title = {{SSETPAN: Spatial-Spectral Enhanced Transformer based network for pansharpening}},
  booktitle = {ICME},
  year = {2024},
  url = {https://openreview.net/forum?id=69Y8Mrq9KK},
}

Loading related content...

Figure