Frequency-Spatial Domain Information Fusion Network for Pan-Sharpening

Mengjiao Zhao, Mengting Ma, Ao Gao, Wei ZhangICIP 2024 · OpenReview

Abstract

Pan-sharpening aims to fuse panchromatic (PAN) images with low-resolution multi-spectral (LR-MS) images to generate high-resolution multi-spectral (HR-MS) images. Despite the impressive performance of existing learning-based methods, they are constrained by coarse fusion strategies in frequency or spatial domain. In this paper, we discover that PAN images can provide all the spatial textures required for HR-MS images, while spectral information must be provided jointly by PAN and LR-MS images. Inspired by this, we propose a noval frequency-spatial domain information fusion network for pan-sharpening, called FSDNet. Specifically, we design a Dual-Domain Information Processing Module (DDPM) to construct FSDNet. It consists of a Frequency Domain Feature Processing Block (FDB), a Spatial Domain Information Processing Block (SDB), and an Information Fusion Block (IFB). The FDB in the frequency domain uses the Adaptive Amplitude Fusion Block (AAFB) and convolution layers to finely modulate amplitude and phase components, exploring global information. The SDB uses cascaded residual blocks to capture and enhance local information in the spatial domain. The IFB based on invertible neural networks (INNs) introduces Multi-Scale Self-Attention Block (MSAB), achieves effective information fusion and reduce information loss. Extensive experiments on the QuickBird and GaoFen-2 datasets demonstrate the effectiveness and superiority of our method.
BibTeX
@inproceedings{zhao2024frequency,
  author = {Mengjiao Zhao and Mengting Ma and Ao Gao and Wei Zhang},
  title = {{Frequency-Spatial Domain Information Fusion Network for Pan-Sharpening}},
  booktitle = {ICIP},
  year = {2024},
  url = {https://openreview.net/forum?id=TH5qeCyJ74},
}

Loading related content...

Figure