Model-Guided Coarse-to-Fine Fusion Network for Unsupervised Hyperspectral Image Super-Resolution

Jiaxin Li, Ke Zheng, Wengu Liu, Zhi Li, Haoyang Yu, Li NiIEEE GRSL 2023 · OpenReview

Abstract

Fusing a low-resolution hyperspectral image (LrHSI) with an auxiliary high-resolution multispectral image (HrMSI) is a burgeoning technique to realize hyperspectral image super-resolution (HSI-SR), in which learning-based methods have dominated the mainstream direction. However, the underutilization of degradation models and strong dependence on large-scale training triplets severely impede their applicability and performance. Considering these issues, we reformulate the fusion task as a spectral mapping problem and hence propose an unsupervised model-guided coarse-to-fine (C2F) fusion network. Specifically, degradation knowledge learning (DKL) is first performed to fully excavate latent model information, which will serve as guidance for better mapping learning. Following that, a C2F fusion network is constructed with a multiscale attentional fusion (MSAF) module in the head and a C2F structure in the tail. The former is deployed to achieve a more informative compression, and the latter is adopted to capture the spectral relationship, including a spectral degradation-guided (SDG) subnetwork for group-by-group coarse reconstruction and a refinement subnetwork for intergroup correlation and dependencies. Finally, high-resolution HSI can be recovered via established spectral mapping. Extensive experiments on simulated and real datasets verify the superiority of our proposed method. The code is available at https://github.com/JiaxinLiCAS/UMC2FF_GRSL.
BibTeX
@article{li2023model,
  author = {Jiaxin Li and Ke Zheng and Wengu Liu and Zhi Li and Haoyang Yu and Li Ni},
  title = {{Model-Guided Coarse-to-Fine Fusion Network for Unsupervised Hyperspectral Image Super-Resolution}},
  journal = {IEEE Geosci. Remote. Sens. Lett.},
  year = {2023},
  url = {https://openreview.net/forum?id=g9caFYG4Ye},
}

Loading related content...

Figure