Spatial Resolution Enhancement of Remote Sensing Hyperspectral Images With Localized Spatial-Spectral Dictionary Pair

Yifan Zhang, Jin Tian, Tuo Zhao, Shaohui Mei

Abstract

This paper introduces SSLDF (Spatial-Spectral Localized Dictionary Fusion), a novel method for boosting the spatial resolution of hyperspectral images (HSI) by fusing them with high-resolution multispectral images (MSI). By employing a localized strategy and collaborative representation, the authors achieve state-of-the-art fusion quality with significantly lower computational overhead than traditional global dictionary methods.

Introduction

In remote sensing, there is a fundamental hardware trade-off between spatial and spectral resolution. Hyperspectral sensors capture hundreds of narrow, contiguous spectral bands—providing a "fingerprint" for materials—but usually at a coarse spatial resolution. Conversely, multispectral sensors capture high spatial detail but only across a few broad bands. To obtain the "best of both worlds," researchers use image fusion. However, existing dictionary-based fusion methods often struggle with incomplete modeling (failing to capture the spatial-spectral interplay) and high computational complexity due to the use of global dictionaries learned from entire images.

The SSLDF Method

The proposed SSLDF method addresses these challenges through three key innovations designed to improve both accuracy and efficiency.

Localized Spatial-Spectral Dictionary Pairs

Instead of one giant dictionary for the whole image, the HSI and MSI are divided into small, overlapping 3D subimage blocks. For each block, a dictionary pair is constructed where the atoms are the actual patches within that subimage. This ensures the dictionary is perfectly tailored to local features, capturing spatial and spectral information simultaneously rather than treating them as independent dimensions.

Illustration of subimage and patch extraction for localized dictionary construction.

Collaborative Representation (CR)

Most dictionary methods rely on "Sparse Representation" (SR), which assumes a signal is a combination of only a few atoms and requires iterative optimization. SSLDF uses Collaborative Representation (CR), which allows all atoms in the dictionary to contribute. CR is mathematically simpler—often resulting in a closed-form solution—making it much faster than SR. The core assumption is that the low-resolution HSI and the desired high-resolution HSI share the same representation coefficients when mapped to their respective low-resolution and high-resolution dictionaries.

Flowchart of the SSLDF method showing the fusion process from input images to the final high-resolution HSI.

Experimental Evaluation

The study evaluates the method using eight diverse datasets to ensure robustness across different sensors and landscapes, including AVIRIS (Indian Pines, Cuprite, Moffett Field), HYDICE (Washington D.C. Mall), and ROSIS-3 (University of Pavia). Low-resolution HSI and high-resolution MSI were simulated from these reference datasets to provide a ground truth for quantitative comparison.

Key Results

The SSLDF method was compared against 13 other fusion methods, including well-known algorithms like CNMF and HySure.

  • Accuracy: SSLDF consistently produced higher PSNR (Peak Signal-to-Noise Ratio) and lower SAM (Spectral Angle Mapper) values. For example, on the Indian Pines dataset, SSLDF achieved a PSNR of 40.44 dB compared to CNMF's 39.85 dB.
  • Speed: The efficiency gains were dramatic. In one test case, SSLDF completed the fusion in 0.97 seconds, while a competing state-of-the-art method took over 363 seconds—a nearly 375x speedup.
  • Visual Fidelity: RMSE (Root Mean Square Error) maps showed that SSLDF had significantly fewer errors in high-texture areas compared to traditional unmixing-based methods. Using overlapping subimages and averaging the results at the boundaries also prevented "blocking artifacts."

Conclusion and Future Work

SSLDF demonstrates that a localized strategy combined with collaborative representation can significantly outperform global, sparse-representation-based methods in both speed and quality. While the method currently requires empirical tuning for subimage size and regularization parameters, its efficiency makes it a strong candidate for large-scale remote sensing applications. Future work will focus on scaling this approach to even larger datasets and investigating its impact on downstream tasks like automated land-cover classification.

Summary updated

Abstract

The hyperspectral image (HSI) is capable of providing abundant and detailed spectral information in hundreds of contiguous spectral bands. While due to some practical reasons, its spatial resolution is generally lower than that of multispectral image (MSI) and panchromatic image. To deal with the limited spatial resolution issue of HSI, a low resolution (LR) HSI can be fused with a high resolution (HR) MSI of the same scene to generate an HR HSI. A novel dictionary-based HSI and MSI fusion method (SSLDF method) is proposed in this paper, in which a localized spatial-spectral dictionary pair incorporating both spatial and spectral information simultaneously is constructed and adopted, rather than the traditional spectral or spatial one. To construct the HR and LR dictionary pair, HR MSI and its spatial degradation (LR MSI) are divided into overlapped subimages. Furthermore, to reduce the dictionary scale and hence to efficiently reduce the computation cost, a localized strategy is employed for dictionary construction rather than a global one, which makes atoms of the spatial-spectral dictionary actually all patches within the subimage. Based on the appropriate assumption that the LR HSI and HR HSI (expected fusion result) can be collaboratively represented by LR dictionary and HR dictionary respectively sharing the same set of representation coefficients, the desired HR HSI is reconstructed by HR dictionary and the collaborative representation coefficients obtained with LR HSI and LR dictionary. In simulative experiments, the newly proposed SSLDF method is validated and compared with both state-of-the-art dictionary-based fusion methods and representative fusion methods not limited to the dictionary-based ones. Simulative experimental results illustrate that the proposed fusion method is capable of producing better or comparable fused results compared with these representative fusion methods. Its simple structure as well as low computation cost makes it quite promising in practical applications.
BibTeX
@article{zhang2020spatial,
  author = {Yifan Zhang and Jin Tian and Tuo Zhao and Shaohui Mei},
  title = {{Spatial Resolution Enhancement of Remote Sensing Hyperspectral Images With Localized Spatial-Spectral Dictionary Pair}},
  journal = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.2981690},
}
Figure