Hyperspectral Imagery Super-Resolution by Compressive Sensing Inspired Dictionary Learning and Spatial-Spectral Regularization

Wei Huang, Liang Xiao, Hongyi Liu, Zhihui WeiSensors 2015 · OpenReview

TL;DR

This paper introduces a novel hyperspectral image super-resolution (HSI-SR) method that leverages compressive sensing (CS) principles for dictionary learning and incorporates spatial-spectral regularization. The approach effectively enhances spatial resolution while preserving crucial spectral information, outperforming existing methods in both objective metrics and visual quality.

What Problem Does This Paper Address?

Hyperspectral Imagery (HSI) offers rich spectral information valuable for various applications like land use analysis and environmental monitoring. However, HSI often suffers from a trade-off between spatial and spectral resolution, with spatial resolution typically being coarser than desired due to instrumental limitations. Super-resolution (SR) techniques aim to generate high-resolution (HR) images from low-resolution (LR) ones. While SR methods have advanced for natural images, applying them to HSI presents unique challenges: preserving the delicate spectral information is paramount, and traditional pan-sharpening methods often compromise spectral fidelity for spatial enhancement. This paper tackles the problem of achieving both high spatial resolution and accurate spectral preservation in HSI.

Methods

The proposed HSI-SR method is built upon two core innovations:

  • CS-Inspired Dictionary Learning: Instead of relying on external HR training data, the method learns a HR dictionary directly from the LR HSI itself, after an initial bicubic interpolation to create a "pre-HR" image. This dictionary learning process is guided by two key principles from compressive sensing (CS):
  • Sparsity: Encouraging HR image patches to be represented sparsely using a limited number of dictionary atoms.
  • Incoherence: Minimizing the coherence between the learned dictionary and the sensing matrix (which models the down-sampling and blurring process). This dual constraint aims to create an efficient and adaptive dictionary that leads to better sparse representations. The learning process is formulated as an optimization problem that simultaneously minimizes reconstruction error, enforces sparsity, and limits dictionary-sensing matrix coherence.

  • Spatial-Spectral Regularization: A variational regularization model is employed to integrate spatial and spectral information.

  • Spatial Sparsity Regularization: This term leverages the learned dictionary to enforce sparsity in the spatial domain, aiding in the reconstruction of high-frequency details.
  • Local Spectral Similarity Preserving Term: This novel term is designed to maintain spectral fidelity. It uses a similarity measure (Image Patch Distance - IPD) that considers both spatial context (neighboring pixels within a window) and spectral similarity (comparing spectral curves). This ensures that pixels with similar spectral characteristics in the LR image are reconstructed with similar spectral signatures in the HR image.

The overall model combines these regularization terms to solve an ill-posed inverse problem, aiming for a unique and optimal solution.

Datasets

The paper evaluates the proposed method on both simulated and real HSI datasets:

  • Simulated Datasets:
    • Pavia University (PaviaU) dataset: 103 spectral bands (after noise removal), 210x210 pixels.
  • Pavia Center (PaviaC) dataset: 102 spectral bands (after noise removal), 210x210 pixels. LR HSI datasets were generated by convolving original HR HSI with a 3x3 Gaussian kernel, down-sampling by a factor of 3, and adding no noise.

  • Real Datasets:

    • Indoor scene (letter paper): 60 spectral bands, 60x120 pixels.
  • Outdoor scene (Fountain Square of Nanjing University of Science and Technology - FS-NUST): 110 spectral bands, 86x86 pixels. For real experiments, the blur kernel was assumed to be a Dirac delta function, with a scaling factor of 3 and no noise.

Key Insights and Ideas

  • Learning from the Data Itself: The ability to learn a HR dictionary directly from the LR HSI, rather than requiring external HR training data, makes the method more practical and adaptable to different scenes.
  • Synergy of CS and HSI-SR: Applying CS principles (sparsity and incoherence) to dictionary learning for HSI-SR is a key innovation. It addresses the inherent sparsity of image patches and aims to improve the efficiency of the reconstruction process.
  • Balancing Spatial and Spectral: The core challenge in HSI-SR is the trade-off between spatial enhancement and spectral preservation. The proposed method's strength lies in its explicit regularization terms designed to tackle both aspects simultaneously.
  • Novel Spectral Similarity Measure: The Image Patch Distance (IPD) metric, which integrates spatial and spectral information, is crucial for preserving spectral fidelity. It goes beyond simple pixel-wise comparisons to consider local neighborhoods and spectral curve shapes.

Key Results

The proposed method demonstrated superior performance across various metrics:

  • Quantitative Evaluation:
    • PaviaU Dataset: Achieved the highest A-PSNR (27.9708), A-SSIM (0.8017), and the lowest SAM (0.0866) compared to bicubic interpolation, PCA fusion, WT fusion, P+XS fusion, and SR-SR methods.
  • PaviaC Dataset: Also outperformed other methods with the highest A-PSNR (30.8177), A-SSIM (0.8630), and lowest SAM (0.0866).
  • Qualitative Evaluation: Visual results consistently showed that the proposed method produced sharper edges, clearer details, and better overall spatial quality, closely resembling the original HSI. Crucially, spectral curves reconstructed by the proposed method were significantly closer to the original spectral curves than those from competing methods, indicating superior spectral preservation.

Limitations and Future Work

  • Computational Cost: The paper acknowledges that the proposed method is computationally more intensive than others. This is attributed to processing hundreds of HSI channels band-by-band and the search for similar pixels using IPD.
  • Future Work: To address the computational cost, the authors suggest reducing HSI dimensions and exploring parallel computing approaches (e.g., many-core algorithms) to speed up the processing.

Summary updated

Abstract

Due to the instrumental and imaging optics limitations, it is difficult to acquire high spatial resolution hyperspectral imagery (HSI). Super-resolution (SR) imagery aims at inferring high quality images of a given scene from degraded versions of the same scene. This paper proposes a novel hyperspectral imagery super-resolution (HSI-SR) method via dictionary learning and spatial-spectral regularization. The main contributions of this paper are twofold. First, inspired by the compressive sensing (CS) framework, for learning the high resolution dictionary, we encourage stronger sparsity on image patches and promote smaller coherence between the learned dictionary and sensing matrix. Thus, a sparsity and incoherence restricted dictionary learning method is proposed to achieve higher efficiency sparse representation. Second, a variational regularization model combing a spatial sparsity regularization term and a new local spectral similarity preserving term is proposed to integrate the spectral and spatial-contextual information of the HSI. Experimental results show that the proposed method can effectively recover spatial information and better preserve spectral information. The high spatial resolution HSI reconstructed by the proposed method outperforms reconstructed results by other well-known methods in terms of both objective measurements and visual evaluation.
BibTeX
@inproceedings{huang2015hyperspectral,
  author = {Wei Huang and Liang Xiao and Hongyi Liu and Zhihui Wei},
  title = {{Hyperspectral Imagery Super-Resolution by Compressive Sensing Inspired Dictionary Learning and Spatial-Spectral Regularization}},
  booktitle = {Sensors},
  year = {2015},
  url = {https://openreview.net/forum?id=VEfcQ-vX9VQ},
}
Figure