VISIONARY: Novel Spatial-Spectral Attention Mechanism for Hyperspectral Image Denoising

Aditya Dixit, Nischit Hosamani, Puneet Gupta, Ankur GargWACV 2025

TL;DR

This paper introduces VISIONARY, a novel deep learning architecture for hyperspectral image (HSI) denoising that leverages a new Spatial-Spectral Cubic Transformer (SSCformer) block. VISIONARY effectively captures both spatial and spectral correlations within and across HSI datasets, outperforming existing state-of-the-art methods in both quantitative metrics and visual quality.

What Problem Does This Paper Address?

Hyperspectral Images (HSIs) are incredibly rich in information, capturing data across numerous narrow spectral bands. This makes them invaluable for applications like earth observation and agricultural monitoring. However, the acquisition process introduces various noise artifacts (e.g., Gaussian noise, stripes, impulse noise) that significantly degrade image quality and hinder downstream tasks like classification and segmentation. Existing deep learning methods, particularly those based on Transformers, have shown promise by capturing long-range dependencies. However, they often overlook crucial correlations between different HSI samples in a dataset and struggle to optimally balance spatial and spectral feature importance, leading to suboptimal denoising performance.

Methods

The core innovation of this paper is the Spatial-Spectral Cubic Transformer (SSCformer) block. This block is designed to be a plug-and-play module that can enhance existing Transformer-based HSI denoising models.

Here's a breakdown of the key components:

  • Global Feature Attention (GFA) Block: Unlike standard Self-Attention (SA) which focuses on correlations within a single image, GFA captures correlations across multiple HSI samples in a dataset. This is crucial because HSIs from the same device often share underlying noise patterns. GFA achieves this by using attention mechanisms that consider a common, globally learned weight matrix, allowing it to learn features relevant to the entire training data.
  • SSCformer Block: This block sequentially applies attention mechanisms:
  • Spatial Attention: Captures correlations within spatial dimensions of a set of patches.
  • Spectral Attention: Captures correlations within spectral dimensions of a set of patches.
  • Spatial-Spectral Attention: Captures inter- and intra-dimensional correlations between patches, irrespective of their spatial or spectral locations.
  • Weightage Parameters ($\alpha_1, \alpha_2$): The SSCformer block introduces learnable parameters ($\alpha_1, \alpha_2$) to dynamically control the importance of spatial and spectral attention, allowing the model to adaptively balance these features for optimal denoising.
  • Overall Architecture: VISIONARY integrates the SSCformer blocks with a modified U-Net architecture. The U-Net structure is standard for image-to-image tasks like denoising, providing a hierarchical feature extraction and reconstruction framework.

Datasets

The paper uses the following datasets:

  • ICVL Dataset: Used for training and quantitative/qualitative evaluation under various simulated noise conditions. It consists of 201 HSIs with 31 spectral bands.
  • University of Pavia (PU) Dataset: Used for real-data experiments.
  • Indian Pines (IP) Dataset: Used for real-data experiments.
  • Urban Dataset: Used for real-data experiments.
  • RealHSI Dataset: Used for real-data experiments.

Key Insights and Ideas

  • Inter-Sample Correlation is Key: The paper strongly emphasizes that overlooking correlations between different HSI samples in a dataset significantly limits denoising performance. The GFA mechanism directly addresses this by learning from the entire dataset.
  • Adaptive Spatial-Spectral Balancing: The introduction of learnable weights ($\alpha_1, \alpha_2$) for spatial and spectral attention allows the model to dynamically prioritize these features, which is more effective than fixed or heuristic balancing.
  • GFA's Superiority over SA: Ablation studies clearly show that replacing SA with GFA leads to improved performance, validating the importance of inter-sample correlation.
  • Spectral Importance: The ablation study suggests that spectral correlations might be slightly more critical for HSI denoising than spatial ones, as dropping $\alpha_2$ (spectral attention weight) caused a larger performance drop.
  • Modularity: The SSCformer block is designed as a plug-and-play module, making it easy to integrate into existing Transformer-based HSI denoising frameworks to boost their performance.

Key Results

  • State-of-the-Art Performance: VISIONARY consistently outperforms existing state-of-the-art (SOTA) methods, including both traditional and deep learning approaches, across various noise levels and types on the ICVL dataset.
  • Quantitative Improvements: Table 1 shows significant improvements in PSNR, SSIM, and SAM metrics under Gaussian noise. For example, at $\sigma=50$, VISIONARY achieves a PSNR of 45.50, SSIM of 0.9977, and SAM of 1.74, surpassing other methods. Table 2 shows similar outperformance on complex noise scenarios.
  • Visual Quality: Figure 5 and Figure 6 demonstrate visually superior denoising results, with VISIONARY effectively reducing noise while preserving fine details and spectral signatures.
  • Classification Accuracy: Table 3 shows that post-denoising with VISIONARY leads to the highest classification accuracy (OA and Kappa) on IP and PU datasets compared to other methods.
  • Ablation Study Validation: Table 4 confirms the effectiveness of the GFA module and the weightage parameters, showing performance degradation when these components are removed or replaced. Table 5 demonstrates that replacing standard Transformer blocks with SSCformer improves existing methods.

Limitations and Future Work

  • Limitations: The paper doesn't explicitly detail limitations, but the reliance on Transformer architectures might imply computational costs for very large datasets or extremely high-resolution images. The focus is primarily on denoising, and while classification accuracy is used as an evaluation metric, it's not the primary task.
  • Future Work: The authors suggest extending their method for allied higher-level remote sensing tasks, such as object detection. This indicates a belief in the generalizability of their spatial-spectral attention mechanism beyond just denoising.

Summary updated

BibTeX
@inproceedings{dixit2025visionary,
  author = {Aditya Dixit and Nischit Hosamani and Puneet Gupta and Ankur Garg},
  title = {{VISIONARY: Novel Spatial-Spectral Attention Mechanism for Hyperspectral Image Denoising}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
}
Figure