Detail Matters: Mamba-Inspired Joint Unfolding Network for Snapshot Spectral Compressive Imaging

Mengjie Qin, Yuchao Feng, Zongliang Wu, Yulun Zhang, Xin Yuan

Abstract

Snapshot Spectral Compressive Imaging (SSCI) systems, such as Coded Aperture Snapshot Spectral Imaging (CASSI), enable high-speed 3D hyperspectral data acquisition via a single 2D measurement. However, reconstructing high-fidelity 3D cubes from these compressed measurements remains a significant challenge. We present MiJUN, a Mamba-inspired Joint Unfolding Network that addresses the limitations of existing Deep Unfolding Networks (DUNs) and Transformer-based methods. By integrating an accelerated optimization framework with a novel multi-mode Mamba architecture, MiJUN achieves state-of-the-art reconstruction quality with superior computational efficiency.

Introduction

Capturing hyperspectral images (HSI) typically requires scanning in either the spatial or spectral domain, which is time-consuming and unsuitable for dynamic scenes. CASSI systems circumvent this by using a coded aperture and a disperser to compress 3D spectral information into a 2D snapshot. The reconstruction process—mapping this 2D measurement back to a 3D cube—is a highly ill-posed inverse problem.

Existing reconstruction methods generally fall into two categories: Deep Unfolding Networks (DUNs), which map iterative optimization steps into neural layers for interpretability, and Transformers, which excel at capturing long-range dependencies but suffer from quadratic computational complexity and a tendency to blur local textures. Recently, State Space Models (Mamba) have emerged as a linear-time alternative. However, standard Mamba implementations often struggle with spatial relationships in images due to their 1D sequence processing. MiJUN is designed to bridge these gaps by combining the interpretability of DUNs with the efficiency of Mamba and the precision of attention mechanisms.

The MiJUN Framework

MiJUN introduces three core innovations to enhance HSI reconstruction:

1. Accelerated Half-Quadratic Splitting (A-HQS)

Traditional DUNs often rely on first-order optimization schemes. MiJUN adopts an accelerated scheme based on second-order differential equations. This A-HQS framework improves convergence speed and reduces the network's sensitivity to initial noisy estimates, allowing for high-quality results with fewer unfolding stages.

2. Mamba-inspired Transformer (MMIT)

To mitigate the loss of local spatial information common in Mamba-based models, the authors propose the MMIT block. This block integrates a Global-to-Local Attention Mechanism (GLAM), enabling the model to capture broad contextual information while simultaneously preserving fine-grained local textures.

3. Mode-k Mamba (M-k Mamba)

Hyperspectral data is inherently a 3D tensor with strong correlations across spatial and spectral dimensions. Instead of simple 1D scanning, the M-k Mamba block performs tensor mode-k unfolding. This allows the model to scan the data from 12 different directions across the 3D cube, effectively capturing the low-rank properties and multi-dimensional dependencies of the scene.

The overall architecture of MiJUN, featuring the A-HQS optimization framework and the Mamba-inspired Joint Unfolding Network.

Experimental Results

MiJUN was rigorously evaluated on standard HSI datasets, including CAVE and KAIST for training and testing, and the ICVL dataset to assess generalization. It was also tested on real-world data captured by physical CASSI hardware.

Quantitative and Qualitative Performance

MiJUN sets a new benchmark for SSCI reconstruction. A 5-stage version of MiJUN (MiJUN-5stg) outperformed the previous state-of-the-art model, RDLUF-MixS2-9stg, by 1.01 dB in PSNR. Crucially, this performance boost was achieved with 3x fewer parameters and 3x lower computational cost (FLOPs).

Qualitatively, MiJUN demonstrates a superior ability to recover sharp edges and intricate details, such as bird feathers or fine hair, which are often smeared or lost by Transformer-based methods. The inclusion of the M-k Mamba block specifically contributes to the recovery of cleaner feature maps and more accurate spectral signatures.

Comparison of PSNR vs. Parameters across various SOTA methods, alongside a visual demonstration of the impact of the M-k Mamba block.

Conclusion

MiJUN represents a significant advancement in hyperspectral image reconstruction. By re-interpreting Mamba as a linear-time Transformer variant and applying multi-mode tensor unfolding, the model achieves an optimal balance between global context and local detail. While real-world hardware noise remains a challenge for all SSCI methods, MiJUN's efficiency and fidelity make it a powerful tool for high-speed spectral imaging applications. Future work will focus on enhancing robustness against physical hardware imperfections and further optimizing the scanning strategies.

Summary updated

BibTeX
@inproceedings{qin2025detail,
  author = {Mengjie Qin and Yuchao Feng and Zongliang Wu and Yulun Zhang and Xin Yuan},
  title = {{Detail Matters: Mamba-Inspired Joint Unfolding Network for Snapshot Spectral Compressive Imaging}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i6.32707},
}
Figure