Abstract
Snapshot Spectral Compressive Imaging (SSCI) systems, such as Coded Aperture Snapshot Spectral Imaging (CASSI), enable high-speed 3D hyperspectral data acquisition via a single 2D measurement. However, reconstructing high-fidelity 3D cubes from these compressed measurements remains a significant challenge. We present MiJUN, a Mamba-inspired Joint Unfolding Network that addresses the limitations of existing Deep Unfolding Networks (DUNs) and Transformer-based methods. By integrating an accelerated optimization framework with a novel multi-mode Mamba architecture, MiJUN achieves state-of-the-art reconstruction quality with superior computational efficiency.
Introduction
Capturing hyperspectral images (HSI) typically requires scanning in either the spatial or spectral domain, which is time-consuming and unsuitable for dynamic scenes. CASSI systems circumvent this by using a coded aperture and a disperser to compress 3D spectral information into a 2D snapshot. The reconstruction process—mapping this 2D measurement back to a 3D cube—is a highly ill-posed inverse problem.
Existing reconstruction methods generally fall into two categories: Deep Unfolding Networks (DUNs), which map iterative optimization steps into neural layers for interpretability, and Transformers, which excel at capturing long-range dependencies but suffer from quadratic computational complexity and a tendency to blur local textures. Recently, State Space Models (Mamba) have emerged as a linear-time alternative. However, standard Mamba implementations often struggle with spatial relationships in images due to their 1D sequence processing. MiJUN is designed to bridge these gaps by combining the interpretability of DUNs with the efficiency of Mamba and the precision of attention mechanisms.
The MiJUN Framework
MiJUN introduces three core innovations to enhance HSI reconstruction:
1. Accelerated Half-Quadratic Splitting (A-HQS)
Traditional DUNs often rely on first-order optimization schemes. MiJUN adopts an accelerated scheme based on second-order differential equations. This A-HQS framework improves convergence speed and reduces the network's sensitivity to initial noisy estimates, allowing for high-quality results with fewer unfolding stages.
2. Mamba-inspired Transformer (MMIT)
To mitigate the loss of local spatial information common in Mamba-based models, the authors propose the MMIT block. This block integrates a Global-to-Local Attention Mechanism (GLAM), enabling the model to capture broad contextual information while simultaneously preserving fine-grained local textures.
3. Mode-k Mamba (M-k Mamba)
Hyperspectral data is inherently a 3D tensor with strong correlations across spatial and spectral dimensions. Instead of simple 1D scanning, the M-k Mamba block performs tensor mode-k unfolding. This allows the model to scan the data from 12 different directions across the 3D cube, effectively capturing the low-rank properties and multi-dimensional dependencies of the scene.
Experimental Results
MiJUN was rigorously evaluated on standard HSI datasets, including CAVE and KAIST for training and testing, and the ICVL dataset to assess generalization. It was also tested on real-world data captured by physical CASSI hardware.
Quantitative and Qualitative Performance
MiJUN sets a new benchmark for SSCI reconstruction. A 5-stage version of MiJUN (MiJUN-5stg) outperformed the previous state-of-the-art model, RDLUF-MixS2-9stg, by 1.01 dB in PSNR. Crucially, this performance boost was achieved with 3x fewer parameters and 3x lower computational cost (FLOPs).
Qualitatively, MiJUN demonstrates a superior ability to recover sharp edges and intricate details, such as bird feathers or fine hair, which are often smeared or lost by Transformer-based methods. The inclusion of the M-k Mamba block specifically contributes to the recovery of cleaner feature maps and more accurate spectral signatures.
Conclusion
MiJUN represents a significant advancement in hyperspectral image reconstruction. By re-interpreting Mamba as a linear-time Transformer variant and applying multi-mode tensor unfolding, the model achieves an optimal balance between global context and local detail. While real-world hardware noise remains a challenge for all SSCI methods, MiJUN's efficiency and fidelity make it a powerful tool for high-speed spectral imaging applications. Future work will focus on enhancing robustness against physical hardware imperfections and further optimizing the scanning strategies.
Summary updated