Abstract
Hyperspectral imaging offers new perspectives for diverse applications, ranging from the monitoring of the environment using airborne or satellite remote sensing, precision farming, food safety, planetary exploration, or astrophysics. Unfortunately, the spectral diversity of information comes at the expense of various sources of degradation, and the lack of accurate ground-truth "clean" hyperspectral signals acquired on the spot makes restoration tasks challenging. In particular, training deep neural networks for restoration is difficult, in contrast to traditional RGB imaging problems where deep models tend to shine. In this paper, we advocate instead for a hybrid approach based on sparse coding principles that retains the interpretability of classical techniques encoding domain knowledge with handcrafted image priors, while allowing to train model parameters end-to-end without massive amounts of data. We show on various denoising benchmarks that our method is computationally efficient and significantly outperforms the state of the art.
Introduction
Hyperspectral imaging (HSI) captures a scene across hundreds of narrow spectral bands, providing a unique "spectral signature" for every pixel. This capability is invaluable for fields like precision farming and astrophysics. However, HSI data is frequently degraded by complex noise—including Gaussian noise, thermal noise, and sensor-specific artifacts like stripes.
A significant challenge in HSI restoration is the scarcity of clean ground-truth data. Unlike standard RGB photography, large-scale datasets of noise-free hyperspectral images are non-existent. This makes training traditional deep learning models, such as CNNs, difficult and prone to overfitting. Historically, researchers have relied on learning-free classical methods that use handcrafted priors (e.g., low-rank or sparsity). While interpretable and data-efficient, these methods are often slow and difficult to tune.
To address these issues, we propose T3SC (Trainable Spectral-Spatial Sparse Coding), a hybrid model that combines the interpretability of sparse coding with the learning power of deep neural networks.
Methodology: The T3SC Architecture
The T3SC model is designed as a two-layer architecture that mimics the physical properties of hyperspectral data.
1. Spectral Sparse Coding (Layer 1)
This layer processes each pixel individually, decomposing its spectrum into a sparse combination of basis elements. This effectively performs automated "spectral unmixing," identifying the underlying materials within a single pixel.
2. Spectral-Spatial Sparse Coding (Layer 2)
To capture spatial relationships and textures, this layer operates on patches of pixels. It leverages the spatial redundancy inherent in natural scenes to further refine the denoising process.
3. Key Innovations
- Low-Rank Dictionaries: The dictionaries (basis elements) are constrained to be low-rank, which keeps the model lightweight and reduces the number of parameters to learn.
- Unrolled Optimization (LISTA): Instead of a generic CNN, the model "unrolls" the classical Iterative Soft-Thresholding Algorithm (ISTA) into a fixed number of layers. This makes the entire process differentiable, allowing for end-to-end training via backpropagation.
- Noise Adaptation: A small sub-network estimates band-wise noise levels, allowing the model to dynamically adjust its denoising strength at test time.
- Self-Supervised Learning (T3SC-SSL): For scenarios without ground truth, we introduce a "blind-band" approach where the model learns to reconstruct randomly masked spectral bands from the remaining ones.
Experiments and Results
We evaluated T3SC on several standard HSI benchmarks, including ICVL (high-resolution images with 31 bands) and the Washington DC Mall (remote sensing data with 191 bands).
Performance and Efficiency
T3SC consistently outperformed or matched state-of-the-art supervised and unsupervised methods (such as BM4D, GLF, and QRNN3D) across various noise patterns, including Gaussian noise and stripes. Notably, T3SC is significantly faster than high-performing classical methods. For instance, on the ICVL dataset, T3SC takes approximately 5.8 seconds per image, whereas the classical NGMeet method requires 2686 seconds.
Interpretability and Robustness
The sparse coding foundation provides inherent interpretability: the first layer maps different sensors to a common feature space, while the second layer handles spatial denoising. Furthermore, the internal noise estimator makes the model robust to "blind" denoising tasks where noise levels are unknown beforehand.
Conclusion
T3SC demonstrates that hybridity is a powerful strategy for HSI restoration. By embedding domain knowledge into a trainable architecture, we achieve high performance with limited data. While the self-supervised version (T3SC-SSL) still faces challenges with complex, non-Gaussian noise, the overall framework provides a computationally efficient and interpretable path forward for hyperspectral imaging. Future work will focus on making the pipeline more sensor-agnostic and improving self-supervised handling of structured noise like stripes.
Summary updated