Abstract
Hyperspectral image (HSI) denoising is critical for downstream remote sensing tasks but faces a fundamental trade-off between preserving fine spectral details and capturing broad contextual information. This paper introduces the Multi-scale Adaptive Fusion Network (MAFNet), a deep learning architecture that addresses this challenge by intelligently blending information across different spatial scales. By leveraging a co-attention mechanism and adaptive instance normalization, MAFNet effectively reconstructs noisy HSIs while maintaining sharp textures and spectral consistency, outperforming current state-of-the-art methods.
The Challenge: Detail Preservation vs. Contextual Modeling
HSIs provide a rich "3D data cube" by capturing data across hundreds of narrow spectral bands. However, these images are frequently corrupted by complex noise types, including Gaussian noise, impulse noise, stripes, and deadlines.
A core difficulty in denoising is the limited receptive field of high-resolution processing, which excels at preserving sharp textures but fails to capture global context. Conversely, downsampling to capture broader patterns often results in the loss of essential fine-scale details. Existing multiscale models often struggle to effectively exchange and fuse information between these disparate resolutions.
The MAFNet Architecture
To overcome these limitations, the authors propose MAFNet, which processes features in parallel across multiple scales and adaptively fuses them. The network is organized into four primary stages:
- Initial Layer: Extracts shallow features from the noisy input cube.
- Coarse-Fusion Network: Utilizes Adaptive Instance Normalization (AIN) modules. These modules use features from lower-resolution branches to "guide" high-resolution branches, transferring structural information to assist in the reconstruction of fine details.
- Fine-Fusion Network: Maintaining parallel multiscale subnetworks (inspired by HRNet), this stage facilitates continuous information exchange. The Co-attention Fusion Module dynamically weights features from different scales, ensuring the most informative data is prioritized for each pixel.
- Noise Reconstruction: The network employs residual learning to predict the noise component rather than the clean image directly, which is then subtracted from the original input to produce the final result.
Key Innovations and Training Strategy
Beyond its architecture, MAFNet introduces several key technical contributions:
- Co-attention Mechanism: Unlike standard networks that simply sum or concatenate features, MAFNet's co-attention recognizes that different scales have varying utility across the image, modulating the fusion process adaptively.
- Global Gradient Regularization: A specialized loss term penalizes differences in horizontal, vertical, and spectral gradients, ensuring the denoised output maintains sharp edges and spectral integrity.
- Incremental Learning Policy: The model was trained using a curriculum learning approach, starting with simple Gaussian noise and progressively introducing complex "mixture noise." This strategy was found to improve convergence speed and final performance.
Experimental Results
MAFNet was evaluated on standard benchmarks including the ICVL, CAVE, and Pavia University datasets. It consistently outperformed established methods such as BM4D and deep learning baselines like QRNN3D.
- Quantitative Performance: On the ICVL dataset, MAFNet achieved a PSNR of 43.97 dB, surpassing QRNN3D (43.46 dB). It also demonstrated superior Spectral Angle Mapper (SAM) scores, indicating higher fidelity in preserving spectral signatures.
- Qualitative Quality: Visual analysis confirms that MAFNet is particularly effective at removing complex mixture noise (stripes and deadlines) without the blurring artifacts common in low-rank methods.
- Efficiency: Despite its multi-branch structure, MAFNet is computationally efficient, requiring fewer FLOPs than heavy-duty 3D-CNN models while delivering higher-quality results.
Conclusion
MAFNet represents a significant advancement in HSI denoising by successfully bridging the gap between local detail and global context. While the current model is highly effective, future research will focus on developing lightweight versions of this architecture to enable real-time processing on edge devices, such as drone-mounted sensors.
Summary updated