Parallel channel shuffling and Transformer-based denoising for hyperspectral images

S. P. Hu, Feng Gao, Zhuoran Gong, Shengen Tao, Xinyu ShangGuan, Junyu DongJournal of Image and Graphics 2024 · DOI

Abstract

Hyperspectral image (HSI) denoising is a critical task in remote sensing, as noise significantly degrades image visibility and analysis accuracy. Current methods often struggle to balance the extraction of global context with the preservation of local spatial textures. This paper introduces PSFormer, a novel denoising model that integrates Transformers with parallel channel-shuffling convolutional neural networks (CNNs). By employing a Channel Shuffling Module (CSM), a Block Downsampling Global Enhancement Module (DGEM), and an Adaptive Bidirectional Feature Fusion Module (ABFM), the proposed architecture effectively captures both long-range dependencies and fine-grained details. Experimental results on the Pavia and ICVL datasets demonstrate that PSFormer significantly outperforms existing state-of-the-art methods, achieving up to a 2.18 dB improvement in Peak Signal-to-Noise Ratio (PSNR) under complex noise conditions.

Introduction

Hyperspectral images provide rich spectral information across hundreds of bands, but they are highly susceptible to noise from sensors and atmospheric interference. Traditional CNN-based denoisers are limited by local receptive fields, failing to capture the global spatial patterns necessary for high-quality restoration. Conversely, while Transformers excel at global context modeling, their structural differences from CNNs make seamless integration challenging. PSFormer addresses these issues through a dual-branch architecture that processes local and global features in parallel, ensuring comprehensive feature extraction and robust denoising.

Methodology

The PSFormer architecture is built around three core components designed to optimize information flow and feature integration:

  1. Channel Shuffling Module (CSM): This module handles the high dimensionality of HSI data. By grouping and shuffling channels, it promotes information exchange across the spectral dimension while extracting local spatial features efficiently without a massive increase in parameters.
  2. Block Downsampling Global Enhancement Module (DGEM): To capture global structures while maintaining manageable computational costs, this Transformer-based branch uses a block downsampling strategy. This increases the receptive field, allowing the model to perceive larger image regions.
  3. Adaptive Bidirectional Feature Fusion Module (ABFM): This module acts as a bridge, consisting of Global Feature Fusion (GFM) and Local Feature Fusion (LFM) components. It allows for two-way interaction, ensuring that the local branch benefits from global context and the global branch retains fine-grained spatial details.
The overall architecture of PSFormer, showing the dual-branch structure and the interaction between CSM and DGEM modules.

Multi-Scale Channel Mixing

A key novelty is the CSM's ability to perform multi-scale channel mixing. As shown in the detailed module diagram, the system uses group convolutions and shuffling to ensure that features from different spectral bands are effectively integrated, which is crucial for maintaining spectral integrity during the denoising process.

Detailed view of the Channel Shuffling Module (CSM) illustrating the channel grouping and shuffling mechanism for efficient feature extraction.

Experimental Results

The model was rigorously tested against five mainstream denoising methods, including BM4D, LRMR, HSID-CNN, and QRNN3D, using the ICVL and Pavia University datasets.

  • Quantitative Performance: On the Pavia dataset under high Gaussian noise, PSFormer improved PSNR by up to 0.4 dB over the next best model. On the ICVL dataset with mixed noise (Gaussian, impulse, and stripes), the improvement reached a substantial 2.18 dB.
  • Qualitative Quality: Visual comparisons indicate that PSFormer excels at recovering sharp edges and intricate textures where other models produce over-smoothed or artifact-laden results.
  • Spectral Fidelity: Using the Spectral Angle Mapper (SAM) metric, the researchers confirmed that PSFormer preserves the unique spectral signatures of materials better than competing models, which is vital for downstream classification tasks.

Conclusion

PSFormer represents a significant advancement in HSI denoising by successfully merging the strengths of CNNs and Transformers. The parallel architecture and adaptive fusion mechanism allow for superior noise removal while preserving the rich spatial and spectral information inherent in hyperspectral data. Future work will focus on optimizing the model's computational efficiency for real-time remote sensing applications and exploring its utility in other tasks like HSI super-resolution.

Summary updated

Abstract

目的高光谱图像因设备及环境因素容易受到噪声污染,导致图像的可见性和分析精度降低,因此高光谱图像去噪任务已经成为遥感图像处理领域国内外研究热点。当前的高光谱图像去噪方法主要面临两个难题:1)对特征的全局信息利用不足。当前基于卷积神经网络的方法受限于卷积核的大小,难以捕获特征的全局信息;2)卷积神经网络和Transformer在结构上存在差异,导致两者难以融合,因此,需要考虑合理的特征交互方式,来平衡局部和全局特征提取之间的关系。方法针对上述问题,本文提出了基于Transformer和通道混合并行卷积的高光谱图像去噪模型,包括3个模块:通道混合特征提取模块、基于块下采样的全局增强模块和自适应双向特征融合模块。通过这3个模块的相互作用,可以充分结合全局和局部的特征信息,处理不同区域中的噪声和纹理差异,有效提高模型对空间细节信息的恢复能力。结果实验在2个数据集上与主流的5种方法进行比较,在Pavia数据集中设置不同高斯噪声强度的情况下,相比于性能第2的模型,峰值信噪比(peak signal-to-noise ratio,PSNR)值最大提高了0.4 dB;在ICVL数据集中设置各种混合噪声的情况下,相比于性能第2的模型,PSNR最大提高了2.18 dB。同时可视化的去噪结果图像体现了本文所提出的去噪模型的优异性能。结论本文方法在各种噪声情况下均具有较好的去噪效果,显著优于当前主流方法,能够有效去除高光谱图像中噪声,同时保留图像丰富的纹理信息。
BibTeX
@article{hu2024parallel,
  author = {S. P. Hu and Feng Gao and Zhuoran Gong and Shengen Tao and Xinyu ShangGuan and Junyu Dong},
  title = {{Parallel channel shuffling and Transformer-based denoising for hyperspectral images}},
  journal = {Journal of Image and Graphics},
  year = {2024},
  doi = {10.11834/jig.230381},
}
Figure