Abstract
Hyperspectral pansharpening is a critical task in remote sensing, aimed at fusing low-resolution hyperspectral (LRHS) images with high-resolution panchromatic (PAN) images to produce high-resolution hyperspectral (HRHS) data. Traditional deep learning approaches often utilize static architectures that struggle with spatial variations and feature redundancy. This paper introduces FMPM-DNet, a novel dual-stage dynamic network that leverages complex-domain wave functions and probabilistic masks. By treating features as waves and employing selective embedding via learnable masks, FMPM-DNet achieves superior fusion performance, significantly reducing spectral and spatial distortions across multiple benchmarks.
Introduction
In remote sensing, a fundamental trade-off exists between spectral and spatial resolution. Hyperspectral images provide rich spectral information across hundreds of bands but often at a coarse spatial resolution. Conversely, panchromatic sensors capture high-resolution spatial details but lack spectral depth. Hyperspectral pansharpening seeks to bridge this gap. While deep learning has advanced the field, many existing models are "static," with fixed weights that cannot adapt to the diverse spatial characteristics of different scenes. Furthermore, common feature fusion techniques like simple concatenation often introduce noise and redundancy.
Proposed Methodology: FMPM-DNet
The FMPM-DNet architecture addresses these limitations through a two-stage process: Spectral-Spatial Feature Modulation and Feature Reconstruction.
1. Spectral-Spatial Feature Modulation Stage
This stage utilizes a Dynamic Feature Modulation Unit (DFMU). Instead of standard convolutions, the network defines a wave function based on complex transformations. By mapping spatial features into the complex domain, the model can represent both intensity (amplitude) and structural patterns (phase) more effectively. The DFMU generates adaptive matrices to modulate hyperspectral features using the PAN image as a guide, ensuring a spatially accurate "coarse" fusion.
2. Feature Reconstruction Stage
To refine the coarse fusion, the Feature Probability Mask Unit (FPMU) is introduced. Rather than indiscriminately stacking features, the FPMU uses learnable probability masks to select the most relevant information. This selection process is made differentiable using the Gumbel-Softmax trick. The unit incorporates Global (GPES) and Local (LPES) structures to optimize feature selection across network depths and spatial orientations, effectively filtering out redundant data.
Experimental Results
The model was rigorously evaluated on three benchmark datasets: Pavia Center, Botswana, and Chikusei, following Wald’s protocol.
Quantitative Performance
FMPM-DNet consistently outperformed state-of-the-art (SOTA) methods. On the Pavia Center dataset, it achieved a Peak Signal-to-Noise Ratio (PSNR) of 38.8193, surpassing the previous best method, Tree-SNet (38.1569). It also recorded the lowest Spectral Angle Mapper (SAM) values, indicating minimal spectral distortion.
Qualitative Analysis
Visual comparisons and error maps (Mean Absolute Error) highlight the model's ability to preserve sharp edges and accurate spectral signatures.
As shown in the error maps, FMPM-DNet exhibits significantly lower residuals, particularly around complex urban structures and agricultural boundaries, compared to methods like DARN or HyperRefiner.
Conclusion
FMPM-DNet represents a significant advancement in hyperspectral pansharpening by introducing dynamic, wave-based feature modulation and probabilistic feature selection. By moving beyond static architectures and redundant feature stacking, the network achieves state-of-the-art performance in both reconstruction accuracy and downstream utility, such as land-cover classification. Future work will explore the model's robustness against real-world sensor noise and its applicability to other multi-modal fusion tasks.
This summary is based on the paper "FMPM-DNet: Hyperspectral Pansharpening Dynamic Network Based on Feature Modulation and Probability Mask" presented at AAAI-25.
Summary updated