FMPM-DNet: Hyperspectral Pansharpening Dynamic Network Based on Feature Modulation and Probability Mask

Xiaozheng Wang, Yong Yang, Shuying Huang, Hangyuan Lu, Weiguo Wan, Aoqi ZhaoAAAI 2025 · DOI

Abstract

Hyperspectral pansharpening is a critical task in remote sensing, aimed at fusing low-resolution hyperspectral (LRHS) images with high-resolution panchromatic (PAN) images to produce high-resolution hyperspectral (HRHS) data. Traditional deep learning approaches often utilize static architectures that struggle with spatial variations and feature redundancy. This paper introduces FMPM-DNet, a novel dual-stage dynamic network that leverages complex-domain wave functions and probabilistic masks. By treating features as waves and employing selective embedding via learnable masks, FMPM-DNet achieves superior fusion performance, significantly reducing spectral and spatial distortions across multiple benchmarks.

Introduction

In remote sensing, a fundamental trade-off exists between spectral and spatial resolution. Hyperspectral images provide rich spectral information across hundreds of bands but often at a coarse spatial resolution. Conversely, panchromatic sensors capture high-resolution spatial details but lack spectral depth. Hyperspectral pansharpening seeks to bridge this gap. While deep learning has advanced the field, many existing models are "static," with fixed weights that cannot adapt to the diverse spatial characteristics of different scenes. Furthermore, common feature fusion techniques like simple concatenation often introduce noise and redundancy.

Proposed Methodology: FMPM-DNet

The FMPM-DNet architecture addresses these limitations through a two-stage process: Spectral-Spatial Feature Modulation and Feature Reconstruction.

The detailed architecture of FMPM-DNet, showcasing the Spectral-Spatial Feature Modulation Stage and the Feature Reconstruction Stage with DFMU and FPMU components.

1. Spectral-Spatial Feature Modulation Stage

This stage utilizes a Dynamic Feature Modulation Unit (DFMU). Instead of standard convolutions, the network defines a wave function based on complex transformations. By mapping spatial features into the complex domain, the model can represent both intensity (amplitude) and structural patterns (phase) more effectively. The DFMU generates adaptive matrices to modulate hyperspectral features using the PAN image as a guide, ensuring a spatially accurate "coarse" fusion.

2. Feature Reconstruction Stage

To refine the coarse fusion, the Feature Probability Mask Unit (FPMU) is introduced. Rather than indiscriminately stacking features, the FPMU uses learnable probability masks to select the most relevant information. This selection process is made differentiable using the Gumbel-Softmax trick. The unit incorporates Global (GPES) and Local (LPES) structures to optimize feature selection across network depths and spatial orientations, effectively filtering out redundant data.

Experimental Results

The model was rigorously evaluated on three benchmark datasets: Pavia Center, Botswana, and Chikusei, following Wald’s protocol.

Quantitative Performance

FMPM-DNet consistently outperformed state-of-the-art (SOTA) methods. On the Pavia Center dataset, it achieved a Peak Signal-to-Noise Ratio (PSNR) of 38.8193, surpassing the previous best method, Tree-SNet (38.1569). It also recorded the lowest Spectral Angle Mapper (SAM) values, indicating minimal spectral distortion.

Qualitative Analysis

Visual comparisons and error maps (Mean Absolute Error) highlight the model's ability to preserve sharp edges and accurate spectral signatures.

Visual comparison of pansharpening results and corresponding error maps across different methods on benchmark datasets. FMPM-DNet (Ours) shows the least residual error.

As shown in the error maps, FMPM-DNet exhibits significantly lower residuals, particularly around complex urban structures and agricultural boundaries, compared to methods like DARN or HyperRefiner.

Conclusion

FMPM-DNet represents a significant advancement in hyperspectral pansharpening by introducing dynamic, wave-based feature modulation and probabilistic feature selection. By moving beyond static architectures and redundant feature stacking, the network achieves state-of-the-art performance in both reconstruction accuracy and downstream utility, such as land-cover classification. Future work will explore the model's robustness against real-world sensor noise and its applicability to other multi-modal fusion tasks.

This summary is based on the paper "FMPM-DNet: Hyperspectral Pansharpening Dynamic Network Based on Feature Modulation and Probability Mask" presented at AAAI-25.

Summary updated

BibTeX
@inproceedings{wang2025fmpm,
  author = {Xiaozheng Wang and Yong Yang and Shuying Huang and Hangyuan Lu and Weiguo Wan and Aoqi Zhao},
  title = {{FMPM-DNet: Hyperspectral Pansharpening Dynamic Network Based on Feature Modulation and Probability Mask}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i1.32070},
}
Figure