PanComplex: Leveraging Complex-Valued Neural Networks for Enhanced Pansharpening

Chunhui Luo, Dong Li, Xiaoliang Ma, Xin Lu, Zhiyuan Wang, Jiangtong Tan, Xueyang FuIJCAI 2025 · OpenReview

Abstract

Pansharpening combines panchromatic and low-resolution multispectral images to generate high-resolution multispectral images. Previous studies have explored the connection between pansharpening and the frequency domain, but mostly in the real-valued domain, leaving the complex domain relatively unexplored. To redefine the pansharpening task, we propose a complex-valued spatial-frequency dual-domain framework, PanComplex. To achieve this, we first establish complex representations and introduce basic complex operators tailored to pansharpening, enabling the transformation of multispectral real-valued signals into the complex domain for learning. We then model both spatial and frequency branches to capture global frequency features and local spatial features comprehensively. Finally, we employ a complex-based interaction module to fuse the spatial and frequency features, achieving complementary information across both domains. By using the representation power of the complex domain, PanComplex effectively extracts complementary features from PAN and MS images, thereby enhancing pansharpening performance. Experiments on multiple datasets demonstrate that our method achieves optimal performance with the fewest parameters and exhibits strong generalization ability to other tasks. The source code for this work is publicly available at https://github.com/lch-ustc/PanComplex.

Introduction

Pansharpening is a fundamental process in remote sensing, essential for overcoming the physical limitations of satellite sensors. While sensors can capture high-resolution panchromatic (PAN) images or low-resolution multispectral (MS) images, they typically cannot capture high-resolution multispectral (HRMS) data directly. Pansharpening bridges this gap by merging the spatial details of PAN images with the spectral richness of MS images.

Recent advancements in deep learning have significantly improved pansharpening results. However, most existing models operate strictly within the real-valued domain. When researchers attempt to leverage frequency domain information via Fourier Transforms, they often decompose the resulting complex signals into separate magnitude and phase components or real and imaginary parts. This "split" approach treats these components as independent real-valued channels, which can disrupt the inherent phase information and the internal correlations between spectral and spatial data.

Comparison of traditional real-valued processing versus the proposed complex-valued dual-domain framework.

The PanComplex Framework

To address these limitations, we introduce PanComplex, a novel framework that treats image features as complex numbers ($z = a + ib$). By operating directly in the complex domain, the model preserves the structural integrity of signals, particularly in the frequency domain where data is naturally complex-valued.

Dual-Domain Feature Extraction

The architecture consists of three core components designed to capture complementary information:

  1. Complex Frequency Feature Extractor (CFFE): This module applies a Discrete Fourier Transform (DFT) to the input images. Unlike traditional methods, CFFE utilizes complex convolutions to operate directly on the Fourier spectra. This approach maintains the phase-magnitude relationship, which is crucial for preserving structural details.
  2. Complex Spatial Feature Extractor (CSFE): In the spatial domain, the MS image is represented as the real part and the PAN image as the imaginary part of a unified complex feature map. This allows the network to model the cross-modal interaction between spectral and spatial data as a single complex signal.
  3. Complex Spatial-Frequency Interaction (CSFI): This module fuses features from both the spatial and frequency branches. It employs a complex-valued attention mechanism to dynamically weigh information, ensuring that the most relevant details from both domains are integrated before being converted back to a real-valued HRMS image.
The PanComplex architecture (a) and the mathematical mechanics of complex convolution (b).

The Power of Complex Convolutions

The primary innovation lies in the "complex-first" design. As shown in the complex convolution diagram (b), a complex convolution involves four real-valued convolutions but is treated as a single operation. This allows the model to achieve higher representational power with a significantly smaller parameter footprint. Mathematically, if $Z = A + iB$ and $K = X + iY$, the convolution is calculated as $(A*X - B*Y) + i(A*Y + B*X)$.

Experimental Results and Insights

PanComplex was evaluated on three major satellite benchmarks: WorldView-II (WV2), WorldView-III (WV3), and GaoFen2 (GF2).

Performance and Efficiency

The model consistently outperformed state-of-the-art (SOTA) real-valued models like FAME-Net and HFIN. On the WorldView-II dataset, it achieved a PSNR of 42.3058, the highest among all tested methods. Remarkably, PanComplex achieves these results with only 54.2K parameters, which is less than half the size of models like GPPNN (119.8K) and FAME-Net (140.8K).

Key Insights

  • Phase Preservation: Phase information often carries more structural detail than magnitude in signal processing. By staying in the complex domain, PanComplex avoids the distortions common in real-valued networks.
  • Biological Inspiration: Complex-valued units more closely simulate biological neurons, which encode information through the relative timing (phase) of pulses rather than just firing rates.
  • Generalization: Beyond pansharpening, the framework demonstrated robust performance on depth image super-resolution (NYU v2) and infrared-visible image fusion (RoadScene), suggesting that complex-valued processing is a versatile tool for various image fusion tasks.

Conclusion

PanComplex redefines pansharpening by leveraging the mathematical elegance and representational power of the complex domain. By integrating spatial and frequency information through a unified complex-valued framework, it achieves superior accuracy and visual quality with unprecedented parameter efficiency. This work highlights the untapped potential of complex-valued neural networks in remote sensing and broader computer vision applications.

Summary updated

Abstract

Pansharpening combines panchromatic and low-resolution multispectral images to generate high-resolution multispectral images. Previous studies have explored the connection between pansharpening and the frequency domain, but mostly in the real-valued domain, leaving the complex domain relatively unexplored. To redefine the pansharpening task, we propose a complex-valued spatial-frequency dual-domain framework, PanComplex. To achieve this, we first establish complex representations and introduce basic complex operators tailored to pansharpening, enabling the transformation of multispectral real-valued signals into the complex domain for learning. We then model both spatial and frequency branches to capture global frequency features and local spatial features comprehensively. Finally, we employ a complex-based interaction module to fuse the spatial and frequency features, achieving complementary information across both domains. By using the representation power of the complex domain, PanComplex effectively extracts complementary features from PAN and MS images, thereby enhancing pansharpening performance. Experiments on multiple datasets demonstrate that our method achieves optimal performance with the fewest parameters and exhibits strong generalization ability to other tasks. The source code for this work is publicly available at https://github.com/lch-ustc/PanComplex.
BibTeX
@inproceedings{luo2025pancomplex,
  author = {Chunhui Luo and Dong Li and Xiaoliang Ma and Xin Lu and Zhiyuan Wang and Jiangtong Tan and Xueyang Fu},
  title = {{PanComplex: Leveraging Complex-Valued Neural Networks for Enhanced Pansharpening}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=7LFIAfibwu},
}
Figure