Abstract
Standard convolutional neural networks (CNNs) for pansharpening often suffer from high parameter redundancy and computational costs, making them difficult to deploy on resource-constrained satellite hardware. This paper introduces SpanConv, a lightweight convolution operation that treats the kernel space as a mathematical subspace. By learning a small set of "navigated kernels" (basis vectors) and spanning them with learnable coefficients, the authors develop LightNet. With only ~16,000 parameters, LightNet achieves state-of-the-art performance, outperforming models with millions of parameters while demonstrating superior generalization to unseen satellite data.
Introduction
Pansharpening is a critical task in remote sensing, involving the fusion of a high-resolution panchromatic (PAN) image with a low-resolution multispectral (MS) image. While deep learning has significantly improved fusion accuracy, traditional CNNs are often over-parameterized. This redundancy leads to three main issues: excessive computational cost for satellite deployment, potential overfitting to simulated training data, and a lack of interpretability in kernel weights. SpanConv addresses these challenges by redesigning the fundamental convolution operation through the lens of linear algebra.
The SpanConv Method
The core innovation, SpanConv (Spanning Convolution), replaces standard independent weight learning with a spanning strategy. Instead of learning every weight in a $C_{out} \times C_{in} \times k \times k$ kernel independently, SpanConv learns a small number ($N$) of "navigated kernels" ($\bar{K}_1, \bar{K}_2, \dots, \bar{K}_N$) which act as basis vectors.
For each input channel, the model learns specific coefficients ($\lambda$) to linearly combine these navigated kernels. This process "spans" a kernel space that effectively approximates standard convolution but with a fraction of the parameters.
The authors implemented this in LightNet, a streamlined architecture consisting of a Fusion Head (concatenating PAN and MS), an Extraction Belly (using "belly blocks" for feature extraction), and a Reconstruction Tail.
Analysis of Kernel Redundancy
A key question in designing SpanConv is determining the optimal number of navigated kernels ($N$). To investigate this, the authors performed Principal Component Analysis (PCA) on standard convolution kernels trained for pansharpening tasks.
As shown in Figure 2, the first two principal components capture the vast majority of the representation power, while subsequent components contribute significantly less. Based on this "sweet spot," the authors set $N=2$ for LightNet. This choice provides a balance between representation capacity and extreme efficiency, outperforming single-basis methods like Blueprint Separable Convolution (BSConv).
Experimental Results and Generalization
LightNet was evaluated on WorldView-3, QuickBird, and WorldView-2 datasets using Wald’s protocol.
Efficiency and Accuracy
LightNet utilizes only 16,000 parameters, making it roughly 100x smaller than benchmark models like DRPNN (1.8M) and BDPN (1.4M). Despite this reduction, it achieved top-tier results across standard metrics, including Spectral Angle Mapper (SAM) and relative dimensionless global error (ERGAS).
Robust Generalization
One of the most significant findings was LightNet's performance on unseen data. When trained on one satellite's data and tested on another (e.g., WorldView-2), LightNet outperformed much larger models. This suggests that the constrained kernel space of SpanConv acts as a powerful regularizer, preventing the model from overfitting to sensor-specific noise and allowing it to capture more fundamental physical features of the pansharpening task.
Conclusion
SpanConv offers a theoretically grounded approach to lightweight deep learning by viewing kernels as elements of a spanned subspace. By eliminating redundancy, LightNet achieves a remarkable balance of efficiency, accuracy, and robustness. While the optimal number of basis kernels may vary for other computer vision tasks, SpanConv serves as a highly effective "drop-in" replacement for standard convolutions in remote sensing applications.
Summary updated