Pansharpening Via Neighbor Embedding of Spatial Details

Junmin Liu, Changsheng Zhou, Rongrong Fei, Chunxia Zhang, Jiangshe ZhangIEEE JSTARS 2021 · DOI

Abstract

This paper presents DINE and DINE+, two efficient pansharpening algorithms designed to overcome the computational and structural limitations of Sparse Representation (SR) based methods. By substituting iterative optimization with Neighbor Embedding (NE) and incorporating edge-preserving filters, the proposed framework achieves high-quality image fusion with significantly reduced processing time. Experimental results across multiple satellite datasets demonstrate superior performance in both quantitative metrics and visual fidelity.

Introduction

Pansharpening is a critical task in satellite remote sensing, aiming to fuse low spatial resolution multispectral (LSR MS) images with high spatial resolution panchromatic (PAN) images. Traditional Sparse Representation (SR) methods, while effective, are often computationally expensive due to iterative $\ell_1$-minimization and can lead to edge blurring. This paper introduces a Neighbor Embedding (NE) approach that leverages closed-form solutions and manifold learning to preserve sharp boundaries and fine textures more efficiently.

Methods

The proposed framework is built upon the Spatial Detail Injection (SDI) model. The core innovation lies in the reconstruction of missing high-frequency spatial details using Neighbor Embedding rather than global sparsity.

DINE: Detail Injection via Neighbor Embedding

DINE assumes that low-resolution patches and their high-resolution counterparts share a similar local geometry. For each patch, the algorithm identifies $K$-nearest neighbors in a dictionary and solves a constrained least squares problem. This approach yields a closed-form solution, drastically reducing computational overhead compared to iterative SR methods.

The framework of the proposed DINE/DINE+ method for pansharpening.

DINE+: Enhanced Edge Preservation

To further improve edge preservation, DINE+ integrates an edge-preserving filter. This filter calculates spatially varying injection gains, ensuring that spatial details are transferred to MS bands while respecting structural boundaries. This mechanism acts as a "structural gatekeeper," preventing artifacts like ringing or aliasing.

Experimental Setup

The performance of DINE and DINE+ was evaluated using 48 image pairs from three major satellite sensors: QuickBird (QB), WorldView-2 (WV2), and GeoEye-1 (GE1). These sensors provide varying spectral and spatial resolutions, as detailed in the technical specifications below.

Specifications of the satellite sensors used in the evaluation.

The evaluation covered diverse environments, ranging from dense urban centers to rural landscapes, ensuring a robust assessment of the algorithms' versatility.

Results and Discussion

DINE and DINE+ demonstrate a significant advantage in processing speed. For instance, on the WV2 dataset, DINE achieved a runtime of approximately 219 seconds, outperforming SR-based methods while maintaining superior image quality.

Quantitatively, the proposed methods consistently ranked at the top for standard metrics such as ERGAS, SAM, and Q4/Q8. Qualitatively, DINE+ produced the sharpest results, particularly for small objects like vehicles and building edges, effectively eliminating common artifacts found in Multiresolution Analysis (MRA) techniques. The key insight is that local linear relationships (NE) provide a more effective and efficient prior for spatial details than global sparsity.

Conclusion

The DINE and DINE+ algorithms offer a compelling alternative to traditional pansharpening methods by prioritizing manifold learning over global sparsity. While the choice of the neighborhood parameter $K$ and search complexity remain areas for optimization, the current framework provides a scalable and high-performance solution for large-scale remote sensing applications. Future work will focus on improving robustness to sensor noise and atmospheric interference.

Summary updated

BibTeX
@article{liu2021pansharpening,
  author = {Junmin Liu and Changsheng Zhou and Rongrong Fei and Chunxia Zhang and Jiangshe Zhang},
  title = {{Pansharpening Via Neighbor Embedding of Spatial Details}},
  journal = {IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens.},
  year = {2021},
  doi = {10.1109/jstars.2021.3067877},
}
Figure