Joint Sparse Representations and Coupled Dictionary Learning in Multi-Source Heterogeneous Image Pseudo-color Fusion

Long Bai, Shilong Yao, Kun Gao, Yanjun Huang, Ruijie Tang, Hong Yan, Max Q.‐h. Meng, Hongliang Ren

Abstract

Considering that Coupled Dictionary Learning (CDL) method can obtain a reasonable linear mathematical relationship between resource images, we propose a novel CDL-based Synthetic Aperture Radar (SAR) and multispectral pseudo-color fusion method. Firstly, the traditional Brovey transform is employed as a pre-processing method on the paired SAR and multispectral images. Then, CDL is used to capture the correlation between the pre-processed image pairs based on the dictionaries generated from the source images via enforced joint sparse coding. Afterward, the joint sparse representation in the pair of dictionaries is utilized to construct an image mask via calculating the reconstruction errors, and therefore generate the final fusion image. The experimental verification results of the SAR images from the Sentinel-1 satellite and the multispectral images from the Landsat-8 satellite show that the proposed method can achieve superior visual effects, and excellent quantitative performance in terms of spectral distortion, correlation coefficient, MSE, NIQE, BRISQUE, and PIQE.

Introduction

Remote sensing often utilizes two distinct data modalities: Synthetic Aperture Radar (SAR) and multispectral imaging. SAR provides structural details regardless of lighting or weather conditions but is limited by monochrome output and inherent "speckle noise." Multispectral imaging offers rich spectral information but lacks the spatial consistency of SAR in adverse conditions. Fusing these heterogeneous sources into a high-resolution pseudo-color image remains a significant challenge, particularly regarding spectral distortion. While deep learning models show promise, their high computational demands often preclude deployment on resource-constrained satellite hardware. This paper presents a non-deep-learning alternative using Coupled Dictionary Learning (CDL) to achieve high-fidelity image fusion.

Methodology

The proposed framework consists of three primary stages designed to maximize multimodal correlation while preserving structural integrity:

  1. Pre-processing via Brovey Transform: To bridge the gap between SAR and multispectral data, a Brovey transform is applied to create a "pseudo-SAR" image. This step introduces initial spectral information into the SAR data, facilitating mathematical correlation during the subsequent dictionary learning phase.
  2. Coupled Dictionary Learning (CDL): The core of the method involves learning two coupled dictionaries—one for multispectral data ($D_{MS}$) and one for the pseudo-SAR ($D_B$). By enforcing a shared sparse representation ($\alpha$), the system ensures that corresponding patches from both sources are represented by the same coefficients, capturing the underlying multimodal correlation.
  3. Error-Based Reconstruction: The final fusion utilizes reconstruction errors to generate a pixel-level mask. This mask intelligently blends source information to maximize structural detail while suppressing SAR speckle noise.
The proposed fusion framework, detailing the input data, Brovey pre-processing, and coupled dictionary generation stages.

The method was validated using Sentinel-1 SAR data and Landsat-8 multispectral data from the Liangshan region in Shandong Province, China. Images were registered using standard map coordinates in ENVI 5.5.3 to ensure pixel-perfect alignment before fusion.

Results and Discussion

The CDL-based approach was benchmarked against eight traditional fusion techniques, including PCA, Wavelet, and Gram-Schmidt (GS) methods. Quantitatively, the proposed method achieved the lowest spectral distortion (24.35) and Mean Square Error (106.03). It also excelled in no-reference metrics such as NIQE and BRISQUE, indicating superior visual naturalness.

Qualitatively, the method effectively suppressed SAR speckle noise, producing smooth, high-resolution pseudo-color images without the "washed out" effects or color distortions common in other techniques. By removing restriction items during dictionary updates, the model better captured the relationship between structural details and spectral information.

The resulting fused pseudo-color image, demonstrating high spectral fidelity and noise suppression.

Conclusion

This research demonstrates that a CDL-based framework can provide high-performance heterogeneous image fusion suitable for edge deployment. By leveraging the Brovey transform as a bridge and enforcing joint sparse representations, the method achieves a balance between structural detail and spectral accuracy. Future work will focus on generalizing the method to other fusion tasks, such as medical imaging (PET/MRI), and evaluating its impact on downstream automated tasks like land-use classification and object segmentation.

Summary updated

Abstract

Considering that Coupled Dictionary Learning (CDL) method can obtain a reasonable linear mathematical relationship between resource images, we propose a novel CDL-based Synthetic Aperture Radar (SAR) and multispectral pseudo-color fusion method. Firstly, the traditional Brovey transform is employed as a pre-processing method on the paired SAR and multispectral images. Then, CDL is used to capture the correlation between the pre-processed image pairs based on the dictionaries generated from the source images via enforced joint sparse coding. Afterward, the joint sparse representation in the pair of dictionaries is utilized to construct an image mask via calculating the reconstruction errors, and therefore generate the final fusion image. The experimental verification results of the SAR images from the Sentinel-1 satellite and the multispectral images from the Landsat-8 satellite show that the proposed method can achieve superior visual effects, and excellent quantitative performance in terms of spectral distortion, correlation coefficient, MSE, NIQE, BRISQUE, and PIQE.
BibTeX
@article{bai2023joint,
  author = {Long Bai and Shilong Yao and Kun Gao and Yanjun Huang and Ruijie Tang and Hong Yan and Max Q.‐H. Meng and Hongliang Ren},
  title = {{Joint Sparse Representations and Coupled Dictionary Learning in Multi-Source Heterogeneous Image Pseudo-color Fusion}},
  journal = {IEEE Sensors Journal},
  year = {2023},
  doi = {10.1109/jsen.2023.3325364},
}
Figure