Image Fusion and Reconstruction of Compressed Data: A Joint Approach

Daniele Picone, Laurent Condat, Florian Cotte, Mauro Dalla Mura

TL;DR

This paper proposes a novel approach to fuse high-resolution panchromatic (PAN) and lower-resolution multispectral (MS) satellite imagery by integrating the compression step directly into the fusion process. This "joint approach" aims to reduce data downlink requirements by compressing fused imagery, which is then reconstructed on the ground, offering a practical solution for resource-constrained satellite missions.

What Problem Does This Paper Address?

Satellite remote sensing missions often acquire both high-spatial-resolution panchromatic (PAN) images and lower-spatial-resolution multispectral (MS) images. The goal of "pansharpening" is to combine these to create a synthetic image with both high spatial and spectral resolution, which is not achievable with a single sensor due to physical limitations. However, transmitting these high-resolution images requires significant bandwidth and storage. This paper addresses the challenge of reducing this data burden by proposing a method to compress the data after fusion, rather than compressing the original PAN and MS images separately. This allows for a more efficient data downlink, with the reconstruction and fusion happening on the ground, potentially enabling the use of smaller, less expensive satellites.

Methods

The core of the paper is a joint model for image fusion and reconstruction from compressed data. Instead of decompressing the original PAN and MS images and then fusing them, this approach directly estimates the final fused image from a compressed representation. The authors propose a linear compression transformation that can be implemented optically, leveraging concepts from Color Filter Arrays (CFAs) and coded aperture snapshot spectral imaging (CASSI).

Key methodological aspects include:

  • Joint Reconstruction and Fusion: The inversion problem is formulated to directly estimate the fused image, bypassing intermediate decompression steps.
  • Optical Compression Schemes: The paper explores two optical implementations for on-board compression:
  • CASSI (Coded Aperture Snapshot Spectral Imaging): A system that compresses MS data using masks and down-sampling.
  • CFA (Color Filter Array): Inspired by how digital cameras capture color, this method uses a mosaic of filters to acquire spectral information, which is then compressed. The authors propose a specific CFA mask pattern for 4-band MS data.
  • Variational Reconstruction: The reconstruction is framed as a minimization problem, balancing data fidelity (how well the reconstructed image matches the compressed data) and regularization (enforcing desirable properties on the image).
  • Regularization Techniques: Two common regularization approaches are explored:
  • LASSO (Least Absolute Shrinkage and Selection Operator): Exploits sparsity in a transformed representation of the image.
  • Total Variation (TV): Encourages piecewise constant image regions, effectively smoothing out noise while preserving edges. The paper highlights the effectiveness of isotropic TV regularization.

Datasets

The paper evaluates its proposed method using two datasets:

  • Hobart Dataset: Acquired by the GeoEye-1 satellite, this dataset represents a moderately urban area. A simulated PAN image was generated from the full-scale MS data.
  • Beijing Dataset: Acquired by the WorldView-3 platform, this dataset features the Bird's Nest stadium area. The original PAN image was included.

Both datasets consist of a PAN image (2048x2048 pixels) and a 4-band MS image with a spatial scale ratio of 1:4. The evaluation is performed at a reduced resolution with a scale ratio of r=2.

Key Insights and Ideas

  • Compression as part of fusion: Integrating compression into the fusion pipeline, rather than treating them as separate steps, can lead to more efficient data handling and potentially better reconstruction quality.
  • Optical implementation feasibility: The proposed CFA-inspired and CASSI-based compression schemes are designed with optical implementation in mind, making them suitable for on-board processing in satellites.
  • Total Variation regularization excels: The study found that Total Variation (TV) regularization, when used with the proposed joint fusion and reconstruction framework, yields superior spectral and spatial quality compared to LASSO, especially when compared to existing CASSI-based methods.
  • Reduced data downlink: The joint approach significantly reduces the amount of data that needs to be transmitted from the satellite, enabling the use of smaller, more cost-effective satellites.

Key Results

  • Competitive performance: The proposed CFA-inspired compression with TV regularization achieved results comparable to classical pansharpening methods without compression, demonstrating the viability of the joint approach.
  • Improved quality: The CFA+TV method showed the best performance in terms of both spectral and spatial quality among the tested compression and inversion schemes.
  • Efficiency: The method achieves a compression ratio of 50% in the tested scenario, matching that of a method that ignores PAN information, but with the benefit of incorporating it.

Limitations and Future Work

  • Preliminary tests: The paper presents preliminary tests, suggesting that further investigation is needed.
  • Future work: The authors suggest future research directions including:
  • Fusion with hyperspectral data.
  • Analysis of software satellite compression schemes for comparison.
  • Extensions and enhancements to the proposed model.
  • Exploring the potential for mass-production of low-budget satellite constellations leveraging this approach.

Summary updated

Abstract

International audience
BibTeX
@inproceedings{picone2018image,
  author = {Daniele Picone and Laurent Condat and Florian Cotte and Mauro Dalla Mura},
  title = {{Image Fusion and Reconstruction of Compressed Data: A Joint Approach}},
  booktitle = {ICIP},
  year = {2018},
  doi = {10.1109/icip.2018.8451631},
}
Figure