A Sparse Dictionary Learning-Based Adaptive Patch Inpainting Method for Thick Clouds Removal from High-Spatial Resolution Remote Sensing Imagery

Fan Meng, Xiaomei Yang, Chenghu Zhou, Zhi Li2017 · DOI

TL;DR

This paper introduces a novel adaptive patch inpainting method using sparse dictionary learning to effectively remove thick clouds from high-spatial resolution remote sensing imagery. By learning a dictionary from cloud-free regions and adaptively reconstructing missing patches, the method preserves structures and textures, outperforming existing techniques in visual quality and statistical metrics.

What Problem Does This Paper Address?

Remote sensing (RS) imagery is crucial for various applications like land-use classification and environmental monitoring. However, cloud cover frequently obstructs these images, limiting data availability and hindering accurate analysis. Thick clouds, in particular, contaminate entire spectral bands, making their removal a significant challenge. Existing cloud removal methods often struggle with preserving fine details, leading to smoothing or blocky artifacts, especially in high-resolution imagery with complex structures and textures. This paper aims to address this gap by developing an advanced image inpainting technique specifically tailored for thick cloud removal in high-resolution RS data.

Methods

The core of this work is an adaptive patch inpainting method built upon sparse dictionary learning. The process involves:

  • Patch Selection: Patches are prioritized for inpainting based on their "structure sparsity" and "confidence." Structure sparsity helps identify patches located on image structures (edges, contours), while confidence measures the reliability of the pixel information. This ensures that structural elements are reconstructed first.
  • Sparse Dictionary Learning: A dictionary of "atoms" (representing basic image features) is learned from cloud-free regions of the image using the K-SVD algorithm. This dictionary is then used to represent missing patches as sparse linear combinations of these atoms. The dictionary is adaptively updated throughout the inpainting process to better capture the image's evolving characteristics.
  • Adaptive Patch Inpainting Model: A novel optimization model is formulated to reconstruct missing patches. This model incorporates a neighborhood-consistency constraint, ensuring that the inpainted patch aligns with its surrounding context. Crucially, an adaptive balance factor is introduced to weigh the influence of different constraints based on the patch's structure sparsity, allowing for more tailored reconstruction.
  • Modified OMP Algorithm: A modified Orthogonal Matching Pursuit (OMP) algorithm is proposed to efficiently solve the sparse representation problem within the optimization model. This algorithm is designed for practicability and simplicity.

The overall scheme iteratively applies patch selection and inpainting until the entire cloudy region is reconstructed.

Datasets

The paper evaluates its method on:

  • Barbara Image: A standard dataset in image processing, used for initial validation of the inpainting algorithm's ability to handle textures and structures.
  • Aerial Images: Red-band aerial images with simulated rectangular masks representing clouds.
  • SPOT5 Panchromatic Imagery: High-resolution imagery with simulated clouds.
  • SPOT5 Multispectral Imagery: Urban area imagery with clouds.
  • GaoFen-2 Image: Farmland imagery with clouds.

The authors do not introduce new benchmark datasets but rather use existing imagery and simulate cloud contamination to test their method against various real-world scenarios.

Key Insights and Ideas

  • Structure-Sparsity for Prioritization: Recognizing that structures are harder to reconstruct than textures, the method prioritizes inpainting patches on structural elements. This is a key insight for preserving the integrity of high-resolution imagery.
  • Adaptive Dictionary and Constraints: The adaptive nature of the dictionary learning and the balance factor in the inpainting model are crucial. This allows the algorithm to adjust its reconstruction strategy based on local image characteristics, leading to more contextually appropriate results.
  • Balancing Structure and Texture: The method effectively balances the need to preserve sharp structures with the requirement to synthesize realistic textures, a common challenge in image inpainting.
  • Suitability for High-Resolution RS: The paper highlights that the method is particularly well-suited for high-spatial resolution RS imagery due to its ability to handle salient structures and abundant textured features.

Key Results

  • Visual Quality: The proposed method consistently produced visually superior results compared to existing methods (MRF, Texture Synthesis, Exemplar-based, MCA, and patch-sparsity based approaches) in terms of preserving structures, maintaining texture consistency, and avoiding smoothing or block effects.
  • Statistical Performance: Quantitative evaluations using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) indices showed that the proposed method achieved higher scores, indicating better restoration quality. For instance, on the Barbara image, the method achieved a PSNR of 24.408 dB and an SSIM of 0.966, outperforming other methods.
  • Effectiveness on Real and Simulated Clouds: The method demonstrated strong performance on both simulated cloud masks and real cloud-contaminated RS images across different sensor types and geographical areas.

Limitations and Future Work

  • Patch Size Adaptivity: The authors acknowledge that a fixed patch size (8x8 pixels) can lead to artifacts in regions with varying scales and feature densities (e.g., a green artifact in farmlands when the patch size is too small for the context). They suggest that multiscale adaptive patch inpainting schemes are needed.
  • Scale Effects: The paper notes that image features repeat across scales, and different features benefit from different scales for optimal representation. Future work will investigate multi-scale dictionary learning for RS spatiotemporal data to develop a more robust inpainting method.
  • Computational Cost: While the modified OMP is practical, complex inpainting tasks on large datasets can still be computationally intensive. Further optimization or parallelization might be beneficial.

Summary updated

BibTeX
@misc{meng2017sparse,
  author = {Fan Meng and Xiaomei Yang and Chenghu Zhou and Zhi Li},
  title = {{A Sparse Dictionary Learning-Based Adaptive Patch Inpainting Method for Thick Clouds Removal from High-Spatial Resolution Remote Sensing Imagery}},
  year = {2017},
  doi = {10.3390/s17092130},
}

Loading related content...

Figure