Reconstructing Cloud-Obscured Satellite Imagery via Multi-Modal Fusion

116 concepts102 papers1 proposal

Optical satellite imagery provides a vital window into Earth's changing surface, yet this window is frequently shuttered by atmospheric interference. For applications ranging from crop yield estimation to disaster response, persistent cloud cover and haze represent more than a visual nuisance; they constitute a fundamental data gap. As Zhuangfang Yi et al. note, environmental obstructions like clouds can render target structures invisible to computer vision models regardless of their architectural sophistication.

To overcome this, researchers are moving beyond simple cloud masking toward sophisticated reconstruction techniques that leverage multi-modal data fusion and generative modeling to "see through" the obscuration.

The Detection Challenge

Before clouds can be removed, they must be precisely localized. Traditional spectral thresholding often struggles with the high variability of cloud types and the similarity between clouds and bright ground features like snow or urban structures.

Recent approaches leverage deep learning to capture spatial context. Hong Zheng et al. introduced SFRS-Net, which utilizes "folding" and "unfolding" mechanisms to preserve edge details, significantly improving detection accuracy over traditional pooling methods. To address the scarcity of labeled ground truth, Aleksis Pirinen et al. demonstrated that training models on synthetic datasets of Cloud Optical Thickness (COT) allows for the generation of versatile masks that outperform existing methods on real-world data.

Fusing Optical and Radar Data

The most robust solution to persistent cloud cover lies in Synthetic Aperture Radar (SAR). Unlike optical sensors, SAR uses microwave frequencies that penetrate clouds and smoke, providing a structural map of the surface regardless of weather. The challenge, however, is that SAR and optical data are fundamentally different: one measures electromagnetic backscatter (texture and geometry), while the other measures reflected sunlight (spectral color).

To bridge this gap, researchers use fusion frameworks to translate SAR information into the optical domain. Jianhao Gao et al. proposed a "Simulation-Fusion GAN" that first simulates an optical-like image from SAR data and then merges it with the cloudy optical image. This ensures the final output maintains both the spectral fidelity of the optical sensor and the structural reliability of the radar. Similarly, Shanjing Chen et al. employed a Graph-Based Feature Aggregation Network (G-FAN) to capture non-local spectral and electromagnetic features, allowing the model to better understand the relationship between obscured pixels and their clear surroundings.

Semantic Preservation and Downstream Utility

A recurring theme in recent work is that cloud removal should not just aim for visual realism, but for "interpretability." If a reconstructed image looks clear but loses the semantic signatures of buildings or crops, it is useless for analysis.

Ziyao Wang et al. addressed this with the GACR framework, which anchors the reconstruction process to the original cloudy observation and aligns it with vision foundation models. This ensures that the generated pixels preserve the semantic information necessary for tasks like building extraction or land-cover mapping. This is particularly critical in specialized domains like glaciology; Yuran Cui et al. developed a multi-task GAN that leverages topographic and radar data to delineate glacier boundaries even under thick cloud cover, preventing the data gaps that lead to inaccurate climate inventories.

Temporal and Operational Mitigation

When multi-modal fusion is unavailable, researchers turn to the temporal dimension. By analyzing high-revisit time series, models can identify stable ground textures and distinguish them from transient atmospheric noise. Tristan Dagobert et al. developed an unsupervised algorithm that tracks ground stability over time to detect clouds and shadows, achieving high success rates even with spectrally limited sensors.

Finally, some researchers argue for avoiding clouds entirely through smarter mission planning. Zhiliang Li et al. proposed a GEO-LEO satellite cooperation scheme where geostationary meteorological satellites provide real-time cloud information to low-Earth orbit imaging satellites. This allows the imaging satellites to autonomously adjust their schedules to target clear openings, maximizing the efficiency of expensive orbital resources.

Through these combined efforts—better detection, SAR-optical fusion, and semantic-aware reconstruction—the remote sensing community is transforming cloudy imagery from a discarded liability into a reconstructible asset.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_cloud_obscuration_in_remote_sensing_a853aeac,
  title = {cloud obscuration in remote sensing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cloud-obscuration-in-remote-sensing/rdr_a853aeacf3fa47c2916bb8d288dba9ca},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_cloud_obscuration_in_remote_sensing_a853aeac_v1,
  title = {cloud obscuration in remote sensing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cloud-obscuration-in-remote-sensing/rdr_a853aeacf3fa47c2916bb8d288dba9ca/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake