Domain Adaptation on Multiple Cloud Recognition From Different Types of Meteorological Satellite

Bin Huang, Luming Xiao, Feng Wen, Mengqiu Xu, Ming Wu, Xiang FangFrontiers in Earth Science 2022 · DOI

Abstract: Meteorological satellites have become an indispensable meteorological tool for earth observation, as aiding in areas such as cloud detection, which has important guiding significance for maritime activities. However, it is time-consuming and labor-intensive to obtain fine-grained annotations provided by artificial experience or mature satellite cloud products for multi-spectral maritime cloud imageries, especially when new satellites are launched. Moreover, due to the data discrepancy caused by different detection bands, existing models have inadequate generalization performance compared to new satellites, and some cannot be directly migrated. In this paper, to reduce the data distribution’s discrepancy, an approach is presented based on unsupervised domain adaption method for marine cloud detection task based on Himawari-8 satellite data as a source domain and Fengyun-4 satellite data as a target domain. The goal of the proposed method is to leverage the representation power of adversarial learning to extract domain-invariant features, consisting of a segmentation model, a feature extract model for target domain, and a domain discriminator. In addition, aiming to remedy the discrepancy of detection bands, a band mapping module is designed to implement consistency between different bands. The result of the experiments demonstrated the effectiveness of the proposed method with a 7% improvement compared with the comparative experiment. We also designed a series of statistical experiments on different satellite data to further study cloudy perception representation, including data visualization experiment and cloud type statistics.

1. Introduction

Accurate marine cloud classification is essential for weather forecasting and maritime safety, particularly in detecting hazards like sea fog. While deep learning has significantly advanced cloud recognition, these models typically require extensive pixel-level annotations. When new meteorological satellites are launched, generating high-quality labeled datasets can take years. Furthermore, models trained on one satellite (the source domain) often fail to generalize to another (the target domain) due to "domain shift" caused by differences in spectral bands, spatial resolution, and sensor characteristics. This study proposes an unsupervised domain adaptation (UDA) framework to transfer knowledge from a well-labeled satellite to a new, unlabeled one.

2. Methodology

The proposed UDA architecture utilizes adversarial learning to bridge the gap between domains. It consists of three primary components:

  • Segmentation Model: A VecNet-based network trained on labeled source data for pixel-level classification.
  • Feature Extractor: A module designed to extract features from the target domain.
  • Domain Discriminator: A PatchGAN-based component that attempts to distinguish between source and target feature maps.

Through adversarial training, the feature extractor learns to produce "domain-invariant" features that fool the discriminator, focusing on the underlying physics of cloud formations rather than satellite-specific noise. To address spectral discrepancies—specifically the 16-channel input of Himawari-8 versus the 14-channel input of Fengyun-4A—a dedicated Band Mapping Module was implemented to align spectral inputs into a consistent internal representation.

3. Data and Experiments

The study focused on the Yellow Sea and Bohai Sea regions. The source domain utilized data from the Himawari-8 (H-8) satellite, which features 16 detection bands and a mature labeled dataset. The target domain consisted of unlabeled data from the Fengyun-4A (FY-4A) satellite, which has 14 detection bands.

Overview of satellite data and spectral bands: (A) True color image and cloud labels, (B) Himawari-8 bands H01-H16, (C) Fengyun-4A bands F01-F14.

Images were cropped to 512 × 512 pixels and temporally aligned to ensure consistent atmospheric conditions across both datasets during the training of the UDA framework.

4. Results and Discussion

The proposed UDA method achieved an overall accuracy of 55.8% on the unlabeled FY-4A data, representing a 7% improvement over the comparative Med-CMDA method (48.7%). While a fully supervised model trained on FY-4A labels reaches 72.9%, achieving 55.8% without any target labels marks a significant advancement for rapid satellite deployment.

Key insights include:

  • Spectral Alignment: The band mapping encoder was crucial for reconciling the different number of spectral channels.
  • Feature Consistency: t-SNE visualizations confirmed that the UDA process successfully aligned the feature distributions of both satellites, which were initially distinct.
  • Semantic Success: The model effectively identified clear skies and large-scale stratus clouds, though it encountered more difficulty with fragmented cloud types like cumulus.

5. Conclusion

This research demonstrates that unsupervised domain adaptation can effectively mitigate the challenges of cross-satellite cloud recognition. Future work will focus on improving the detection of fine-grained cloud edges and complex multi-layered structures, as well as addressing data imbalances for less frequent cloud types like alto-cumulus. The ultimate goal is a satellite-agnostic system capable of immediate deployment across diverse meteorological platforms.

Summary updated

Abstract

Meteorological satellites have become an indispensable meteorological tool for earth observation, as aiding in areas such as cloud detection, which has important guiding significance for maritime activities. However, it is time-consuming and labor-intensive to obtain fine-grained annotations provided by artificial experience or mature satellite cloud products for multi-spectral maritime cloud imageries, especially when new satellites are launched. Moreover, due to the data discrepancy caused by different detection bands, existing models have inadequate generalization performance compared to new satellites, and some cannot be directly migrated. In this paper, to reduce the data distribution’s discrepancy, an approach is presented based on unsupervised domain adaption method for marine cloud detection task based on Himawari-8 satellite data as a source domain and Fengyun-4 satellite data as a target domain. The goal of the proposed method is to leverage the representation power of adversarial learning to extract domain-invariant features, consisting of a segmentation model, a feature extract model for target domain, and a domain discriminator. In addition, aiming to remedy the discrepancy of detection bands, a band mapping module is designed to implement consistency between different bands. The result of the experiments demonstrated the effectiveness of the proposed method with a 7% improvement compared with the comparative experiment. We also designed a series of statistical experiments on different satellite data to further study cloudy perception representation, including data visualization experiment and cloud type statistics.
BibTeX
@inproceedings{huang2022domain,
  author = {Bin Huang and Luming Xiao and Feng Wen and Mengqiu Xu and Ming Wu and Xiang Fang},
  title = {{Domain Adaptation on Multiple Cloud Recognition From Different Types of Meteorological Satellite}},
  booktitle = {Frontiers in Earth Science},
  year = {2022},
  doi = {10.3389/feart.2022.947032},
}
Figure