Abstract
This paper introduces a new benchmark for few-shot image-to-image translation using the SEVIR weather dataset, moving beyond saturated "toy" benchmarks. By applying Model-Agnostic Meta-Learning (MAML) and self-supervised contrastive pretraining (MoCoV3) with novel "natural" temporal augmentations, we demonstrate significant improvements in generating realistic, high-intensity radar imagery from sparse satellite data.
Introduction
While deep learning has revolutionized computer vision, its success often hinges on massive, human-curated datasets. In real-world applications like meteorology or medical imaging, data is frequently sparse, expensive to label, and subject to distributional shifts. Existing few-shot learning benchmarks, such as Omniglot or MiniImageNet, often rely on simple icons that fail to capture the complexity of physical phenomena. Furthermore, most few-shot research focuses on classification, leaving image-to-image translation relatively unexplored. This study tackles the challenge of few-shot image translation: learning to generate complex radar imagery (VIL) from satellite infrared (IR) and lightning data using only a handful of examples from a specific weather event.
Methodology
To address the low-data regime, we propose a multi-pronged approach combining meta-learning and self-supervision.
Meta-Learning for GANs
We adapt Model-Agnostic Meta-Learning (MAML) to a Generative Adversarial Network (GAN) framework. The objective is to find a weight initialization for a U-Net generator and a patch discriminator that can rapidly adapt to a new task—a specific storm event—with minimal gradient steps. This involves a "bilevel" optimization process where the model learns an optimal starting point for adaptation.
Self-Supervised Pretraining and Natural Augmentations
We utilize MoCoV3, a contrastive learning method, to pretrain the U-Net's encoder. Recognizing that standard augmentations like color jitter are inappropriate for physical sensor data, we exploit the spatio-temporal structure of weather data. We use adjacent frames in a time series as "natural" augmentations, teaching the model representation invariance over short time intervals by anchoring one frame as a query and a neighboring frame as a positive key.
Experimental Setup
The study utilizes the Storm Event Imagery (SEVIR) dataset, which contains over 10,000 weather events across five sensor modalities. The task is to translate IR and Lightning data into Vertically Integrated Liquid (VIL) imagery. Each task corresponds to a single storm event, where the first 10 frames serve as the support set for training and the subsequent 10 frames serve as the query set for evaluation.
Results and Discussion
Our evaluation revealed several key insights into the performance of meta-learning in generative tasks.
Quantitative and Qualitative Performance
MAML-based models consistently outperformed standard joint training in minimizing Mean Absolute Error (MAE). Qualitatively, MAML generators were significantly better at identifying and "filling in" sparse, high-intensity VIL regions that joint training models often ignored or blurred. Contrastive pretraining with "Level 4" augmentations (combining temporal shifts with spatial transforms) showed consistent improvements across meteorological metrics, particularly the Critical Success Index (CSI) and Probability of Detection (POD) at high thresholds.
The Precision-Recall Trade-off
An interesting observation was the trade-off between optimization objectives and domain-specific metrics. While MAML achieved lower reconstruction error, it sometimes resulted in a lower Probability of Detection (recall) compared to joint training. Meta-learned models tended to be more "cautious," accurately identifying storm centers (high precision) but potentially missing peripheral, lighter precipitation.
Conclusion
This work establishes a foundation for few-shot generative modeling on complex, non-RGB data. While MAML shows great promise for capturing high-intensity features, training stability remains a challenge due to the interplay between adversarial loss and bilevel optimization. Future work will explore techniques like spectral normalization to improve stability and investigate better evaluation metrics that correlate more strongly with visual realism in meteorological contexts.
Summary updated