Introduction
Earth System Models (ESMs) are the gold standard for predicting climate evolution under various emission scenarios. However, their extreme computational cost—often requiring weeks on supercomputers—limits the scope of uncertainty quantification and policy assessment. Traditional machine learning emulators have attempted to replace ESMs using weather-scale autoregression, but these models suffer from significant statistical drift over long horizons and are computationally inefficient for generating coarse-grained data like annual averages.
Spatiotemporal Pyramid Flows (SPF)
To address these challenges, we introduce Spatiotemporal Pyramid Flows (SPF). SPF is a generative model that bypasses autoregression by producing climate data hierarchically across multiple spatial and temporal resolutions. Unlike previous models that generate frames sequentially, SPF generates data in parallel across a spatiotemporal pyramid.
Key Innovations
- Spatiotemporal Cascading: SPF generates coarse-resolution, long-horizon climate states first and progressively refines them into higher temporal (e.g., monthly) and spatial resolutions. This hierarchical structure matches the physical reality where large-scale forced trends organize small-scale weather fluctuations.
- Temporal Funneling: The model can focus on specific time windows of interest, preventing GPU memory exhaustion during fine-resolution generation of long sequences.
- Multi-Timescale Sampling: The flow matching objective is modified to allow direct sampling at any level of the pyramid. Users can stop the process early if only coarse data (like decadal trends) is required, saving significant computation.
- Pixel-Space Operation: Unlike latent diffusion models that rely on pre-trained VAEs, SPF operates directly in pixel space, avoiding potential information loss from compression.
- Forcing Conditioning: Each stage is conditioned on physical forcings (like greenhouse gases and aerosols) to ensure the generated trajectories capture slow, externally driven climate trends.
ClimateSuite: A Large-Scale Dataset
A significant contribution of this work is the release of ClimateSuite, the largest ML-ready climate dataset to date. It encompasses over 33,000 simulation-years from 10 different ESMs. Notably, it is the first dataset to include simulations of climate interventions—specifically, 39 Stratospheric Aerosol Injection (SAI) experiments—enabling the training of robust "superemulators" that generalize across different models.
Results and Performance
SPF demonstrates state-of-the-art accuracy on the ClimateBench benchmark, achieving the lowest Continuous Ranked Probability Score (CRPS) for probabilistic skill compared to strong baselines like ClimaX and UNets.
The efficiency gains are substantial:
- Speed: SPF provides up to a 28x speedup over autoregressive flow baselines.
- Throughput: A 100-year monthly probabilistic sample can be generated in just 1 to 2 minutes on a single GPU.
- Generalization: A 600M parameter version of SPF pre-trained on ClimateSuite shows zero-shot generalization to held-out climate models and intervention scenarios, such as SAI experiments in the UKESM1-0-LL model for which it had seen no prior intervention data.
Discussion and Future Work
By structuring the generative process hierarchically, SPF provides a physics-inspired inductive bias that avoids the compounding errors of step-by-step emulation. However, current iterations do not explicitly enforce physical laws like mass or energy conservation. Future research will focus on integrating these hard constraints into the flow matching framework to ensure physical consistency in all generated trajectories, especially under extreme or out-of-distribution forcing conditions.
Summary updated