Climate models cannot resolve clouds and convection directly. Each grid cell is 50–100 km wide, so a learned or hand-written sub-grid parameterization has to supply the missing heating, moistening and rain as a function of the coarse state. A coarse state does not fix those values. It fixes only a distribution over them, which is why stochastic closures, which sample tendencies instead of predicting their average, keep outperforming deterministic ones online. The variables that matter most have awkward distributions, though. Precipitation is exactly zero in most columns. Relative humidity piles up against a hard ceiling at saturation. Cloud fraction lives in $[0,1]$. Probability mass sits on these edges as spikes, called atoms, next to a continuous spread of values in the interior.
The theory that justifies flexible generative estimators assumes away exactly this structure. Diffusion models are generators that learn by gradually denoising random noise into samples. They are provably minimax-optimal conditional density estimators, but only when the target density is Hölder-smooth up to the boundary of its support (Conditional Diffusion Models are Minimax-Optimal and Manifold-Adaptive; Diffusion Models are Minimax Optimal Distribution Estimators). Transport maps lose regularity when the target has sharp features or disjoint support (An Approximation Theory Framework for Measure-Transport Sampling Algorithms).
We propose to split each sub-grid output into a boundary atom and a smooth interior before learning it. A classifier predicts the probability of sitting on the physical boundary, such as no rain or saturation. A nonparametric conditional density estimator, run in coordinates where the boundary becomes harmless, models everything else. The decomposition makes the smoothness assumptions of the theory true by construction instead of violated by the data. We also test the estimator under buffered chronological splits. Flexible density estimators are the models best placed to memorize near-duplicate columns sampled 20 minutes apart, so random splits may flatter them most.
Core Claim
Write the conditional law of a sub-grid output $y$ given the coarse column state $x$ as
$$P(dy \mid x) = \pi(x)\,\delta_b(dy) + \big(1-\pi(x)\big)\,q(y \mid x)\,dy,$$
where $b$ is the physical boundary value (zero precipitation, or the saturation limit), $\delta_b$ is a point mass there, $\pi(x)$ is the state-dependent probability of being on the boundary, and $q$ is a density on the interior.
After a boundary-flattening change of variables, the interior density $q$ is plausibly smooth up to its edge. A logarithm of rain rate is one such change; a reflection across the saturation limit is another. In that case the cited minimax guarantee applies to $q$ and an ordinary classification rate applies to $\pi$. A diffusion model fit to raw $y$ has no such guarantee, because it must approximate an atom with a smooth density, which forces either a near-singular score near $b$ or leakage of mass across the boundary.
We claim three things. First, offline, the decomposed estimator matches raw generative baselines in the interior and beats them clearly near boundaries: on the probability of rain, on light-rain calibration, and on the rate of supersaturated samples. Second, online, the decomposed closure reduces the unphysical states that crash hybrid models. Third, the ranking among model families changes between random and buffered chronological splits.
Relation to Prior Work
Three lines of work make sub-grid closures stochastic or distributional.
Generative data-driven approaches for stochastic subgrid parameterizations trains a Wasserstein GAN with a gradient penalty and a conditional VAE for ocean eddy forcing in a two-layer quasi-geostrophic model. Its forcing is a signed, unbounded field with no atoms. Our targets are dominated by atoms, so that paper motivates generative closures without testing the regime we address.
The ClimSim benchmark includes conditional-VAE and random-perturbation baselines and reports that convective precipitation is markedly harder to predict than heating. That finding is the gap this work targets. Our decomposition is aimed specifically at the precipitation and moisture outputs.
Embedding machine-learnt sub-grid variability predicts only a sub-grid standard deviation with a multi-output Gaussian process and perturbs SPEEDY with Gaussian noise. That is the closest simple competitor: a state-dependent spread with a fixed Gaussian shape and no boundary handling.
Two papers constrain the design. CondensNet shows that a deterministic emulator crashed after about 107 simulated days from runaway energy, and that a hard mask triggered above 100% relative humidity restored decade-scale stability. The saturation boundary is therefore where online failure starts. Our approach puts that boundary into the probability model instead of patching it after prediction. Learning Closed-Form Equations for Subgrid-Scale Closures shows offline correlation above 0.95 alongside universal online instability, so offline scores alone cannot validate a closure, and our design always pairs offline with online evaluation.
On evaluation, Temporal Convolutional Networks for the Advance Prediction of ENSO holds out a 10-year buffer between training and test periods to defeat ocean persistence. The adverse effects of code duplication and Exploring Data Splitting Strategies show that non-chronological splits inflate scores through near-duplicates. We borrow the buffered protocol, but set the buffer from measured decorrelation instead of a fixed decade, since ClimSim spans only 10 simulated years.
Classical kernel density estimation enters in two roles. It serves as a nonparametric baseline for the interior density. It also carries a known hazard: Consistency Analysis of an Empirical Minimum Error Entropy Algorithm shows that fixed-bandwidth kernel objectives break down when the noise depends on the input. We therefore use state-adaptive bandwidths and do not train with kernel entropy losses.
Research Program
We would build the estimator in three parts. The first is a boundary classifier for $\pi(x)$, applied per output and per vertical level where an atom exists: surface precipitation, condensate, and the saturation edge of moistening that would push humidity above 100%. The second is a change of variables for each bounded output. The third is a conditional diffusion model for $q$ that generates the interior jointly across levels and variables, so that vertical correlation is preserved.
Sampling first draws boundary indicators and then the interior values. A conservation step adjusts the column so that column water and energy are consistent with the sampled surface precipitation. The analytical part states the conditions under which the transformed $q$ inherits Hölder smoothness, so that the cited conditional diffusion rate applies to the decomposed model. It also shows by a simple construction why an atom forces either score blow-up or boundary leakage in a raw diffusion fit.
The evaluation protocol is built first, before any model. We measure the autocorrelation decay of ClimSim input columns over time lags, define the buffer as the lag where it falls below a fixed threshold, and also sweep buffers of zero, one day, one month, and one season. The primary split trains on early years, discards a buffer, and tests on the final years. A random column shuffle is the leakage reference.
Baselines
The naive reference is a deterministic MLP trained with mean squared error, as in ClimSim, with outputs clipped to their physical ranges after prediction. It shows what any stochastic model must beat on a probabilistic score, the continuous ranked probability score (CRPS), once its point forecast is treated as a degenerate distribution. Its online behaviour is expected to resemble the CondensNet paper's unconstrained baseline, which crashed after about 107 days.
The strongest existing stochastic baselines on the target benchmark are the ClimSim conditional VAE and random-perturbation network. We will rerun these with released code on our splits. The file reports that the ClimSim baselines show "much lower $R^2$" for precipitation, but no reported result found on Lacuna for an exact precipitation $R^2$ or CRPS. Our reference numbers must therefore come from our reruns, not from the paper.
The strongest stochastic closure with a reported number is the quasi-geostrophic generative model, which reduced online distributional error 5–10x versus no parameterization and 3–5x versus a Gaussian-noise ML closure. That result is for ocean eddies, so it frames the expected size of gains rather than serving as a direct target.
For online stability the bar is CondensNet: stable 10- and 50-year runs in CAM5.2, against 107 days for the unconstrained network. We will run CondensNet's mask on top of the deterministic baseline in ClimSim-Online.
Three ablations isolate the key ingredient. The first is a raw conditional diffusion model on untransformed outputs, which isolates the decomposition. The second keeps the decomposition but uses a zero-inflated Gaussian interior, in the spirit of the Gaussian-process variability approach; this isolates the nonparametric interior. The third keeps the decomposition with a k-nearest-neighbour conditional kernel density estimator using adaptive bandwidth, which isolates the neural estimator.
All models get the same input set (the 617-variable configuration), the same training years, and an equal hyperparameter-search budget measured in GPU-hours. Every model is selected on a validation block that lies after the training years. Beating a baseline means a lower buffered-test CRPS on precipitation and moistening, with confidence intervals from blocked bootstrap over test weeks, together with longer or equal online survival and lower precipitation bias.
Why not just…?
Why not just train a larger diffusion model on the raw outputs? Capacity cannot remove an atom. A smooth density can only approximate a point mass by concentrating sharply, which is exactly where the cited approximation results fail and where samples cross into negative rain or supersaturation. The raw-diffusion ablation tests this directly. If, at equal compute, it matches the decomposed model on light-rain calibration, boundary-violation rate and online survival, the claim is wrong and we will report that.
Why not just use a deterministic network with a hard humidity mask, as CondensNet does? CondensNet shows that masking buys stability, but its output is a single value per state. Stochastic closures exist because sub-grid physics is non-unique given the coarse state, as Stochastic climate theory and modeling argues. The comparison is therefore two-sided: CondensNet may win on survival time, while the decomposed model should win on CRPS and on the variability of precipitation statistics. If CondensNet matches on both, then distributional modeling adds nothing here.
Why not just a zero-inflated Gaussian (a "hurdle" model)? This is the real threat. It is cheap, respects the atom, and the Gaussian-process variability paper already gets 17–20% precipitation error reductions from Gaussian spread alone. Whether the interior is non-Gaussian is an empirical question. We first test it with adaptive-bandwidth kernel estimates of buffered-test residuals, checking for multimodality and skew conditional on regime. If the interior turns out Gaussian enough, the hurdle model wins and the contribution reduces to the boundary decomposition plus the evaluation protocol. We would state that plainly.
Why not just keep random splits, since everyone uses them? At 20-minute sampling, neighbouring columns are near-duplicates. Flexible density estimators can memorize them, so random splits may reward exactly the models we propose. Reporting both splits makes the inflation per model class measurable rather than assumed.
Proposed Experiments
All offline work uses the ClimSim low-resolution data, with the high-resolution data used for the final runs. Online coupling uses ClimSim-Online.
Offline metrics are CRPS and an energy score for joint vertical profiles, a Brier score for the boundary indicator, a probability-integral-transform calibration score, interior Wasserstein distance, and the fraction of samples outside physical bounds. Online metrics are time to crash, global energy and water drift, the precipitation-intensity histogram, and tropical precipitation bias over multi-year runs.
The first decisive check is the leakage curve. For each model family, we plot test CRPS against buffer length. The prediction is that the decomposed and raw generative models lose more skill between zero buffer and the decorrelation buffer than the deterministic MLP does. That result, or its absence, is independently publishable as a benchmark finding.
The second check is the boundary-stratified error. For columns within a small distance of the zero-rain or saturation boundary, the decomposed model should cut boundary-violation rates to near zero and improve light-rain CRPS over raw diffusion. Away from boundaries the two models should be comparable.
The third check is online. We couple the best models into ClimSim-Online and count crashes and drift across several seeds. Success means surviving at least as long as the masked deterministic baseline while reproducing the precipitation-intensity distribution better.
If the hurdle-Gaussian ablation equals the full model on all three checks, we will drop the nonparametric interior from the claim.
References
- Conditional Diffusion Models are Minimax-Optimal and Manifold-Adaptive for Conditional Distribution Estimation
- Diffusion Models are Minimax Optimal Distribution Estimators
- An approximation theory framework for measure-transport sampling algorithms
- Generative Data‐Driven Approaches for Stochastic Subgrid Parameterizations in an Idealized Ocean Model
- ClimSim: A large multi-scale dataset for hybrid physics-ML climate emulation
- Embedding machine-learnt sub-grid variability improves climate model precipitation patterns
- CondensNet: enabling stable long-term climate simulations via hybrid deep learning models with adaptive physical constraints
- Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges
- Temporal Convolutional Networks for the Advance Prediction of ENSO
- The Adverse Effects of Code Duplication in Machine Learning Models of Code
- Exploring Data Splitting Strategies for the Evaluation of Recommendation Models
- Consistency analysis of an empirical minimum error entropy algorithm
- Stochastic climate theory and modeling