Neuroscience and weather modeling face the same hidden question. Neuroscientists cataloguing cells have found that some circuits sort into clean, discrete types, while others vary smoothly along a continuum. A deep network trained to predict neural responses recovers sharp cell types in the retina but finds only a continuous landscape in visual cortex (Learning to cluster neuronal function). Weather and climate models face a parallel choice when they represent what happens inside a grid cell too large to resolve storms. This is called subgrid parameterization: estimating the fine-scale weather hidden inside each coarse cell from the coarse state. Should a learned subgrid model sort columns into a few discrete regimes, as a mixture-of-experts would (a network that sends each input to one of several specialized sub-networks)? Or should it vary its behavior smoothly with the coarse state?
Neither field has a reliable way to answer this from learned representations. One reason stands out. The squared-error training that makes weather forecasts blurry, by pulling predictions toward the conditional mean, may also blur a model's internal representation. That would make truly discrete regimes look like a continuum. This note proposes to build and calibrate a test that separates "discrete or continuous?" from "discrete but smeared by the loss?". The test is then applied to subgrid variability in reanalysis data. The note also asks whether the test's verdict predicts which subgrid architecture actually works best.
Core Claim
We claim that whether learned subgrid regimes look discrete or continuous is partly an artifact of the training objective. In particular, mean-squared-error training should hide discreteness that a probabilistic objective, such as the continuous ranked probability score (CRPS, which rewards a predicted distribution for matching the observed outcome), preserves. Once that artifact is controlled, the measured discreteness of a region's subgrid states should predict whether hard expert routing or smooth conditioning gives the better subgrid model there.
Two lines of prior work motivate this claim. The first is the neuroscience framing. Learning to cluster neuronal function itself names the open problem: continuous embeddings might reflect real biological continua, or they might be artifacts of regression objectives. That paper's method (DECEMber) also needs the number of clusters fixed in advance. The second is the weather work. (Sparse) Attention to the Details and WeatherBench 2 show that regression losses damp high-frequency structure in forecast outputs.
The new step is to treat these as one mechanism acting on two different things: outputs in one case, representations in the other. We would test it where ground truth for fine-scale weather exists. Neither line of work has done this. DECEMber was never applied to geophysical closures. The weather papers study output spectra, not the structure of learned representations.
Research Program
The testbed is a well-posed subgrid problem built from ERA5, a global atmospheric reconstruction that blends a weather model with observations. We start from hourly ERA5 data at 0.25° resolution and average it onto the 1.5° grid used by WeatherBench 2. Each coarse cell then contains about 36 fine cells. The task is to predict the distribution of fine-scale precipitation, near-surface wind, and mid-level vertical velocity inside each coarse cell. The inputs are the coarse vertical column plus a 3×3 horizontal neighborhood.
This mirrors the column-physics module of GAT-MLP, which handles vertical physics with a small network over a 3×3 patch. There is one difference: the target here is a measured within-cell distribution, not a single tendency. We should be honest about the data. ERA5's own fine-scale precipitation is produced by a model with its own parameterization, so this is a study of subgrid structure in reanalysis, not in nature.
The work has three parts. First, we build a calibration suite in which the truth is known. We generate synthetic columns from K discrete regimes. We then add smooth within-regime variation, mirroring the "discrete families with internal gradients" structure seen in the striatum and hippocampus (Muñoz‐Manchado et al., Hochgerner et al.). Separately, we generate truly continuous controls.
Second, we train identical encoders on the real and synthetic tasks under three objectives: mean squared error, CRPS from a small sampled ensemble, and CRPS combined with DECEMber's joint mixture prior. DECEMber's mixture uses multivariate t-distributions fitted by expectation–maximization. To address the instability the file notes for such models in high dimensions, we restrict each component's covariance to a diagonal-plus-low-rank form.
Third, we measure discreteness without fixing K. The measure is the held-out log-likelihood gain of the best mixture over a range of K, relative to a single heavy-tailed component. We report it alongside a dip test of unimodality along the leading principal curve. The detector counts as calibrated if three things hold on synthetic data. It must recover the true discreteness under CRPS. It must lose that signal under mean squared error, which would confirm the confound. It must not hallucinate clusters in the continuous controls.
We then apply the calibrated detector to ERA5 separately for the tropics, the midlatitude storm tracks, and orographic regions. We have a prior expectation, not a result: tropical convective and stratiform columns may separate more cleanly than midlatitude frontal zones.
Baselines
The naive reference is to do nothing at subgrid scale. Each fine cell gets the coarse-cell value, or a bicubic interpolation of the coarse field. This is the post-hoc resizing that CSTF and Multimodal Atmospheric Super-Resolution report loses physical gradients and spectral energy. Any learned model must beat it on CRPS and on 99th-percentile precipitation error.
The first learned baseline is a deterministic column network trained with mean squared error, in the style of the GAT-MLP physics module. It is the simplest architecture that already beats such coarse models.
The strongest competitor is a dense probabilistic column network with no regime structure, trained on CRPS. This is the subgrid analogue of how MOSAIC restores spectral fidelity through stochastic perturbation. If regime structure adds nothing beyond probabilistic training, this model will match everything else.
We also test two structured models trained on CRPS. One uses hard top-2 routing among experts, as in EWMoE. The other is a smooth model whose weights are modulated continuously by the learned embedding.
The key ablation removes the DECEMber mixture prior while holding the encoder fixed. This isolates whether the clustering inductive bias does anything.
All models share the same data split (2016–2019 for training, 2020 for validation, 2021 for test), the same parameter budget within 10%, the same compute, and the same 30-trial hyperparameter search. A model "beats" the dense probabilistic baseline only if its CRPS is lower on the 2021 test year by more than the spread across three training seeds, and its extreme-quantile error does not get worse.
We found no reported result on Lacuna for this coarse-to-fine subgrid distribution task. The only numbers available are context from other tasks, such as the 3-day Z500 RMSE of about 154 m²s⁻² for operational IFS in WeatherBench. Those numbers are not comparable here. All baseline numbers will therefore be produced within this study.
Why not just…?
Why not just train a dense probabilistic model and skip the regime question? This is the real threat. It may be the best model everywhere. If so, the study's practical conclusion is that regime structure is unnecessary for subgrid closures at this resolution. That is still useful, because the field keeps adopting expert-routing designs (ARROW, EWMoE) without measuring whether regimes exist. The decisive check below asks whether the detector's verdict tracks, region by region, the gap between hard-routed and dense models. Without that link, the diagnostic claim fails.
Why not just run k-means or a Gaussian mixture on frozen embeddings? Learning to cluster neuronal function reports that post-hoc clustering of regression embeddings performs worse than its joint method, which recovers retinal types at an adjusted Rand index of 0.96. Even so, we include post-hoc clustering as a detector baseline on the synthetic suite. If it matches the joint prior there, we drop the prior.
Why not just use existing meteorological regime labels, such as convective versus stratiform? These labels are a strong, cheap alternative. They serve as an extra routing baseline: experts are assigned by label instead of by a learned router. If the labels match learned routing, the conclusion is that the discrete structure was already known. The learned detector then matters only in regions where no such labels exist.
Why not just raise the resolution so subgrid modeling is unnecessary? Estimates of roughly 20,000 H100 GPU-hours to bring MOSAIC to 0.25° (MOSAIC), and the exascale argument of Earth Virtualization Engines, make coarse ensembles with closures necessary for the foreseeable future. Scale only moves the problem: some process always remains below the grid.
Proposed Experiments
The first decisive check is the confound test on synthetic data. Discreteness scores are measured under mean squared error versus CRPS, in suites with known K and in continuous controls. The hypothesis is supported only if mean squared error substantially lowers the score in discrete suites and neither objective produces false clusters in continuous ones.
The second check repeats the comparison on ERA5. It asks whether the regional discreteness verdict flips when the objective changes.
The third check is predictive. Across the three regions and four seasons (twelve cells), the detector's score should correlate positively with the CRPS advantage of hard routing over smooth conditioning.
Downstream, the CRPS-trained subgrid models are evaluated at the 1.5° WeatherBench 2 grid against ERA5. We measure CRPS, spread-skill ratio, the kinetic-energy spectrum against the 0.25° truth, and fine-scale precipitation quantiles.
A null result on the third check, with dense probabilistic models winning everywhere, would argue against using regime-routed designs for subgrid closure. A positive result would give modelers a measured, objective-controlled criterion for when discrete experts suit a subgrid process and when a smooth closure does.
References
- Learning to cluster neuronal function
- (Sparse) Attention to the Details: Preserving Spectral Fidelity in ML-based Weather Forecasting Models
- WeatherBench 2: A Benchmark for the Next Generation of Data-Driven Global Weather Models
- Decomposing weather forecasting into advection and convection with neural networks
- Diversity of Interneurons in the Dorsal Striatum Revealed by Single-Cell RNA Sequencing and PatchSeq
- Classes and continua of hippocampal CA1 inhibitory neurons revealed by single-cell transcriptomics
- Learning Continuous Regional Temperature Fields with Lead-Time and Resolution Queries
- Multimodal Atmospheric Super-Resolution With Deep Generative Models
- EWMoE: An Effective Model for Global Weather Forecasting with Mixture-of-Experts
- WeatherBench: A Benchmark Data Set for Data-Driven Weather Forecasting
- ARROW: An Adaptive Rollout and Routing Method for Global Weather Forecasting
- Earth Virtualization Engines (EVE)