Weather models lose most of their skill after about two weeks. What predictability remains at three to six weeks comes mostly from slow parts of the climate system, chiefly soil moisture. A dry Mediterranean spring stops evaporative cooling, pushes surface energy into heating the air, and primes Europe for a summer heatwave. Forecasting from "analogs," meaning finding past situations like today and reading off what happened next, is an old way to exploit this kind of slow memory. A modern Hopfield network (an associative memory whose recall step is the same softmax-weighted lookup as Transformer attention) is a soft, tunable version of analog forecasting.
The theory of such memories says recall has phases. Below a capacity limit, a query falls cleanly into one stored memory. Above it, or when memories are too similar, the query lands on a blurred mixture of many memories, which in forecasting is just climatology. We propose to use this phase structure directly. Over a 45-year reanalysis archive of land–atmosphere states, a query that falls into a separated recall phase should mark a "window of opportunity" where weeks 3–6 are predictable. A query that falls into the mixture phase should mark a day where the honest forecast is climatology. The new pieces are threefold. First, we measure retrieval phase diagrams on strongly correlated geophysical fields, which capacity theory does not yet cover. Second, we test whether capacity is set by the effective dimension of the slow land-surface keys rather than the raw grid size. Third, we test whether the phase of each query predicts forecast skill better than simpler confidence signals.
Core Claim
Use stored land-surface states (soil moisture layers, surface heat fluxes) as keys and the subsequent 15–45-day atmospheric anomalies as values. Then two things should hold. First, the number of past episodes that can be told apart as separate attractors should scale with the effective dimension of the keys, not with the number of grid points. Second, the sharpness of each query's recall should be an a priori indicator of week 3–6 skill. That indicator should be calibrated well enough to beat raw soil-moisture anomaly size and plain nearest-neighbor distance as confidence signals.
Concretely, with stored keys $K$ and values $V$, a query key $k$ is retrieved as
$$\hat{v} = V^\top \mathrm{softmax}(\beta K k),$$
where $\beta$ is the inverse temperature. The entropy of the softmax weights, read in units of independent past episodes rather than days, measures how many distinct histories the forecast is averaging over.
The capacity hypothesis comes from replacing the grid size $N$ with the participation ratio $N_{\mathrm{eff}} = (\sum_i \lambda_i)^2 / \sum_i \lambda_i^2$, where $\lambda_i$ are the eigenvalues of the key covariance:
$$P_{\max} \approx \exp(\alpha\, N_{\mathrm{eff}}),$$
with $\alpha$ the load at which recall collapses. This form takes the i.i.d. exponential-capacity result of Exponential Capacity of Dense Associative Memories and applies it to whitened keys. Whitening makes correlated fields roughly isotropic in $N_{\mathrm{eff}}$ directions. Whether the formula survives non-Gaussian, multiscale fields is exactly what we test.
Why This Intersection, and What Prior Work Leaves Open
The physical motivation is well established. Using station records and MM5 regional twin simulations, Summertime European heat and drought waves induced by wintertime Mediterranean rainfall deficit found a correlation of $r=-0.55$ between January–May southern European rainfall frequency and summer maximum temperature further north. Its dry-soil twin runs warmed parts of central Europe by up to 2°C. Hot European Summers and the Role of Soil Moisture in the Propagation of Mediterranean Drought repeated the perturbation across the ten hottest summers and found 2–5°C anomalies together with anticyclonic responses. Both papers are sensitivity studies inside fixed-boundary regional models, not forecasting systems. They tell us which slow variable should form the key, and nothing about how to retrieve from history.
The memory side contributes both method and constraint. The exponential-capacity analysis supplies the softmax–attention equivalence and the abrupt retrieval cliff. It also states explicitly that its guarantees assume uncorrelated random patterns. The S/N Analysis of Associative Memory and Basins of Attraction for Electronic Neural Networks show how crosstalk between correlated or overloaded memories breaks basins into spurious states. That is the failure mode we expect when consecutive summers look alike.
The closest applied work is Probabilistic Forecasting via Modern Hopfield Networks. It builds a Hopfield encoder–decoder and reports that $\beta$ moves forecasts between averaged and sharp pattern lookup, but only on one-dimensional series, with spatio-temporal data left unexplored. We differ in three ways. Our memories are spatial land states. We treat $\beta$ and the retrieval phase as a per-query diagnostic rather than a tuned hyperparameter. And our evaluation question is when skill exists, not only average error.
Two other memory papers are deliberately not used as the core mechanism. The sequence memory of Exponential Dynamic Energy Network for High Capacity Sequence Memory and the Wasserstein-geometry memory of Dense associative memory for Gaussian distributions are elegant. However, the binding constraint here is not sequence length or distributional geometry. It is that a 45-year archive contains only a few dozen independent summers, which are highly correlated. A simple retrieval rule whose phases we can measure is the right first object.
Research Program
The first component is a measured phase diagram on real fields. We build key sets from the land-surface and atmospheric channels in ChaosBench at 1.5°, restricted first to Europe and the Mediterranean in April–June. Four representations are compared: raw grid anomalies, EOF-truncated anomalies, whitened anomalies, and land-only versus atmosphere-only keys. For each, we store $P$ states, corrupt held-out copies with structured noise (spatially correlated perturbations matched to reanalysis uncertainty, and masked regions), and record retrieval accuracy as a function of $\beta$ and $P$. We then compare the observed retrieval cliff with the $N_{\mathrm{eff}}$ prediction.
One subtlety is central. Neighboring days within a season are nearly identical, so mixtures among them are harmless. Mixtures across different years are what destroy information. We therefore score retrieval at the level of independent episodes (year–season blocks), not individual days.
The second component is a forecaster built from this memory. The value attached to each key is the observed 15–45-day anomaly of 2-meter temperature, 850-hPa temperature, and precipitation. For each query we compute $\hat{v}$, an ensemble formed from the softmax-weighted episodes, and the episode-level retrieval entropy. $\beta$ is chosen by leave-one-year-out cross-validation within the training years only. In a second variant, $\beta$ is set per query by a self-consistency check: is the fixed point reached from the query stable under small key perturbations?
The third component tests the claim that the retrieval phase is a skill indicator. We bin test forecasts by retrieval entropy and ask whether skill rises monotonically in the low-entropy bins. We also ask whether those bins concentrate in physically interpretable states, such as springs with dry southern Europe in the sense of the MM5 studies.
Baselines
The naive references are climatology and anomaly persistence. Climatology is the hardest naive bar at these leads. ChaosBench reports a climatology RMSE of 3.39 K for 850-hPa temperature at a 44-day lead. Any method that does not beat this on average must at least beat it inside its claimed windows of opportunity to be worth anything.
The strongest learned competitor with a reported number is the direct (non-autoregressive) ViT/ClimaX model in ChaosBench. It reaches 3.56 K RMSE on 850-hPa temperature at a 44-day lead, slightly worse than climatology. Autoregressive Pangu-Weather lags at 5.85 K in the same benchmark. We will rerun ClimaX-direct on the same ChaosBench splits and variables. Our memory forecaster gets the same training years and the same hyperparameter search budget (an equal number of validation evaluations). It counts as beating ClimaX if it is lower than ClimaX on overall RMSE and also lower than climatology in the top-quintile confidence bin, with the gap holding across the held-out years.
For the CONUS replication on SubseasonalClimateUSA, the strongest competitor is Adaptive Bias Correction applied to dynamical forecasts, reported as roughly a 10% RMSE improvement over CFSv2. No absolute RMSE values were found in the file. We will therefore run the released ABC variants (Persistence++, Climatology++) on our test years and compare against them directly.
The most important baseline is classical k-nearest-neighbor analog forecasting. Neighbors are found by Euclidean distance on the same keys, with $k$ and the distance weighting tuned under the same budget, and mean neighbor distance serves as the confidence signal. No paper in the file reports results for this baseline, so we report our own run.
Three ablations isolate the key ingredients:
- Atmosphere-only keys versus land keys. This tests whether the slow land memory is what creates sharp retrieval.
- Unwhitened versus whitened keys. This tests whether capacity is governed by $N_{\mathrm{eff}}$.
- Fixed $\beta$ versus phase-aware $\beta$. This tests whether the per-query phase adds anything.
Why not just…?
Why not just use k-nearest-neighbor analogs? This is the real threat. A softmax memory at large $\beta$ is close to one nearest neighbor, and at small $\beta$ it is kernel regression, so average skill may be indistinguishable. Our claim lives elsewhere. The theory says which representation makes episodes separable, and the episode-level retrieval phase should be a better-calibrated confidence signal than raw neighbor distance. The decisive comparison is the reliability curve of retrieval entropy against the curve of kNN distance. If they match, we will report that the contribution is the capacity analysis and representation choice, not the forecaster.
Why not just threshold the soil-moisture anomaly itself? A large dry anomaly may already flag windows of opportunity without any memory machinery. We include anomaly magnitude, area-averaged over southern Europe, as a competing confidence signal in the same reliability analysis. Retrieval phase earns its place only if it identifies skillful cases that anomaly size misses. Candidates are configurations with moderate anomalies but a distinctive spatial pattern, or large anomalies whose historical outcomes disagreed.
Why not just train a bigger sequence model or roll out GraphCast longer? The evidence in the file is unfavorable. SubseasonalClimateUSA reports that deep sequence models such as Informer overfit and underperform simple baselines at these ranges. ChaosBench shows autoregressive Pangu-Weather far worse than climatology at 44 days. GraphCast uses a fixed 6-hour step and an MSE objective that blurs long-range features. We include ClimaX-direct as the representative and do not attempt month-long rollouts.
Why not just use bias-corrected dynamical forecasts? ABC is strong and cheap, and on CONUS it may win on average. The two approaches are complementary rather than exclusive, so we also test the memory forecast's retrieval entropy as a gate on when to trust ABC's anomalies over climatology. If the gate improves ABC in the confident bins, the result stands even if the memory forecaster itself loses.
Proposed Experiments
The data split trains on 1979–2010 and tests on 2011 onward, with all $\beta$ and representation choices made by leave-one-year-out within the training years. Metrics are latitude-weighted RMSE, anomaly correlation, CRPS of the episode ensemble, and ChaosBench's spectral divergence, which checks whether retrieved fields keep spatial detail where MSE-trained models blur.
Three checks decide the proposal:
- Capacity scaling. In the phase-diagram experiment, the observed retrieval cliff for whitened land keys should fall within a factor of the $\exp(\alpha N_{\mathrm{eff}})$ prediction. Atmosphere-only keys should collapse far earlier. If raw and whitened keys behave identically, the effective-dimension hypothesis is wrong.
- Calibration of the phase signal. The top quintile of queries ranked by low retrieval entropy should show week 3–6 2-meter temperature anomaly correlation clearly above the bottom quintile. Its reliability curve should dominate those of kNN distance and soil-moisture anomaly magnitude on the held-out years.
- Physical content of the confident cases. Confident European cases should be enriched for dry-southern-soil springs, consistent with the MM5 mechanism. If confident cases are physically arbitrary, the signal is likely exploiting sampling luck rather than land memory.
References
- Exponential Capacity of Dense Associative Memories
- Summertime European heat and drought waves induced by wintertime Mediterranean rainfall deficit
- Hot European Summers and the Role of Soil Moisture in the Propagation of Mediterranean Drought
- The S/N Analysis of Associative Memory
- Basins of Attraction for Electronic Neural Networks
- Probabilistic Forecasting via Modern Hopfield Networks
- Exponential Dynamic Energy Network for High Capacity Sequence Memory
- Dense associative memory for Gaussian distributions
- ChaosBench: A Multi-Channel, Physics-Based Benchmark for Subseasonal-to-Seasonal Climate Prediction
- SubseasonalClimateUSA: A Dataset for Subseasonal Forecasting and Benchmarking
- GraphCast: Learning skillful medium-range global weather forecasting