Between Places and Between Years: Measuring When Space-for-Time Misleads Crop Climate Risk

Much of what we believe about how crops will respond to warming comes from comparing places. Maize yields in hot districts are compared with yields in cool districts, and the difference is read as the damage a cool district will suffer when it warms. This "space-for-time" logic is a cross-sectional design: a snapshot across units standing in for a trajectory within one unit. Developmental neuroscience learned the hard way that such snapshots confuse stable baseline differences with actual rates of change (A Baseline for the Multivariate Comparison of Resting-State Networks A Baseline for the Multivariate Comparison of Resting-State Networks, Brain ageing in schizophrenia: evidence from 26 international cohorts via the ENIGMA Schizophrenia consortium; Brain ageing in schizophrenia: evidence from 26 international cohorts via the ENIGMA Schizophrenia consortium). Agriculture has the same problem in a sharper form. Hot places differ from cool places in soil, irrigation, cultivar choice, and decades of adaptation, and these differences are tangled with temperature itself.

We propose to measure that confusion directly and crop by crop. For each crop we split the temperature–yield partial correlation, meaning the correlation left after statistically removing precipitation, into a between-place part and a within-place, year-to-year part. The gap between the two becomes a quantity worth studying. We then calibrate what the gap means on a simulated benchmark where the true response is known and where we deliberately plant the confounders that real farms have. The result is a diagnostic that tells an analyst, for a given crop and region, whether a cross-sectional estimate of warming damage can be trusted, is picking up genuine adaptation, or is biased by site differences.

Core Claim

The most careful existing comparison of statistical designs, On the use of statistical models to predict crop yield responses to climate change, tested cross-section, time-series, and panel regressions against CERES-Maize simulations at 198 Sub-Saharan sites. It found that pure cross-sectional models predicted the +2°C response best, with a site-level median absolute deviation (MAD) of 7.7% against 10.0% for panel models and 14.2% for site time-series. That result motivates this proposal and also limits how far it can be generalized. In the simulation, every site ran the same crop model with fixed management. Nothing in the setup could create the omitted spatial confounders or adaptation gradients that make space-for-time dangerous. We claim that the cross-sectional advantage is conditional on the absence of confounders. We will show how quickly it reverses as realistic confounding is added, and we will show that the between-minus-within gap in partial correlation, measured on observed data, tracks the resulting error. Across crops, we expect the gap to be largest where farmers have spread a crop across a wide climate range by switching cultivars, and smallest where the crop is grown under narrow, uniform practice. If that holds, "differential vulnerability" across crops is partly a property of the estimator and not only of the plant.

Research Program

The decomposition works as follows. For crop $c$, county or site $i$, and year $t$, we fit

$$ y_{it} = a_t + \beta_W\,(T_{it}-\bar T_i) + \beta_B\,\bar T_i + \gamma_W\,(P_{it}-\bar P_i) + \gamma_B\,\bar P_i + \varepsilon_{it}. $$

In this model, $y_{it}$ is log yield, $a_t$ is a year effect absorbing shared technology and price shocks, $T$ and $P$ are growing-season temperature and precipitation, and bars denote each unit's multi-year mean. Here $\beta_W$ is the within-unit sensitivity to a temperature shock, $\beta_B$ is the across-space sensitivity, and $\Delta = \beta_B - \beta_W$ is the space-for-time gap.

We report the same split in partial correlation form. We residualize the unit-mean and anomaly components of temperature and yield on their precipitation counterparts, then correlate the residuals. The unit-free version lets us compare corn, soybeans, cotton, and wheat, even though their yields are measured in different units. The within-unit part is exactly a fixed-effects panel estimate, and we say so plainly. We do not claim a new estimator. The contribution is the gap as a crop-level diagnostic and its calibration against known truth.

Where the method comes from matters. The logic of using partial correlation to separate co-varying heat and water comes from Regional and seasonal partitioning of water and temperature controls on global land carbon uptake variability. That paper showed that temperature and water storage effects on carbon uptake only separate cleanly after partialling. It also showed that a data-driven upscaling product (FLUXCOM) reached a spurious temperature dominance. T. Wang et al. found a negative summer warming effect that only appeared once precipitation and cloud cover were held fixed. Both papers compute temporal partial correlations only. Neither asks how spatial and temporal partial correlations disagree, and neither studies crops.

Climate change adaptation in crop production: Beware of illusions argues conceptually that ordinary yield-raising management is often miscounted as climate adaptation. It supplies the interpretation we will test quantitatively: a between-place slope flatter than the within-place slope may reflect real adaptation, or it may reflect impact-neutral management that happens to correlate with climate.

The benchmark is built with confounding by design. We will reproduce the design of On the use of statistical models to predict crop yield responses to climate change On the use of statistical models to predict crop yield responses to climate change: CERES-Maize at 198 sites, 39 simulated years of Marksim weather, WISE soils, FAO planting windows, and +2°C and −20% precipitation perturbations. We will then add controlled layers of realism, one at a time and in combination.

The first layer is adaptation. Cultivar maturity and planting date are set by each site's mean temperature, as farmers would choose them, but they are held fixed when the +2°C shock is applied. This defines two truths: the short-run response with management fixed, and the long-run response with management re-optimized.

The second layer is pure confounding. Soil fertility and irrigation access are assigned to correlate with mean temperature and have no causal link to it.

The third layer is survivorship. Sites whose simulated baseline yield falls below a viability threshold are dropped from the sample, mimicking the selection filter that the cross-sectional literature warns about.

For each layer and each strength of correlation, we know the true short-run and long-run responses. We can therefore map which true quantity each estimator actually recovers, and how the observable $\Delta$ relates to its error.

The empirical application uses the CropNet dataset from An Open and Large-Scale Dataset for Multi-Modal Climate Change-aware Crop Yield Predictions. It covers more than 2,200 US counties, four crops, WRF-HRRR daily and monthly weather, and USDA county yields for 2017–2022. Six years is too short for per-county time series, and On the use of statistical models to predict crop yield responses to climate change On the use of statistical models to predict crop yield responses to climate change shows that time-series estimates already degrade when records shrink from 39 to 20 years. We therefore pool the within-county anomalies across counties to estimate one within slope per crop and region, which uses thousands of county-years. We estimate uncertainty by bootstrapping whole counties with spatial blocks.

We deliberately do not control for satellite greenness (NDVI). Greenness lies on the causal path from weather to yield, so partialling it out would remove the effect we are trying to measure. This is a common error in pipelines that combine imagery and weather.

Standard partial correlation is linear, while heat damage has thresholds. As an ablation, we will replace the linear residualization with flexible nuisance models, gradient-boosted trees fit with cross-fitting, and a degree-day temperature term. This checks whether the gap survives once nonlinearity is handled.

Baselines

The naive reference point is the bivariate correlation between temperature and yield, pooled across counties and years, together with the "no change" forecast of zero yield impact. On the simulated benchmark, the zero forecast has an error equal to the true median loss, which for +2°C is 14.4% in On the use of statistical models to predict crop yield responses to climate change On the use of statistical models to predict crop yield responses to climate change. Any estimator has to beat that to count as useful.

The strongest existing competitor is the pure cross-sectional regression from the same paper. It reports site-level MAD of 7.7% and country-level MAD of 4.2% for the +2°C response. For −20% precipitation, it reports 5.4% at site level and 4.1% at country level. We will re-run it on our reproduced clean benchmark first, to confirm we recover roughly those numbers, and then run it on every confounded variant. The panel fixed-effects regression, reported at 10.0% site and 5.7% country MAD for temperature, is our within-unit estimator itself. It serves both as a baseline and as the $\beta_W$ component. Time-series models (14.2% site MAD) complete the set from that paper. All estimators see the same sites, years, and covariates. None receives hyperparameter tuning beyond what the original paper specified, apart from the flexible-residualization ablation, which gets one fixed cross-fitting setup.

For CropNet, the strongest predictive system is MMST-ViT from MMST-ViT: Climate Change-aware Crop Yield Prediction via Multi-Modal Spatial-Temporal Vision Transformer. It reports soybean RMSE of 3.9 and Pearson correlation of 0.918, against 5.4 for GNN-RNN and 7.2 for ConvLSTM. We will not try to beat its prediction accuracy, since that is not our question. Instead, we will extract the temperature sensitivity it implies by shifting the input HRRR temperatures by +2°C and recording the predicted yield change. We then compare that implied sensitivity with $\beta_W$ and $\beta_B$ from the same counties.

The key ablation removes the within/between split and reports a single pooled partial correlation. If crop rankings and error diagnostics are unchanged, the decomposition adds nothing.

The proposal counts as successful on the benchmark if two conditions hold. First, under at least moderate injected confounding, the cross-sectional estimator's MAD against short-run truth rises above that of the within estimator. Second, the observable $\Delta$ predicts the cross-sectional estimator's error with a rank correlation clearly above zero across benchmark variants. A third requirement is that the diagnostic separates confounding from adaptation. Partialling out observed site covariates should shrink $\Delta$ under pure confounding, but not under adaptation.

Why not just…?

Why not just run a fixed-effects panel regression, which already gives the partial effect of temperature holding precipitation fixed? It does, and we use it as one half of the decomposition. What a panel regression alone cannot tell you is whether the cross-sectional number that policy documents often quote is wrong, and in which direction. Those numbers are quoted because cross-sectional estimates are stable with as little as 5 years of data. The informative quantity is the disagreement between the two estimates, and the benchmark tests whether that disagreement predicts error. If the ablation shows that $\Delta$ carries no information about error, this objection wins and the paper reduces to a recommendation to use fixed effects.

Why not just trust cross-sectional models, since they won in the only direct test? The win in On the use of statistical models to predict crop yield responses to climate change On the use of statistical models to predict crop yield responses to climate change came from a world with no spatial confounders or adaptation. This is a real threat to our thesis, because realistic confounding might turn out to be too weak to matter. The dose-response over injected confounding strength settles it. We will report the level of correlation between site covariates and climate at which the cross-sectional advantage disappears, and compare it with the size of $\Delta$ observed in CropNet. That tells us whether real US counties sit on the safe or unsafe side of that threshold.

Why not just let a spatio-temporal transformer learn the interactions? A high-accuracy predictor can still use temperature as a proxy for place. (Regional and seasonal partitioning of water and temperature controls on global land carbon uptake variability; Regional and seasonal partitioning of water and temperature controls on global land carbon uptake variability) documents a data-driven flux product reaching the wrong dominant climate driver. MMST-ViT's own ablation shows that removing long-term climate costs only about 0.06 in correlation, which suggests its accuracy rests mostly on concurrent imagery. The perturbation test shows directly whether the transformer's implied warming response sits near the within or the between estimate. If it matches the within estimate, the transformer is a legitimate alternative, and we will say so.

Proposed Experiments and Decisive Checks

The first experiment is a replication. The clean reproduced benchmark must return cross-section, panel, and time-series MADs close to 7.7%, 10.0%, and 14.2% for +2°C at site level. Otherwise, the later comparisons are not anchored to the published result.

The second experiment sweeps confounding. We vary the correlation of cultivar choice, soil fertility, and irrigation with site temperature from zero to strong. At each level, we record each estimator's MAD against the short-run and long-run truths, along with the observed $\Delta$.

The decisive check has two parts. First, under pure confounding, the cross-sectional estimator should track neither truth, and $\Delta$ should shrink once the confounders are partialled out. Second, under the adaptation layer, it should track long-run truth, and $\Delta$ should persist after partialling. If $\Delta$ behaves identically in both cases, the diagnostic cannot tell bias from adaptation. We would then report it only as a warning flag.

The third experiment applies the decomposition to the four CropNet crops. We report $\beta_W$, $\beta_B$, $\Delta$, and the within and between partial correlations for each crop, with block-bootstrap intervals. We also compare these against the MMST-ViT perturbation sensitivities. The substantive test is whether the crop ranking of vulnerability changes between the within and between estimates. If corn and soybeans swap places, or winter wheat moves from least to most vulnerable, then a published ranking of differential crop vulnerability depends on the choice of study design.

The final ablation swaps linear residualization for cross-fitted flexible models with a degree-day heat term, which checks whether any crop gap is an artifact of forcing a straight line through threshold damage.