Do Drought Forecasters Predict Drought, or Just Place?

Deep learning models that forecast drought weeks to months ahead are usually scored by one number: the average error over every pixel and every week of a continental map. That number can look good even when the model has learned very little about drought. Deserts are usually dry and humid croplands are usually wet. A model that reads a pixel's elevation and land cover can therefore predict a lot of the map from "where am I" alone. Computer vision has a name for this failure: contextual bias, or shortcut learning, where a model leans on the background (water behind a boat) instead of the object. Such models collapse when the background and the object disagree. The drought analogue of "a cow in a desert" is a drought striking a place that rarely has one. That case matters most in practice, and it is where a location-reliant model should fail.

We propose to test this directly and, if it holds, to fix it. The test uses the DroughtSet benchmark, which covers the contiguous United States from 2003 to 2013. We define evaluation groups by how rare a drought state is given the pixel's static context, and we measure worst-group error instead of pooled error. We also run counterfactual probes that swap static context between pixels that share the same weather. The fix trains the model in two parts: one predicts each place's normal behavior from static features, and a second predicts departures from normal from the weather. The second part is penalized when its accuracy differs across landscape types. The contribution is foremost a diagnostic. If simple fixes close the gap, we will say so, and the diagnostic still stands as the result.

Core Claim

DroughtSet / SPDrought (companion) forecasts three drought indicators up to 26 weeks ahead at 4 km, weekly resolution:

  • normalized surface soil moisture;
  • the Evaporative Stress Index (ESI, a standardized anomaly of actual-to-potential evapotranspiration);
  • solar-induced chlorophyll fluorescence (SIF, a proxy for photosynthesis).

The model uses static features (elevation, land cover) to decide how much each neighboring cell's weather contributes. It is trained by plain average-loss minimization and scored by pooled mean absolute error (MAE).

We claim that a substantial share of its skill on soil moisture and SIF is location-conditional climatology, meaning the typical value for that kind of place at that time of year. We further claim that its error concentrates on "context-atypical" drought, where a pixel falls into a drought state that its land-cover and elevation stratum rarely showed in training.

The claim is falsifiable in a specific, internal way. ESI is already an anomaly, so the static shortcut has less to offer there. The shortcut signature should therefore be much weaker for ESI than for soil moisture and SIF. If all three targets show equal static sensitivity, our account is wrong.

Research Program

Groups. We first build the evaluation groups. Each pixel gets a static-context stratum, formed by crossing three things: land-cover class (from the land-cover map in DroughtSet), elevation band, and a training-period aridity class (mean potential evapotranspiration over precipitation).

Each pixel-week gets a drought state, defined relative to that pixel's own training-period distribution. We borrow the percentile logic of the Evaporative Stress Percentile, which calls a pixel in drought below the 20th percentile. We also flag flash-drought onsets using that paper's rule: a fall from above the 40th to below the 20th percentile with a decline of more than 6.5 percentile points per week.

A group is a (stratum, state) pair. Its rarity is the training-set frequency of that state within that stratum. This definition turns the vision field's "rare attribute combination" into climate terms. CB-SLICE showed that such combinations hide behind high averages in image classifiers. It needed a predefined concept dictionary, and here the dictionary comes directly from physical metadata.

Probes. Second, we run three probes on SPDrought and an iTransformer baseline. All are trained exactly as published.

  1. The static-only probe is a model that sees only static features plus week-of-year. We measure what fraction of SPDrought's pooled MAE it recovers.
  2. The context-swap probe works at test time. We cluster pixels by their dynamic forcing trajectory (precipitation, temperature, vapor pressure deficit, potential evapotranspiration). Within each cluster, we swap static features between pixels from different strata. We then measure how much predicted anomalies move, which isolates how much the output depends on context once the weather is held fixed.
  3. The hidden-year test holds out 2012, the widespread Central US drought year, in which humid croplands entered a rare state.

This replaces the post-hoc Integrated Gradients audit in the DroughtSet papers. That audit ranks input importance but cannot separate "static features are used legitimately" from "static features stand in for the answer." We considered head-level localization in the style of Locate-Then-Correct, which splits CLIP's output into contributions from individual attention heads to find spurious heads. That tool assumes a classification readout, and the paper itself notes it struggles in multi-class settings. We keep it as an optional follow-up rather than the core.

Fix. Third, the intervention. We separate the prediction as

$$\hat y_{i,t} = b_\phi(s_i, \tau_t) + a_\theta(x_{i,\le t}, s_i),$$

where $y_{i,t}$ is the target at pixel $i$ and week $t$, $s_i$ are the pixel's static features, $\tau_t$ is week-of-year, and $x_{i,\le t}$ is the dynamic forcing history of the pixel and its neighbors. The term $b_\phi$ is a static-only climatology network, fit first and then frozen. The term $a_\theta$ is the anomaly network, keeping SPDrought's architecture.

Static features stay available to $a_\theta$, because soils and canopies genuinely change how fast a landscape dries. We add a penalty borrowed from Invariant Risk Minimization, which seeks predictors that work equally well across training environments. We use the simpler variance-of-risk form:

$$\mathcal{L} = \frac{1}{|E|}\sum_{e\in E} R_e(\theta) + \lambda\,\mathrm{Var}_{e\in E}\,R_e(\theta).$$

Here each environment $e$ in the set $E$ is a static-context stratum. $R_e$ is the anomaly MAE on drought-state weeks within that stratum, and $\lambda$ sets the strength of the penalty.

The physical motivation comes from regional studies. Each one finds that the rules linking temperature, vegetation, and water change across settings, and each uses classical statistics:

  • at high elevations, warming lowers the ratio of actual to potential evapotranspiration and suppresses greening (Tibetan alpine study);
  • in sparse drylands, soil background corrupts vegetation-based dryness signals (Central Asia study);
  • near wind farms, turbine mixing dries soils locally (wind-farm study).

We use these studies as a reason to expect unstable pooled rules, not as a method.

Baselines

The naive reference points are persistence and the static-only probe. Persistence means carrying the last observed value forward, and the ESP-percentile persistence of the flash-drought paper plays the same role for the drought-state metrics. The static-only probe tells us how much of the benchmark a model can "solve" without weather. Both are cheap to compute, and both must be beaten by a large margin on worst-group error for any learned model to claim drought skill.

The strongest existing method is SPDrought as published. It reports a total 26-week-ahead MAE of 38.01 × 10⁻³, against 45.18 × 10⁻³ for iTransformer (DroughtSet). We retrain it with the released configuration on our splits, and we give it the same hyperparameter-search budget as every variant. No worst-group or rare-context result has been reported for it: no reported result found on Lacuna for worst-group MAE on DroughtSet.

We will count our method as beating SPDrought if two things hold across three seeds. First, it cuts worst-group anomaly MAE and flash-onset miss rate on the 2012 hold-out by a margin larger than the seed spread. Second, its pooled MAE stays within 5% of SPDrought's.

Three ablations isolate the key ingredient. The first is the separated $b_\phi + a_\theta$ model with $\lambda = 0$, which tests whether the separation alone does the work. The second is SPDrought with static inputs removed. The third is SPDrought trained with group-reweighted loss using our rarity weights. Every variant uses identical data, epochs, and search budget.

Why not just…?

Why not just train on anomalies and drop the static features? Predicting departures from climatology may remove most of the shortcut on its own, and this is the most serious threat to the proposal. Removing static features entirely should cost accuracy, because soil and land cover really do govern how fast soil moisture drops. Our first ablation settles the question. If the separated model with $\lambda = 0$ matches the full method on worst-group error, we drop the invariance penalty and report the separation plus the diagnostic as the finding.

Why not just reweight rare drought weeks? Group reweighting is the standard fix for worst-group failures. In vision, Shi et al. found that class-balanced resampling lowered CIFAR-100-LT accuracy from 38.3% to 34.1%. The model overfit to the narrow contexts of the repeated samples, which is exactly the risk when a few rare drought episodes are upweighted within their own geography. That is evidence from images, not climate, so we run reweighting as a full competitor rather than dismissing it.

Why not just swap contexts, as CutMix-style augmentation does? CMO pastes rare objects onto backgrounds taken from common classes. The drought analogue would pair a forcing sequence with a foreign landscape. But no ground-truth soil-moisture response exists for such a pairing, so the labels would be fabricated. We therefore use context swapping only as a test-time probe, never as training data.

Why not just use physical indices like the Temperature Vegetation Drought Index (TVDI) or ESP? These indices describe current conditions rather than forecasting 26 weeks ahead. TVDI's dry and wet edges also become miscalibrated over heterogeneous terrain (Heilongjiang TVDI). They serve as our naive references and as sources for group definitions, not as rivals.

Proposed Experiments

We evaluate on DroughtSet with two splits. One is a temporal split that holds out 2012. The other is a spatial split that holds out whole land-cover strata, with a buffer region excluded between training and test cells.

The metrics are:

  • pooled MAE;
  • worst-group anomaly MAE across (stratum, state) groups;
  • MAE as a function of group rarity;
  • flash-drought onset hit rate and false-alarm rate under the ESP rule, applied to ESI forecasts.

Three results decide the claim.

  1. Static-only share. If the static-only probe recovers most of SPDrought's pooled MAE on soil moisture and SIF but not on ESI, the benchmark rewards place-memorization. That is publishable regardless of what follows.
  2. Rarity slope. If SPDrought's error grows steeply with group rarity, and its context-swap sensitivity on held-out 2012 cropland pixels is large, the shortcut is real.
  3. Swap sensitivity and worst-group gain. The full method should reduce swap sensitivity and worst-group error beyond all three ablations. If it does not, the anomaly separation or reweighting is the answer, and we will report it as such.