Deep learning models now predict river flow in basins that have never had a stream gauge. They do this by training on thousands of gauged basins and describing each basin by fixed attributes such as soil, slope and forest cover. Their accuracy has been reported under evaluation designs that mix years freely or hold out a single static block. Climate, land use and water management change over decades, though. When a deployed model's skill decays at an ungauged river, nobody can see it and nobody can refit it, because that river has no flow measurements. Medicine's standard answer to model decay is to watch recent labels and refit, and it has nothing to watch at an ungauged river.
This note proposes a way to monitor and correct an ungauged basin using its gauged neighbours. First, backtest global streamflow models the way they would actually be deployed: train on early decades, then predict later decades at basins held out in both space and time. Second, split any loss of skill into two parts. One is loss of timing, meaning the hydrograph's shape and event ranking. The other is loss of scale, meaning total volume and variability. Third, if the damage is mostly in scale, estimate it at each ungauged basin from the recent errors of gauged "sentinel" basins, weighted by how similar each sentinel's physical attributes and observed weather shift are to the target's. Weather inputs remain available everywhere even where flow is never measured, so the correction needs no flow data at the target. That removes the missing-labels problem that makes the medical approach inapplicable.
Core Claim
The claim has two parts, and both can be falsified. The first part is that temporal drift in ungauged streamflow prediction behaves like temporal drift in clinical prediction. Calibration, meaning the absolute volume and spread of predicted flow, degrades much more than discrimination, meaning timing and correlation. The systematic review of temporal shift in clinical ML found this asymmetry repeatedly and found that recalibration restores calibration but not discrimination. Nobody has checked whether hydrology shows the same pattern.
The second part is that the scale error at an ungauged basin can be predicted from two inputs. One is the recent errors of gauged sentinels. The other is drift descriptors the target basin itself exposes without labels: shifts in its mean precipitation, temperature, aridity and precipitation variability relative to the training era.
The correction is an affine rescaling of each basin's predicted hydrograph: it changes the hydrograph's level and spread without changing its shape or timing. Because it leaves correlation untouched, the method fails in a transparent way. If decay turns out to sit in correlation, the method cannot help and the diagnostic shows that directly.
Relation to Prior Work
Two hydrology papers establish the models whose decay we measure. The Entity-Aware LSTM routes static catchment attributes through a dedicated gate. A long short-term memory network (LSTM) is a recurrent network that keeps an internal memory across time steps, and here the gate conditions that memory on basin identity. It reached a median Nash–Sutcliffe efficiency (NSE, where 1 is a perfect fit and 0 is no better than predicting the mean flow) of 0.74 on CAMELS. Global prediction of extreme floods in ungauged watersheds scaled an encoder–decoder LSTM to 5,680 gauges and matched or beat GloFAS v4, an operational physics-based flood system, at roughly 70% of gauges for 2-year floods.
Both treat catchments as static, and neither backtests rolling deployments. The global flood system is operational without any adaptation loop. We take these models as given and add what they lack: a deployment-faithful evaluation and a label-free update for the basins that need it most.
The method borrows directly from Transfer Learning Using Inaccurate Physics Rule. That paper models a physics equation's residual error in gauged basins and learns a network from static attributes to the residual coefficients, then transfers them to ungauged basins. That mapping is stationary over its 19-year record. Ours is indexed by deployment window and conditioned on observed weather drift, so it transfers errors that change over time rather than fixed ones.
Three results from the temporal-shift literature shape the design. The first motivates the evaluation. Evaluating Model Performance in Medical Datasets Over Time showed that random splits overstate deployed performance relative to chronological backtesting. It also showed that the choice between pooled and sliding-window training depends on data volume. We carry that backtesting protocol into the joint space-and-time setting.
The second constrains the method. The clinical evaluation of domain generalization and adaptation found that invariance-seeking methods such as Invariant Risk Minimization did not beat plain empirical risk minimization (ERM) on pooled history. That is why we do not try to make the network invariant to climate epochs. Instead we leave the network alone and correct its output.
The third frames the operational loop. The clinical quality-improvement perspective argues for continual monitoring and refitting. It also notes that label latency breaks those loops. An ungauged basin is the extreme case of label latency, with labels that never arrive, which is exactly where our proposal operates.
On the hydrology side, The proper care and feeding of CAMELS found that adding basins helps more than adding years. That gives us reason to expect errors that are shared across many basins to be learnable. The same paper found that low flows respond weakly to more data, which warns us to report low-flow results separately.
Research Program
The substrate is Caravan, built with the kratzert/Caravan code. It contains 6,830 basins with a median flow record of 31 years. All weather forcings come from ERA5-Land, one consistent reanalysis product. That consistency matters here: drift we detect is less likely to be an artifact of sensor or product changes, a confound the clinical literature highlights.
Basins are split into spatial folds. Within each fold, models train on an early era, for example up to 1995. They are then deployed in rolling multi-year windows through the end of the record at held-out basins, and are never refit on those basins' flows.
For every deployment window we split error with the Kling–Gupta decomposition, a standard hydrology score. It separates the correlation between predicted and observed flow from the bias ratio (predicted mean over observed mean) and the variability ratio (predicted spread over observed spread). We treat correlation as the analogue of discrimination and the two ratios as the analogue of calibration. We add error on the top 2% of flows and on the lowest 30% of flows, so that flood behaviour and drought behaviour are reported apart.
The correction works in log-flow space so that corrected flows stay positive. Let $\hat y_{b,t}$ be the frozen model's log-flow prediction for basin $b$ on day $t$ inside window $w$. Let $\hat\mu_{b,w}$ and $\hat\sigma_{b,w}$ be the mean and standard deviation of those predictions over the window. The corrected prediction is
$$\tilde y_{b,t} = \hat\mu_{b,w} + \delta_{b,w} + e^{\gamma_{b,w}}\,(\hat y_{b,t} - \hat\mu_{b,w}),$$
where $\delta_{b,w}$ shifts the level and $\gamma_{b,w}$ rescales the spread.
At gauged sentinels, $\delta$ and $\gamma$ are computed from their recent observed errors. Only observations older than an assumed reporting latency are used. A small regressor, such as ridge regression or a two-layer network, maps a feature vector to $(\delta, \gamma)$. The feature vector holds the basin's static attributes and its window-level forcing-drift descriptors, the latter being shifts in mean precipitation, mean temperature, aridity index and precipitation variance relative to the training era. The regressor then predicts $(\delta, \gamma)$ for every ungauged basin.
The regressor is refit each window and is tiny, so the correction runs at nearly no cost between full retrainings of the global model.
Baselines
The naive reference is the frozen model: an Entity-Aware LSTM trained on the early era and never updated. It also gives the reference NSE under a time-agnostic split of the same basins. The gap between the two, the optimism gap, is our first measurement.
The strongest published number for this architecture is a median NSE of 0.74 on CAMELS, reported in Benchmarking a Catchment-Aware LSTM and matched in The proper care and feeding of CAMELS. That figure comes from a static split, not a chronological ungauged backtest. No reported result was found on Lacuna for chronological ungauged backtesting on Caravan, so we establish that number ourselves.
The strongest real competitor is periodic pooled refitting. At each window, the global model is retrained from scratch on all gauged basins using every label available up to the latency cutoff. It then predicts the ungauged basins. This is the ERM-on-everything strategy that won in the clinical domain-generalization study. It lets updated gauged data flow to ungauged basins through the shared weights, so it is the most natural way to capture the same information our correction targets.
We also run a sliding-window variant that refits on only the most recent years. This follows the recency trade-off documented in Evaluating Model Performance in Medical Datasets Over Time.
Every refit baseline gets the same architecture, the same hyperparameter search budget as the original training, and the same labels and latency as the sentinel correction.
Three further baselines cover obvious shortcuts. A RevIN variant, from Reversible Instance Normalization, normalizes each input window by its own statistics before the network sees it. Two cruder corrections test whether the regressor earns its place. The global-shift correction applies the median sentinel $(\delta, \gamma)$ everywhere. The nearest-neighbour correction copies $(\delta, \gamma)$ from the most similar gauged basin.
The key ablation removes the forcing-drift descriptors from the regressor and keeps only static attributes. This isolates the claim that weather drift observed at the target predicts its error. An oracle that refits using the target's own recent flows provides the upper bound.
The proposal wins if two conditions hold. First, the correction closes a substantial share of the gap between frozen and oracle skill in bias and variability. Second, applied on top of the last pooled refit, it improves median NSE over pooled refitting alone in windows between retrainings. If it only ties pooled refitting when that refit happens every window, its value is cost and speed, not accuracy, and we will report it that way.
Why not just…?
Why not just retrain the global model on all recent gauged data? This is the real threat. Evidence from clinical ERM suggests it may capture most of the recoverable drift. The experiment settles the question directly by comparing pooled refitting alone with pooled refitting plus correction, at several refit frequencies and label latencies.
We expect the correction to add most when refits are infrequent. That is how operational flood systems actually run: the global flood model has no adaptation loop at all. We also expect the correction to add most when drift is regional, such as a multi-year drought. In that case the weather-drift descriptors carry information that one set of pooled weights averages away. If neither pattern appears, the correction is unnecessary, and the backtest and error decomposition remain the contribution.
Why not just recalibrate the outputs post hoc? Standard recalibration fits a correction against the target's own recent labels, and ungauged basins have none. Our method is recalibration with the labels borrowed from sentinels. The clinical finding that recalibration cannot restore discrimination (systematic review) applies here too. That is why we measure the correlation term separately rather than assume the correction is enough.
Why not just add RevIN? RevIN rescales outputs using the statistics of the input window. For rainfall to runoff, the absolute level of precipitation is the signal: a wet year should produce more flow, and normalizing that level away removes it. The flow scale is also not the precipitation scale. The RevIN paper itself notes that the method handles only mean and variance drift. We still run RevIN as a baseline, because it is the cheapest architectural fix and our argument could be wrong.
Why not just use calibrated physics models, which should be robust to climate change? Per-basin calibration is impossible at ungauged sites. Even where calibration is possible, individually calibrated SAC-SMA reached only 0.60 median NSE and mHM 0.67, against 0.74 for the Entity-Aware LSTM (Benchmarking a Catchment-Aware LSTM). The robustness argument has not been tested under chronological backtests, however. Where regionalized mHM outputs exist for CAMELS basins, we will score them in the same windows to check whether their skill decays more slowly.
Decisive Checks
Three results decide the proposal.
The first is the optimism gap. Chronological ungauged backtesting on Caravan should show lower median NSE than the time-agnostic split. If the gap is negligible, the premise weakens.
The second is the error decomposition. We check whether the decay concentrates in the bias and variability ratios while correlation stays stable, and whether the pattern differs for arid and flashy basins, where the global flood study already reports degradation. If decay sits in correlation, the affine correction cannot help by construction. That result would redirect the work toward catchment descriptors that change over time.
The third is the correction itself. We check whether the sentinel correction beats both the static-attributes-only ablation and pooled refitting between retrainings, with gains holding separately on high-flow and low-flow error. It must also hold across continents, since Caravan's heavy North American and European skew could make sentinel coverage fail exactly where it matters.
References
- Systematic Review of Approaches to Preserve Machine Learning Performance in the Presence of Temporal Dataset Shift in Clinical Medicine
- Benchmarking a Catchment-Aware Long Short-Term Memory Network (LSTM) for Large-Scale Hydrological Modeling
- Global prediction of extreme floods in ungauged watersheds
- Transfer Learning Using Inaccurate Physics Rule for Streamflow Prediction
- Evaluating Model Performance in Medical Datasets Over Time
- Evaluation of domain generalization and adaptation on improving model robustness to temporal dataset shift in clinical medicine
- Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare
- The Proper Care and Feeding of CAMELS: How Limited Training Data Affects Streamflow Prediction
- Caravan - A global community dataset for large-sample hydrology
- Reversible Instance Normalization for Accurate Time-Series Forecasting against Distribution Shift