Data-driven modelling of hydraulic-head time series: results and lessons learned from the 2022 Groundwater Time Series Modelling Challenge
Raoul Collenteur, Ezra Haaf, Mark Bakker, Tanja Liesch, Andreas Wünsch, Jenny Soonthornrangsan, Jeremy T. White, Nick Martin, Rui Hugman, Ed de Sousa, Didier Vanden Berghe, Xinyang Fan, Tim Peterson, Jānis Bikše, Antoine Di Ciacca, Xinyue Wang, Yang Zheng, Maximilian Nölscher, Julian Koch, Raphael Schneider, Nikolas Benavides Höglund, Sivarama Krishna Reddy Chidepudi, Abel Henriot, Nicolas Massei, Abderrahim Jardani, Max Gustav Rudolph, Amir Rouhani, J. Jaime Gómez‐Hernández, Seifeddine Jomaa, Anna Pölz, Tim Franken, Morteza Behbooei, Jimmy Lin, Rojin MeysamiHydrology2024 · DOI
The 2022 Groundwater Time Series Modelling Challenge pitted 15 international teams against each other to predict groundwater levels using everything from simple reservoir models to complex Deep Learning architectures. The results show that while AI is powerful, "simple" hydrological models remain remarkably competitive, and the choices made by the human modeller are often as important as the algorithm itself.
What Problem Does This Paper Address?
Managing groundwater—the hidden resource providing half of the world’s drinking water—requires accurate predictions of "hydraulic head" (water levels in wells). Traditionally, hydrologists used physics-based models that require detailed knowledge of the soil and rock layers. Recently, data-driven models (AI) have exploded in popularity because they only need historical weather and water level data.
However, the field lacks a "level playing field" comparison. Most papers claim their new AI model is the best, but they rarely test against other methods on the same data with a truly "blind" validation period. This paper summarizes a community-wide challenge to see which data-driven approaches actually hold water when predicting the future.
Methods
The organizers categorized the 15 participating teams into three modeling "families":
Lumped-Parameter (LP) Models: These are traditional hydrological "bucket" models (e.g., Pastas, HydroSight). They use a few simple equations to represent how rain enters the ground and moves toward a well. They are fast, transparent, and have very few parameters (5–15).
Machine Learning (ML): Models like Random Forests and Support Vector Regression. These rely heavily on "feature engineering"—modellers manually creating inputs like "average rainfall over the last 30 days" to help the model understand time delays.
Deep Learning (DL): The heavy hitters, including LSTMs (Long Short-Term Memory networks), CNNs, and Transformers. These are designed to "learn" temporal patterns automatically but are often seen as "black boxes."
Models were evaluated on their Nash–Sutcliffe efficiency (NSE) (a measure of how much better the model is than just using the average) and their ability to provide a 95% prediction interval (uncertainty).
Datasets
The challenge provided 15 years of daily weather and groundwater data for four distinct hydrogeological settings:
Netherlands: Unconfined aquifer in peat and sand (temperate).
Sweden: Hillside well in glacial till (subarctic/snow-influenced).
USA (Connecticut): Confined bedrock aquifer near a river.
Participants were given weather data for a 5-year "blind" validation period but were not shown the actual water levels for those years.
Key Insights and Ideas
Complexity is not a Silver Bullet: Simple lumped-parameter models performed nearly as well as the most complex Deep Learning models. In hydrology, a few well-chosen physical parameters can often capture the "essence" of a system as well as a neural network with thousands of weights.
The "Modeller Effect": There was massive performance variance within the Deep Learning group. This suggests that "off-the-shelf" AI isn't enough; the expert's choices in data preprocessing and hyperparameter tuning are decisive.
AI Can Handle Extremes: A common myth is that AI cannot predict values outside its training range. This challenge proved otherwise: several AI models successfully simulated extreme lows and highs that hadn't been seen in the training data.
Uncertainty is the Weakest Link: Most teams struggled to provide reliable 95% prediction intervals. Many intervals were far too narrow, meaning the models were "overconfident" in their wrong predictions.
Key Results
General Performance: Most models were quite accurate. The average NSE across all wells in the validation period was 0.64, though it was higher (0.72) if the tricky Swedish site was excluded.
No Single Winner: No single team or method performed best across all four wells. This highlights that the "best" model depends heavily on the specific site.
The River Factor: In the USA well, which was influenced by a nearby river, Deep Learning models significantly outperformed the simpler models because they were better at integrating the complex, non-linear relationship between river stage and groundwater.
Efficiency: Lumped-parameter models were developed in hours, whereas some Deep Learning models took days to calibrate, raising questions about the return on investment for high-complexity models in routine tasks.
Limitations and Future Work
The authors acknowledge that four wells are a small sample size. The sites were all in relatively "easy" temperate or subarctic climates with consistent data.
Future challenges should focus on:
* Arid and Semi-Arid Zones: Where groundwater response to rain is much more sporadic and harder to model.
* Data Gaps: Testing how models handle "messy" real-world data with missing months or years.
* Drought Prediction: Specifically testing the ability to forecast multi-year groundwater depletion during extended dry spells.
Summary updated
Abstract
This paper presents the results of the 2022 Groundwater Time Series Modelling Challenge, where 15 teams from different institutes applied various data-driven models to simulate hydraulic-head time series at four monitoring wells. Three of the wells were located in Europe and one was located in the USA in different hydrogeological settings in temperate, continental, or subarctic climates. Participants were provided with approximately 15 years of measured heads at (almost) regular time intervals and daily measurements of weather data starting some 10 years prior to the first head measurements and extending around 5 years after the last head measurement. The participants were asked to simulate the measured heads (the calibration period), to provide a prediction for around 5 years after the last measurement (the validation period for which weather data were provided but not head measurements), and to include an uncertainty estimate. Three different groups of models were identified among the submissions: lumped-parameter models (three teams), machine learning models (four teams), and deep learning models (eight teams). Lumped-parameter models apply relatively simple response functions with few parameters, while the artificial intelligence models used models of varying complexity, generally with more parameters and more input, including input engineered from the provided data (e.g. multi-day averages). The models were evaluated on their performance in simulating the heads in the calibration period and in predicting the heads in the validation period. Different metrics were used to assess performance, including metrics for average relative fit, average absolute fit, fit of extreme (high or low) heads, and the coverage of the uncertainty interval. For all wells, reasonable performance was obtained by at least one team from each of the three groups. However, the performance was not consistent across submissions within each group, which implies that the application of each method to individual sites requires significant effort and experience. In particular, estimates of the uncertainty interval varied widely between teams, although some teams submitted confidence intervals rather than prediction intervals. There was not one team, let alone one method, that performed best for all wells and all performance metrics. Four of the main takeaways from the model comparison are as follows: (1) lumped-parameter models generally performed as well as artificial intelligence models, which means they capture the fundamental behaviour of the system with only a few parameters. (2) Artificial intelligence models were able to simulate extremes beyond the observed conditions, which is contrary to some persistent beliefs about these methods. (3) No overfitting was observed in any of the models, including in the models with many parameters, as performance in the validation period was generally only a bit lower than in the calibration period, which is evidence of appropriate application of the different models. (4) The presented simulations are the combined results of the applied method and the choices made by the modeller(s), which was especially visible in the performance range of the deep learning methods; underperformance does not necessarily reflect deficiencies of any of the models. In conclusion, the challenge was a successful initiative to compare different models and learn from each other. Future challenges are needed to investigate, for example, the performance of models in more variable climatic settings to simulate head series with significant gaps or to estimate the effect of drought periods.
BibTeX
@inproceedings{collenteur2024data,
author = {Raoul Collenteur and Ezra Haaf and Mark Bakker and Tanja Liesch and Andreas Wünsch and Jenny Soonthornrangsan and Jeremy T. White and Nick Martin and Rui Hugman and Ed de Sousa and Didier Vanden Berghe and Xinyang Fan and Tim Peterson and Jānis Bikše and Antoine Di Ciacca and Xinyue Wang and Yang Zheng and Maximilian Nölscher and Julian Koch and Raphael Schneider and Nikolas Benavides Höglund and Sivarama Krishna Reddy Chidepudi and Abel Henriot and Nicolas Massei and Abderrahim Jardani and Max Gustav Rudolph and Amir Rouhani and J. Jaime Gómez‐Hernández and Seifeddine Jomaa and Anna Pölz and Tim Franken and Morteza Behbooei and Jimmy Lin and Rojin Meysami},
title = {{Data-driven modelling of hydraulic-head time series: results and lessons learned from the 2022 Groundwater Time Series Modelling Challenge}},
booktitle = {Hydrology},
year = {2024},
doi = {10.5194/hess-28-5193-2024},
}