Attention-Based Models for Snow-Water Equivalent Prediction

Krishu K. Thapa, Bhupinderjeet Singh, Supriya Savalkar, Alan Fern, Kirti Rajagopalan, Ananth Kalyanaraman

Introduction

In the Western United States, snowmelt serves as the primary driver of streamflow, critically fueling agriculture, hydropower, and municipal water supplies. Effective management of these resources depends on accurate Snow-Water Equivalent (SWE) data—the measure of water content within a snowpack. Currently, this information is primarily gathered from the Snow Telemetry (SNOTEL) network. However, SNOTEL stations are geographically sparse, with an average density of only one station per 4,014 square kilometers.

Traditional methods for creating spatially-complete SWE maps rely on simple interpolation or basic machine learning models. These often fail to capture the complex, non-linear interactions between topography, weather, and time. To address this, researchers have adapted Transformer-based attention mechanisms to model SWE, treating geographic locations or time steps as sequences to better capture spatial and temporal correlations.

Methodology

The researchers developed three distinct attention-based approaches to predict SWE:

  1. Spatial Attention: Treats a set of SNOTEL locations on a given day as a sequence, learning which locations (e.g., those with similar elevations) are most relevant to one another.
  2. Temporal Attention: Treats a sequence of days at a single location as input, learning how past weather patterns like temperature and precipitation influence current snowpack.
  3. Ensemble Attention: A combined approach that averages the predictions of the spatial and temporal models.

The core architecture utilizes a modified Transformer encoder. A critical innovation is the use of Locationwise Concatenation. As shown in Figure 4, the model concatenates original input features with encoded attention features before the final output layers. This prevents the loss of specific local characteristics, such as elevation, while the model processes broader contextual information.

The Transformer-based architecture featuring Locationwise Concatenation to preserve local features.

Dataset and Features

The study utilized a robust dataset spanning 18 "water years" (2002–2019), focusing on 323 SNOTEL stations with high data integrity. The models incorporated static features such as elevation, land cover, and aspect, alongside dynamic features like daily precipitation, temperature, and satellite-based passive microwave brightness temperatures.

Results and Discussion

The models were evaluated using the Nash-Sutcliffe Efficiency (NSE) metric, where a value of 1.0 represents a perfect prediction and values above 0.5 are considered "good" in hydrological modeling.

The Ensemble Attention model demonstrated superior performance, achieving an NSE > 0.5 for 82.04% of locations. This significantly outperformed the spatial-only (65.63%) and temporal-only (62.22%) models, as well as traditional LSTM and Linear Regression baselines.

Comparison of model performance (NSE) across SNOTEL stations, highlighting the superiority of the Ensemble Attention model.

Key insights from the study include:

  • Complementary Errors: The spatial model tended to under-predict while the temporal model over-predicted. Ensembling these models allowed errors to partially cancel out, resulting in more stable predictions.
  • Elevation Sensitivity: Spatial Attention performed notably well in lower-elevation areas, which are typically harder to predict due to their sensitivity to minor weather fluctuations.
  • Non-Local Correlations: Unlike standard interpolation, the attention mechanism can identify correlations between distant stations that share similar physical characteristics, such as forest cover and elevation.

Conclusion and Future Directions

This research demonstrates that attention-based models can significantly improve SWE prediction accuracy over traditional recurrent networks and linear models. However, several steps remain for operational deployment. Future work will focus on transitioning from station-based imputation to generating high-resolution gridded maps for ungauged areas. Additionally, the researchers aim to expand the model's capabilities to include multi-day forecasting and explore Spatiotemporal Graph Neural Networks to explicitly model physical connections within watersheds.

Summary updated

Abstract

Snow Water-Equivalent (SWE) -- the amount of water available if snowpack is melted -- is a key decision variable used by water management agencies to make irrigation, flood control, power generation and drought management decisions. SWE values vary spatiotemporally -- affected by weather, topography and other environmental factors. While daily SWE can be measured by Snow Telemetry (SNOTEL) stations with requisite instrumentation, such stations are spatially sparse requiring interpolation techniques to create spatiotemporally complete data. While recent efforts have explored machine learning (ML) for SWE prediction, a number of recent ML advances have yet to be considered. The main contribution of this paper is to explore one such ML advance, attention mechanisms, for SWE prediction. Our hypothesis is that attention has a unique ability to capture and exploit correlations that may exist across locations or the temporal spectrum (or both). We present a generic attention-based modeling framework for SWE prediction and adapt it to capture spatial attention and temporal attention. Our experimental results on 323 SNOTEL stations in the Western U.S. demonstrate that our attention-based models outperform other machine learning approaches. We also provide key results highlighting the differences between spatial and temporal attention in this context and a roadmap toward deployment for generating spatially-complete SWE maps.
BibTeX
@inproceedings{thapa2024attention,
  author = {Krishu K. Thapa and Bhupinderjeet Singh and Supriya Savalkar and Alan Fern and Kirti Rajagopalan and Ananth Kalyanaraman},
  title = {{Attention-Based Models for Snow-Water Equivalent Prediction}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30337},
}
Figure