Attention-based Models for Snow-Water Equivalent Prediction

Krishu K. Thapa, Bhupinderjeet Singh, Supriya Savalkar, Alan Fern, Kirti Rajagopalan, Ananth Kalyanaraman

Abstract

Accurate monitoring of Snow-Water Equivalent (SWE) is vital for water resource management in the Western United States. Traditional interpolation methods often fail due to the sparse nature of sensor networks and the complex topography of mountainous regions. This paper presents a novel framework utilizing Transformer-based attention mechanisms to capture spatial and temporal dependencies in snowpack data. Our ensemble attention model demonstrates superior performance over baseline LSTM and linear regression models, achieving a Nash-Sutcliffe Efficiency (NSE) greater than 0.5 for over 82% of study locations.

Introduction

In the Western U.S., snowmelt provides the majority of water for agriculture, hydropower, and municipal use. Water managers rely on Snow-Water Equivalent (SWE) measurements—the amount of water released if the snowpack melted instantly—to make critical decisions regarding reservoir storage and drought mitigation. However, the SNOTEL (Snow Telemetry) network is sparse, with an average of only one station per 4,014 square kilometers. The chaotic influence of topography, elevation, and weather makes simple interpolation between these points inaccurate.

Map of SNOTEL locations in the Western USA and a time-series comparison of predicted vs. actual SWE at Albro Lake.

While machine learning models like LSTMs have been applied to this problem, they often struggle with long-range dependencies and the "big picture" of spatial correlations. This research explores the use of attention mechanisms to overcome these limitations by allowing models to identify relevant patterns across both space and time simultaneously.

Methods

The proposed framework leverages the Transformer architecture's attention mechanism. Unlike sequential models, attention allows the system to "look" at all parts of a dataset to find relevant patterns. Three variations were evaluated:

  1. Spatial Attention: Processes data from multiple locations on a single day to learn regional trends and topographic influences.
  2. Temporal Attention: Processes a sequence of days at a single location to capture how historical weather events impact current snow density.
  3. Ensemble Attention: A simple average of the spatial and temporal models, designed to leverage the complementary strengths of both.

The architecture employs a modified Transformer encoder with 16 attention heads and 24 feed-forward layers. A key innovation is location-wise concatenation, which ensures the model retains original raw input features like elevation and latitude throughout the processing pipeline.

Datasets

The study utilized 18 years of data (2002–2019) from 323 SNOTEL stations. Features included daily SWE, precipitation, and temperature, alongside static characteristics such as elevation, land cover, and "southness" (aspect/slope). Additionally, satellite-based passive microwave brightness temperatures (19GHz and 37GHz) were incorporated as indicators of snow properties.

Results

Model performance was primarily measured using Nash-Sutcliffe Efficiency (NSE), where 1.0 is a perfect prediction. The Ensemble Attention model emerged as the most robust, significantly outperforming traditional baselines.

Stacked bar chart showing the distribution of NSE scores across stations for various models.

Key findings include:

  • Superior Accuracy: The Ensemble model achieved NSE > 0.5 for 82.04% of locations.
  • Baseline Comparison: Both spatial (65.63%) and temporal (62.22%) attention models outperformed LSTM and Linear Regression baselines.
  • Complementary Strengths: Spatial attention performed better at lower elevations where regional trends dominate, while temporal attention excelled at higher altitudes where local historical patterns are more critical. The ensemble effectively canceled out individual model errors, leading to a tighter fit to ground truth.

Discussion and Conclusion

The success of attention-based models highlights the importance of non-linear, global context in hydrological modeling. By allowing the model to focus on relevant spatial or temporal "neighbors," we achieve a much more accurate representation of snowpack dynamics.

Future work will address the "SNOTEL bias" by validating performance in ungauged areas and exploring more sophisticated ensembling techniques. Transitioning from state estimation (imputation) to forecasting remains a primary goal for enhancing water management tools in the face of increasing climate variability.

Summary updated

BibTeX
@inproceedings{thapa2024attention,
  author = {Krishu K. Thapa and Bhupinderjeet Singh and Supriya Savalkar and Alan Fern and Kirti Rajagopalan and Ananth Kalyanaraman},
  title = {{Attention-based Models for Snow-Water Equivalent Prediction}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30337},
}
Figure