Transformer Based Water Level Prediction in Poyang Lake, China

Jiaxing Xu, Hongxiang Fan, Minghan Luo, Piji Li, Taeseop Jeong, Ligang XuWater 2023 · DOI

TL;DR

This paper introduces the Transformer neural network, a powerful deep learning architecture, for predicting water levels in Poyang Lake, China. The study demonstrates that the Transformer model significantly outperforms traditional methods like LSTM, offering highly accurate predictions and providing valuable insights into the influence of the Yangtze River on lake dynamics.

What Problem Does This Paper Address?

Accurately predicting lake water levels is crucial for managing water resources, mitigating flood risks, and understanding ecosystem health. Poyang Lake, China's largest freshwater lake, experiences dramatic water level fluctuations due to its unique connection with the Yangtze River and seasonal precipitation. These fluctuations are complex and non-linear, making traditional forecasting methods challenging. While deep learning models have shown promise, the application of attention-based models like Transformers for lake water level prediction remains largely unexplored. This paper aims to fill this gap by evaluating the Transformer model's performance and its ability to capture the intricate hydrological processes influencing Poyang Lake.

Methods

The core of this research is the application of the Transformer neural network, specifically its encoder component, for time-series prediction. Unlike recurrent neural networks (RNNs) like LSTM that process data sequentially, the Transformer utilizes a self-attention mechanism. This allows the model to weigh the importance of different parts of the input sequence simultaneously, enabling it to capture long-range dependencies more effectively and efficiently. The model was trained using historical meteorological data, river inflows from five major rivers, and the water level of the Yangtze River at Hukou station. Two scenarios were tested: one excluding the Yangtze River's influence (S1) and another including it (S2). The model's performance was evaluated using standard metrics like Root Mean Square Error (RMSE), Coefficient of Determination (R²), and Nash-Sutcliffe Efficiency (NSE).

Datasets

The study utilized daily data for Poyang Lake and its catchment from 1960 to 2013.

  • Meteorological Data: Daily averages of temperature, precipitation, solar radiation, relative humidity, and wind speed from 14 national meteorological stations.
  • Catchment Inflow Data: Daily discharge observations from five rivers (Ganjiang, Fuhe, Xinjiang, Raohe, and Xiushui).
  • Yangtze River Water Level Data: Daily observations at Hukou station.
  • Poyang Lake Water Level Data: Daily observations from four gauging stations: Xingzi, Duchang, Tangyin, and Kangshan.

The training period was from 1960 to 2000, and the testing period was from 2001 to 2013. No new benchmark datasets were introduced; the study leverages existing long-term hydrological and meteorological records for the Poyang Lake region.

Key Insights and Ideas

  • Transformer's Superiority: The Transformer model significantly outperforms LSTM in predicting Poyang Lake's water levels, achieving much lower RMSE and higher NSE values. This highlights the advantage of the self-attention mechanism in capturing complex temporal dependencies in hydrological data.
  • Yangtze River's Dominance: The influence of the Yangtze River is a critical factor in Poyang Lake's water level fluctuations, particularly during flood and receding periods. Including Yangtze River data in the model (Scenario 2) dramatically improved prediction accuracy, underscoring its importance for effective forecasting.
  • Hyperparameter Sensitivity: The model's performance is highly sensitive to hyperparameter choices. The study identified a window size of 90 days and 6 model layers as optimal for Poyang Lake, suggesting that careful tuning is essential for maximizing accuracy and avoiding overfitting.
  • Lead Time Matters: While the Transformer model maintains high accuracy for short lead times (1-7 days), prediction accuracy degrades significantly for longer lead times (15-30 days). This is a common challenge in hydrological forecasting and emphasizes the need for realistic expectations regarding prediction horizons.

Key Results

  • High Accuracy: In the testing stage, the Transformer model achieved RMSE values between 0.26–0.70 m and NSE values higher than 0.94 across the four gauging stations when considering the Yangtze River's influence.
  • Yangtze River Contribution: The Yangtze River's contribution to improved RMSE and NSE was substantial, exceeding 80% for RMSE and 270% for NSE in flood and receding periods.
  • Optimal Hyperparameters: A window size of 90 days and 6 model layers were found to be the most suitable hyperparameters for Poyang Lake water level prediction.
  • Lead Time Performance: For lead times of 1-7 days, RMSE ranged from 0.46–0.73 m. However, RMSE increased to 1.37 m and 1.82 m for lead times of 15 and 30 days, respectively.
  • LSTM Comparison: The Transformer model consistently outperformed LSTM, with training stage RMSE for LSTM being higher than 0.76 m compared to the Transformer's RMSE below 0.16 m.

Limitations and Future Work

  • Lead Time Limitations: The study acknowledges that prediction accuracy decreases significantly for lead times beyond 7 days. This limits the model's utility for long-term flood forecasting or management decisions requiring extended foresight.
  • Hyperparameter Optimization: While optimal hyperparameters were identified for Poyang Lake, further research could explore more automated and comprehensive hyperparameter tuning strategies.
  • Generalizability: The study suggests that the Transformer model could be applied to other hydrological processes (e.g., lake temperature modeling) and other lakes. Future work should focus on testing its performance and generalizability across diverse hydrological systems and forecasting tasks.
  • Data Requirements: The paper notes that deep learning models, including the Transformer, benefit from large training datasets and short temporal scales. Further investigation into the "breakeven point" where Transformers surpass traditional models with varying data availability would be valuable.
  • Influence of TGD: The paper highlights the significant impact of the Three Gorges Dam (TGD) on the Yangtze River and, consequently, on Poyang Lake's water levels. Future models could explicitly incorporate TGD operational data for even more refined predictions.

Summary updated

Abstract

Water level is an important indicator of lake hydrology characteristics, and its fluctuation significantly affects lake ecosystems. In recent years, deep learning models have shown their superiority in the long-time range prediction of hydrology processes, while the application of deep learning models with the attention mechanism for lake water level prediction is very rare. In this paper, taking Poyang Lake as a case study, the transformer neural network model is applied to examine the model performance in lake water level prediction, to explore the effects of the Yangtze River on lake water level fluctuations, and to analyze the influence of hyper-parameters (window size and model layers) and lead time on the model accuracy. The result indicated that the transformer model performs well in simulating the lake water level variations and can reflect the temporal water level variation characteristics in Poyang Lake. In the testing stage, the RMSE values were recorded in the range of 0.26–0.70 m, and the NSE values are higher than 0.94. Moreover, the Yangtze River inflow has a great influence on the lake water level fluctuation of Poyang Lake, especially in flood and receding periods. The contribution rate of the Yangtze River in RMSE and NSE is higher than 80% and 270%, respectively. Additionally, hyper-parameters, such as window size and model layers, significantly influence the transformer model simulation accuracy. In this study, a window size of 90 d and a model layer of 6 are the most suitable hyper-parameters for water level prediction in Poyang Lake. Additionally, lead time may affect the model accuracy in lake water level prediction. With the lead time varied from one to seven days, the model accuracy was high and RMSE values were in the range of 0.46–0.73 m, while the RMSE value increased to 1.37 m and 1.82 m with the lead time of 15 and 30 days, respectively. The transformer neural network model constructed in this paper was the first to be applied to lake water forecasting and showed high efficiency in Poyang Lake. However, few studies have tried to use transformer model coupling with the attention mechanism for forecasting hydrological processes. It is suggested that the model can be used for long sequence time-series forecasting in hydrological processes in other lakes to test its performance, providing further scientific evidence for the control of lake floods and management of lake resources.
BibTeX
@inproceedings{xu2023transformer,
  author = {Jiaxing Xu and Hongxiang Fan and Minghan Luo and Piji Li and Taeseop Jeong and Ligang Xu},
  title = {{Transformer Based Water Level Prediction in Poyang Lake, China}},
  booktitle = {Water},
  year = {2023},
  doi = {10.3390/w15030576},
}
Figure