Water Quality Prediction Based on LSTM and Attention Mechanism: A Case Study of the Burnett River, Australia

Honglei Chen, Junbo Yang, Xiaohua Fu, Qingxing Zheng, Xinyu Song, Zeding Fu, Jiacheng Wang, Yingqi Liang, Hailong Yin, Zhiming Liu, Jie Jiang, He Wang, Xinxin YangSustainability 2022 · DOI

TL;DR

This paper introduces an AT-LSTM model, which enhances traditional LSTM networks with an attention mechanism, to predict dissolved oxygen (DO) levels in the Burnett River, Australia. The AT-LSTM model significantly improves prediction accuracy and generalization compared to a standard LSTM, offering a more robust tool for water quality management and early warning systems.

What Problem Does This Paper Address?

Accurate prediction of water quality is crucial for effective pollution control and environmental management. Traditional methods often struggle with the complex, nonlinear, and time-dependent nature of water quality data, which is influenced by numerous factors and exhibits long-term dependencies. Existing deep learning models like LSTMs can capture some of these temporal patterns but may overlook the varying importance of different historical data points. This research aims to address this by developing a more sophisticated prediction model that can dynamically weigh the influence of past data, leading to more accurate forecasts for critical water quality parameters like dissolved oxygen.

Methods

The core of this work is the development and evaluation of an Attention-based Long Short-Term Memory (AT-LSTM) model.

  • Long Short-Term Memory (LSTM): LSTMs are a type of recurrent neural network (RNN) specifically designed to handle sequential data and overcome the vanishing gradient problem common in traditional RNNs. They use gating mechanisms (input, forget, output gates) to selectively remember or forget information over long periods, making them suitable for time-series forecasting.
  • Attention Mechanism: The novel contribution is the integration of an attention mechanism into the LSTM architecture. This mechanism allows the model to dynamically assign different weights to different time steps in the input sequence. Essentially, it learns to focus on the most relevant historical data points when making a prediction, rather than treating all past information equally. This is achieved by calculating similarity scores between a "query" (representing the current prediction context) and "keys" (representing historical data points), which are then used to weight the "values" (the historical data itself) to produce an attention-weighted output.
  • Data Preprocessing: Before model training, several preprocessing steps were applied:
  • Data Cleaning: Outlier detection using box-line plots, with detected outliers treated as missing values.
  • Missing Value Imputation: Linear interpolation was used to fill in missing data points.
  • Feature Selection: Pearson correlation analysis was employed to identify key input features (pH, temperature, chlorophyll-a) that significantly correlate with the target variable (DO).
  • Normalization: Min-max normalization was applied to scale all input features to a range between 0 and 1, mitigating the impact of different data scales on model training.
  • Time Series to Supervised Data: A sliding window approach was used to convert the time-series data into a supervised learning format, where past data points form the input features and future points form the target.

Datasets

The study utilized historical water quality data collected from the Burnett River in southeastern Queensland, Australia. The dataset spans from January 2015 to January 2020, with data collected every half-hour. The key water quality parameters included were water temperature (Temp), pH, dissolved oxygen (DO), conductivity (EC), chlorophyll-a (Chl-a), and turbidity (NTU). Dissolved oxygen (DO) was specifically chosen as the target variable for prediction.

For model verification, a new independent dataset from the Gregory River was also used.

Key Insights and Ideas

  • Attention is Key for Temporal Relevance: The core insight is that not all historical data points are equally important for predicting future water quality. An attention mechanism can effectively learn which past moments are most influential, leading to more accurate and robust predictions.
  • Beyond Simple Averaging: Traditional LSTMs, while good at remembering sequences, can still be improved by explicitly learning to "pay attention" to specific parts of the sequence. This is analogous to how a human expert might focus on particular historical events when assessing a current situation.
  • DO as a Critical Indicator: The study highlights the importance of Dissolved Oxygen (DO) as a key indicator of river health and pollution, making its accurate prediction a valuable goal for environmental management.
  • Generalization Matters: The AT-LSTM model not only improved prediction accuracy on the training data but also demonstrated better generalization capabilities on unseen data, which is crucial for real-world applications.

Key Results

The AT-LSTM model consistently outperformed the standard LSTM model in predicting dissolved oxygen levels:

  • One-Step-Ahead Prediction:
    • The AT-LSTM model achieved a Root Mean Square Error (RMSE) of 0.130 and a coefficient of determination (R²) of 0.953.
  • The standard LSTM model had an RMSE of 0.171 and an R² of 0.918.
  • This represents a 23.9% reduction in RMSE and a 27.7% reduction in Mean Absolute Error (MAE) for the AT-LSTM model.
  • Multistep-Ahead Prediction (up to 48 hours):
    • Across various prediction horizons (4-48 hours), the AT-LSTM model consistently showed lower MAE and RMSE values and higher R² values compared to LSTM.
  • On average, the AT-LSTM model achieved a 14.6% decrease in MAE and a 12.2% decrease in RMSE compared to LSTM, with an average 10.8% increase in R².
  • Model Verification: On the independent Gregory River dataset, the AT-LSTM model consistently showed lower RMSE values than the LSTM model for prediction horizons of 1-12 hours, further validating its superior performance and generalization.

Limitations and Future Work

  • Single Monitoring Point: The current study focused on water quality data from a single monitoring point in the Burnett River. The authors suggest that incorporating data from multiple sites within the study area could further enhance prediction accuracy by increasing the dimensionality and amount of data.
  • Linear Feature Screening: The Pearson correlation test used for feature selection is a linear method. Future work could explore more advanced, nonlinear feature screening methods to identify a more comprehensive set of predictive indicators.
  • Further Optimization: The AT-LSTM model itself has room for further optimization. Future research could explore different hyperparameter tuning strategies and potentially integrate other advanced deep learning techniques.

Summary updated

Abstract

Prediction of water quality is a critical aspect of water pollution control and prevention. The trend of water quality can be predicted using historical data collected from water quality monitoring and management of water environment. The present study aims to develop a long short-term memory (LSTM) network and its attention-based (AT-LSTM) model to achieve the prediction of water quality in the Burnett River of Australia. The models developed in this study introduced an attention mechanism after feature extraction of water quality data in the section of Burnett River considering the effect of the sequences on the prediction results at different moments to enhance the influence of key features on the prediction results. This study provides one-step-ahead forecasting and multistep forward forecasting of dissolved oxygen (DO) of the Burnett River utilizing LSTM and AT-LSTM models and the comparison of the results. The research outcomes demonstrated that the inclusion of the attention mechanism improves the prediction performance of the LSTM model. Therefore, the AT-LSTM-based water quality forecasting model, developed in this study, demonstrated its stronger capability than the LSTM model for informing the Water Quality Improvement Plan of Queensland, Australia, to accurately predict water quality in the Burnett River.
BibTeX
@inproceedings{chen2022water,
  author = {Honglei Chen and Junbo Yang and Xiaohua Fu and Qingxing Zheng and Xinyu Song and Zeding Fu and Jiacheng Wang and Yingqi Liang and Hailong Yin and Zhiming Liu and Jie Jiang and He Wang and Xinxin Yang},
  title = {{Water Quality Prediction Based on LSTM and Attention Mechanism: A Case Study of the Burnett River, Australia}},
  booktitle = {Sustainability},
  year = {2022},
  doi = {10.3390/su142013231},
}
Figure