FM-GRU: A Time Series Prediction Method for Water Quality Based on seq2seq Framework

Jianlong Xu, Kun Wang, Che Lin, Lianghong Xiao, Xingshan Huang, Yufeng ZhangWater 2021 · DOI

TL;DR

This paper introduces FM-GRU, a novel time series prediction model for water quality that combines Factorization Machines (FM) with a seq2seq framework using GRUs. By integrating FM to handle sparse, high-dimensional data and a dual-attention mechanism to address long-range dependencies, FM-GRU achieves superior accuracy in water quality prediction compared to existing methods.

What Problem Does This Paper Address?

Accurate and efficient prediction of water quality is crucial for environmental management and industrial operations. However, existing methods struggle with several challenges inherent in water quality data:

  • High dimensionality: Numerous indicators (up to 109) require significant computational resources.
  • Inconsistent sampling: Different indicators are monitored at varying frequencies (hourly, daily, monthly), complicating data organization.
  • Complex correlations: Indicators are not independent and exhibit intricate, often non-linear, relationships that are hard to capture.
  • Long time spans: Water quality data often spans years, making it difficult for models to retain information from earlier periods.
  • Data sparsity and volatility: Real-world data can be sparse, contain missing values, and exhibit significant fluctuations.

Traditional models often fail to adequately address these issues, leading to reduced prediction accuracy.

Methods

The proposed FM-GRU model builds upon the sequence-to-sequence (seq2seq) framework, leveraging Gate Recurrent Units (GRUs) as both the encoder and decoder. The key innovations are:

  • Factorization Machines (FM) Integration: FM is incorporated to effectively extract high-dimensional feature interactions from sparse and high-dimensional water quality data. This addresses the limitations of traditional models in capturing complex correlations and handling missing data.
  • Dual-Attention Mechanism: To combat the "memory failure" and information loss common in deep learning models dealing with long time series, a dual-attention mechanism is added. This mechanism applies attention to both time steps and covariates, allowing the model to focus on the most relevant historical information and input features.
  • Improved GRU Architecture: The GRU model is adapted to incorporate the output of the FM module and attention mechanisms as inputs, enhancing its ability to process complex sequential data.

Datasets

The primary dataset used for training and evaluation is water quality reference index data from the Haimen Bay monitoring station in the Lianjiang River, Shantou City, China. This dataset spans from January 2019 to June 2020. For experiments, a subset of 1000 time points was used.

To test the model's generalization ability, a second dataset was created by swapping the roles of dissolved oxygen (target) and pH (covariate) from the original dataset.

Key Insights and Ideas

  • FM for Sparse Data: The paper highlights that Factorization Machines are particularly well-suited for handling sparse data and extracting complex feature interactions, a common issue in real-world datasets like water quality monitoring.
  • Dual-Attention for Long-Term Dependencies: The dual-attention mechanism is crucial for overcoming the limitations of standard seq2seq models in capturing long-range dependencies in time series data, which is vital for environmental monitoring over extended periods.
  • Synergy of FM and Seq2seq: The combination of FM's feature extraction capabilities with the seq2seq framework's sequence modeling power creates a synergistic effect, leading to improved predictive performance.
  • Generalization is Key: The successful application of the model to a modified dataset (swapping target and covariate roles) demonstrates its robustness and ability to generalize across different water quality prediction tasks.

Key Results

The FM-GRU model consistently outperformed seven other competing methods (including HA, ARIMA, LR, XGBoost, FFNN, FC-LSTM, and FC-GRU) across various error metrics (MAE, MSE, RMSE, NRMSE) on the water quality prediction task.

  • Accuracy Improvement: FM-GRU achieved significantly lower error rates. For instance, its MAE was 0.57, compared to the next best (XGBoost) at 1.2, and FC-LSTM/FC-GRU at 1.73/1.75. The NRMSE was also notably lower at 0.16.
  • Performance on Sparse Data: The paper demonstrates that FM-GRU's performance degrades less significantly than traditional deep learning models when faced with sparse data, attributing this to FM's inherent ability to handle such data.
  • Generalization: On a new dataset with swapped target and covariate roles, FM-GRU again showed superior performance, achieving an NRMSE of 0.4, significantly outperforming other models.

Limitations and Future Work

  • Parameter Sensitivity: The paper notes that the performance of the FM module is influenced by the hyperparameter K, and finding the optimal value is important. The optimal K (84) was determined through experimentation.
  • Computational Cost: While FM-GRU shows improved accuracy, the paper mentions that XGBoost, while performing well, often requires a long calculation time, which can be a limitation for real-time forecasting.
  • Future Directions:
    • Combine XGboost's strengths with FM-GRU to optimize overall implementation and improve operating efficiency.
  • Explore the integration of CNNs for feature extraction to further enhance the high-dimensional potential information capture ability of the FM module.

Summary updated

Abstract

Water quality prediction plays a crucial role in both enterprise management and government environmental management. However, due to the variety in water quality data, inconsistent frequency of data acquisition, inconsistency in data organization, and volatility and sparsity of data, predicting water quality accurately and efficiently has become a key problem. This paper presents a recurrent neural network water quality prediction method based on a sequence-to-sequence (seq2seq) framework. The gate recurrent unit (GRU) model is used as an encoder and decoder, and a factorization machine (FM) is integrated into the model to solve the problem of high sparsity and high dimensional feature interaction in the data, which was not addressed by the water quality prediction models in prior research. Moreover, due to the long period and timespan of water quality data, we add a dual attention mechanism to the seq2seq framework to address memory failures in deep learning. We conducted a series of experiments, and the results show that our proposed method is more accurate than several typical water quality prediction methods.
BibTeX
@inproceedings{xu2021fm,
  author = {Jianlong Xu and Kun Wang and Che Lin and Lianghong Xiao and Xingshan Huang and Yufeng Zhang},
  title = {{FM-GRU: A Time Series Prediction Method for Water Quality Based on seq2seq Framework}},
  booktitle = {Water},
  year = {2021},
  doi = {10.3390/w13081031},
}
Figure