A Dual-Attention Recurrent Neural Network Method for Deep Cone Thickener Underflow Concentration Prediction

Zhaolin Yuan, Jinlong Hu, Di Wu, Xiaojuan Ban

TL;DR

This paper introduces a novel Dual-Attention Recurrent Neural Network (DARNN) method to accurately predict the underflow concentration of deep cone thickeners, a critical parameter in industrial mining. By integrating spatial and temporal attention mechanisms with domain-specific feature engineering, their model significantly outperforms existing methods, offering improved accuracy and efficiency for process control.

What Problem Does This Paper Address?

The underflow concentration of deep cone thickeners is a crucial indicator of the efficiency and stability of industrial mining processes. Maintaining this concentration within a stable range is vital for preventing issues like pipe plugging and ensuring the quality of backfill materials. However, predicting this concentration is challenging due to several factors:

  • Long Time Delays: Changes in input parameters don't immediately affect the underflow concentration; the impact is often delayed and can vary over time.
  • Complex Spatial Correlations: Multiple sensors monitor various parameters, and their interactions influencing the underflow concentration are intricate and not always intuitively understood.
  • Noisy Sensor Data: Industrial environments are prone to sensor noise and external disturbances, which can degrade the quality of collected data.

Traditional methods, including mathematical models and simpler data-driven approaches, struggle to effectively capture these complex dynamics and noisy, high-dimensional sensor data. This paper aims to bridge this gap by developing a more robust and accurate prediction model.

Methods

The core of this paper is the proposed Dual-Attention Recurrent Neural Network (DARNN) method. It builds upon the encoder-decoder architecture commonly used in sequence-to-sequence tasks but enhances it with two key innovations:

  • Dual-Stage Attention Mechanism:
    • Spatial Attention (Encoder): This mechanism adaptively assigns weights to different input sensor features at each time step, allowing the model to focus on the most relevant spatial information from the high-dimensional sensor data.
  • Temporal Attention (Decoder): This mechanism dynamically weighs the importance of encoded hidden states from the encoder, enabling the model to capture long-term temporal dependencies and focus on crucial past information.
  • Feature Enhancement: The authors incorporate domain knowledge about thickener systems to create six additional features beyond the raw sensor data. These features are derived from physical principles like mass balance and density-concentration relationships, aiming to provide the model with richer, more informative inputs.

The model consists of an encoder that processes the input sequences (sensor data and engineered features), a decoder that generates the predicted underflow concentration, and a final fully connected neural network for output. The entire model is trained end-to-end using backpropagation.

Datasets

The study utilizes a real-world dataset collected from an Industrial Internet of Things (IIoT) platform installed on a Tailings Thickener from FLSmidth. The dataset comprises historical production data from May to June 2018 for training and September 2019 for testing. After preprocessing (outlier removal, service interval deletion, normalization), approximately 14,800 data points were obtained, with a sampling period of 2 minutes. Each data point includes eight original parameters, and six additional features were engineered, resulting in a dataset with 14 features per time step.

Key Insights and Ideas

  • Attention is Crucial for Complex Dynamics: The paper strongly suggests that both spatial and temporal attention mechanisms are vital for accurately modeling the delayed and interconnected nature of thickener processes. Simply using a standard RNN or LSTM without attention is insufficient.
  • Domain Knowledge Enhances Data-Driven Models: Engineering features based on physical principles significantly boosts prediction accuracy, demonstrating that combining data-driven learning with expert knowledge is a powerful strategy.
  • Dual Attention Outperforms Single Attention: The combination of spatial and temporal attention in DARNN yields superior results compared to models using only one type of attention, highlighting the benefit of capturing both feature relevance and temporal importance.
  • Global Residual Connection Aids Stability: The inclusion of a global residual connection helps the model learn small changes in concentration more effectively, which is important for a slow-moving process like thickening, and improves training stability.

Key Results

The DARNN method demonstrated superior performance across multiple evaluation metrics compared to baseline models like LightGBM, RNN, and LSTM.

  • MAE: DARNN achieved the lowest Mean Absolute Error (MAE) of 0.61 ± 0.03, significantly outperforming LightGBM (0.83), RNN (0.78 ± 0.03), and LSTM (0.75 ± 0.02).
  • RMSE: Similarly, DARNN achieved the lowest Root Mean Squared Error (RMSE) of 1.01 ± 0.03.
  • MAPE & RMLSE: DARNN also showed the best performance in Mean Absolute Percentage Error (MAPE) and Root Mean Squared Logarithmic Error (RMSLE).
  • Feature Enhancement Impact: Ablation studies showed that using enhanced features with LSTM or DARNN led to significantly higher prediction accuracy compared to using original features alone.
  • Attention Mechanism Impact: Further ablation experiments indicated that temporal attention improved performance over no attention, spatial attention improved over no attention, and the combination (dual attention) yielded the best results.

Limitations and Future Work

  • Computational Cost: The authors acknowledge that the numerous parameters and complex operations of their model, while effective, restrict its efficiency and make it less suitable for real-time applications. They suggest that a more lightweight network structure might be explored for future studies.
  • Interpretability: While the attention mechanisms offer some insight into feature importance, the paper notes that the pure deep neural network framework makes the model less interpretable and potentially harder to transfer to different thickener systems.
  • Generalizability: The model was tested on a specific industrial case. Future work could focus on improving its generalizability to different thickener configurations or industrial processes.

Summary updated

Abstract

This paper focuses on the time series prediction problem for underflow concentration of deep cone thickener. It is commonly used in the industrial sedimentation process. In this paper, we introduce a dual attention neural network method to model both spatial and temporal features of the data collected from multiple sensors in the thickener to predict underflow concentration. The concentration is the key factor for future mining process. This model includes encoder and decoder. Their function is to capture spatial and temporal importance separately from input data, and output more accurate prediction. We also consider the domain knowledge in modeling process. Several supplementary constructed features are examined to enhance the final prediction accuracy in addition to the raw data from sensors. To test the feasibility and efficiency of this method, we select an industrial case based on Industrial Internet of Things (IIoT). This Tailings Thickener is from FLSmidth with multiple sensors. The comparative results support this method has favorable prediction accuracy, which is more than 10% lower than other time series prediction models in some common error indices. We also try to interpret our method with additional ablation experiments for different features and attention mechanisms. By employing mean absolute error index to evaluate the models, experimental result reports that enhanced features and dual-attention modules reduce error of fitting ~5% and ~11%, respectively.
BibTeX
@article{yuan2020dual,
  author = {Zhaolin Yuan and Jinlong Hu and Di Wu and Xiaojuan Ban},
  title = {{A Dual-Attention Recurrent Neural Network Method for Deep Cone Thickener Underflow Concentration Prediction}},
  journal = {Sensors},
  year = {2020},
  doi = {10.3390/s20051260},
}
Figure