Transformer-based Drum-level Prediction in a Boiler Plant with Delayed Relations among Multivariates

Gang Su, Sun Yang, Zhishuai Li, Ziyue LiISGT EUROPE 2024 · DOI

TL;DR

This paper introduces a novel Transformer-based framework for predicting steam drum water levels in power plants, addressing the challenges of complex, delayed interdependencies between numerous variables. By incorporating causal relation analysis and optimal delay inference, the model significantly improves prediction accuracy, especially for longer horizons, leading to more stable and efficient boiler operations.

What Problem Does This Paper Address?

Maintaining a stable steam drum water level is crucial for the safe and efficient operation of power plants. However, this is a complex task due to the intricate, non-linear dynamics within boiler systems. These dynamics involve numerous variables that are not only interconnected but also exhibit significant time delays in their influence on the drum level. Traditional control methods often struggle to capture these complex relationships, leading to suboptimal performance and potential instability. Existing deep learning approaches, while promising, may not fully account for these intricate causal links and temporal delays, hindering their ability to provide highly accurate and robust predictions for proactive control.

Methods

The authors propose a multi-stage framework to tackle the prediction challenge:

  1. Data Preprocessing: Standard steps like cleaning, smoothing, and standardization are applied to the input data. LayerNorm is used for stable gradient propagation.
  2. Causal Relation Analysis: This is a key innovation. It combines domain knowledge with Granger Causality to identify variables that have a direct causal link to the drum level, filtering out indirect influences. This is based on the principle of conditional independence in causal graphs.
  3. Optimal Time Delay Selection: For each identified causal variable, the optimal time delay at which its influence is most strongly felt on the drum level is determined. This is achieved by maximizing the covariance between the lagged variable and the drum level.
  4. Variable Augmentation: To leverage the identified delays, both the original variable and its time-shifted (delayed) version are included as features for the prediction model. This explicitly provides the model with historical context relevant to the current state.
  5. Transformer-based Prediction: A Transformer architecture is employed as the core prediction model. Its self-attention mechanism is well-suited for capturing long-range dependencies in sequential data, making it ideal for handling the complex temporal dynamics of boiler systems. The model architecture includes input embedding, positional encoding, self-attention, and feedforward networks.

Datasets

The study utilizes a real-world dataset from a steam drum water level control system. This dataset comprises 63 variables related to the steam drum, augmented with 9 key variables covering feedwater, main steam, coal consumption, generator power, and desuperheating water. The data spans from January 2nd, 2017, to January 14th, 2017, containing over 1 million data points. No new benchmarks were introduced; the focus was on applying the proposed framework to this existing industrial dataset.

Key Insights and Ideas

  • Causality is King: Simply feeding all available variables into a model isn't enough. Identifying direct causal relationships is crucial for building an effective predictive model, reducing noise and focusing on relevant signals.
  • Delays Matter (A Lot): The influence of many variables on the drum level isn't immediate. Accurately identifying and incorporating these time delays is essential for accurate prediction, especially as prediction horizons increase.
  • Transformer's Long-Range Power: The Transformer's self-attention mechanism shines when dealing with longer prediction horizons, outperforming traditional methods like LSTM by effectively capturing distant temporal dependencies.
  • Augmentation for Context: Explicitly providing the model with both current and delayed versions of causal variables significantly boosts predictive performance.

Key Results

The proposed Transformer-based framework with causal relation analysis and delay inference demonstrated superior performance compared to baseline methods and LSTM, particularly for longer prediction horizons.

  • Accuracy Improvement: While Transformer models (with or without delay consideration) were comparable to LSTM for short-term predictions (1 step), they showed significant advantages as the prediction horizon extended. For instance, for a 60-second prediction horizon, the Transformer (+ delay) achieved a Mean Absolute Error (MAE) of 0.6710, compared to 3.34 for the baseline and 2.63 for Transformer (- delay).
  • Stability: The model achieved a mean prediction error close to zero, with a two-sigma range of approximately 5mm, indicating high stability.
  • Effectiveness of Delay: Considering causal relationships and delays consistently improved prediction accuracy for both LSTM and Transformer models, highlighting the importance of these steps.

Limitations and Future Work

  • Domain Knowledge Reliance: The causal relation analysis relies partly on domain knowledge. While this is beneficial, it might limit generalizability to systems where such expert knowledge is scarce.
  • Computational Cost: While Transformers are powerful, their computational demands can be significant, especially with very large datasets or complex architectures.
  • Future Work: The authors suggest integrating their predictive framework with PID controllers in a cascade control configuration for enhanced drum level control. Further exploration into more sophisticated causal discovery methods and adaptive delay inference could also be valuable.

Summary updated

BibTeX
@inproceedings{su2024transformer,
  author = {Gang Su and Sun Yang and Zhishuai Li and Ziyue Li},
  title = {{Transformer-based Drum-level Prediction in a Boiler Plant with Delayed Relations among Multivariates}},
  booktitle = {ISGT EUROPE},
  year = {2024},
  doi = {10.1109/isgteurope62998.2024.10863020},
}
Figure