CHP Engine Anomaly Detection Based on Parallel CNN-LSTM with Residual Blocks and Attention

Won Hee Chung, Yeong Hyeon Gu, Seong Joon YooSensors 2023 · DOI

TL;DR

This paper introduces PCLRA, a novel parallel CNN-LSTM model enhanced with residual blocks and an attention mechanism, for detecting anomalies in Combined Heat and Power (CHP) engines. PCLRA significantly outperforms existing models by effectively capturing complex spatiotemporal patterns in sensor data, leading to improved engine safety and efficiency.

What Problem Does This Paper Address?

Combined Heat and Power (CHP) engines operate under demanding conditions, making them prone to anomalies and potential failures. Detecting these anomalies early is crucial for maintaining operational efficiency, preventing costly breakdowns, and ensuring safety. Traditional rule-based anomaly detection methods are often rigid and difficult to develop. While data-driven machine learning and deep learning approaches show promise, existing hybrid models often struggle to fully capture the intricate spatiotemporal dynamics inherent in multivariate sensor data from these complex systems. This paper aims to bridge this gap by developing a more robust and effective deep learning model for CHP engine anomaly detection.

Methods

The core of this research is the proposed Parallel CNN-LSTM Residual Blocks Attention (PCLRA) model. Unlike serial CNN-LSTM architectures where CNN features feed into LSTM, PCLRA processes spatial (CNN) and temporal (LSTM) features in parallel, allowing for more direct extraction of spatiotemporal information from the raw sensor data.

Key components of PCLRA include:

  • Parallel CNN-LSTM: Both CNN and LSTM networks receive the raw input data simultaneously. This allows them to independently learn spatial and temporal features, respectively, and then combine them for a more comprehensive understanding.
  • Residual Blocks: These are incorporated into both the CNN and LSTM components to combat the vanishing gradient problem common in deep neural networks. Residual connections allow gradients to flow more easily through the network, preserving information and improving training.
  • Attention Mechanism: After the parallel CNN and LSTM extract features, an attention mechanism is applied. This allows the model to focus on the most relevant spatiotemporal features for anomaly detection, further enhancing its accuracy and interpretability.
  • Bayesian Optimization: To find the optimal hyperparameters for all models, including PCLRA, Bayesian optimization was employed. This systematic approach helps in tuning parameters like learning rate, batch size, and network architecture to maximize performance.

Datasets

The study utilizes real-world sensor log data collected from three CHP engines at a power plant in South Korea. This data represents multivariate time series, capturing various operational parameters over a specific period. The data was divided into training, validation, and testing sets with a ratio of 6:2:2. The paper does not introduce a new benchmark dataset but uses existing operational data from CHP engines.

Key Insights and Ideas

  • Parallelism is Key: The research strongly suggests that a parallel architecture for CNN and LSTM is superior to serial combinations for extracting spatiotemporal features from multivariate time series data. This allows for a more direct and effective learning process.
  • Residual Blocks are Crucial for Information Preservation: The inclusion of residual blocks significantly helps in mitigating information loss due to the vanishing gradient problem, leading to better performance, especially in complex deep learning architectures.
  • Attention Focuses on What Matters: The attention mechanism acts as a powerful filter, enabling the model to concentrate on the most discriminative features for anomaly detection, thereby improving precision and recall.
  • Hybridization is More Than the Sum of Its Parts: Combining CNN, LSTM, residual blocks, and attention mechanisms in a well-designed architecture (PCLRA) yields performance far exceeding that of individual components or simpler hybrid models.

Key Results

The PCLRA model demonstrated superior performance across various metrics compared to nine other baseline models (including CNN, LSTM, serial CNN-LSTM variants, and parallel CNN-LSTM variants).

  • Macro F1 Score: PCLRA achieved a mean macro F1 score of 0.951 ± 0.033, which was the highest among all models.
  • Anomaly F1 Score: The model also secured the highest mean anomaly F1 score of 0.903 ± 0.064.
  • Accuracy: PCLRA reached an impressive accuracy of 0.999 ± 0.002.

These results indicate that PCLRA is highly effective in identifying both normal and anomalous operational states in CHP engines. The model consistently outperformed others across different engines and operational parts.

Limitations and Future Work

  • Generalizability: While PCLRA performed exceptionally well on the collected dataset, further validation on diverse CHP engine types and operating conditions would be beneficial to confirm its generalizability.
  • Hyperparameter Sensitivity: Although Bayesian optimization was used, the paper notes a need for further research into how various hyperparameters affect performance and how to optimize the search process more efficiently.
  • Anomaly Type Granularity: The current anomaly detection is binary (normal/anomaly). Future work could involve classifying specific types of anomalies for more detailed diagnostics.
  • Long-Term Data Collection: The authors plan to expand their research by collecting data over longer periods and further subdividing anomaly types to enhance the model's capabilities.

Summary updated

Abstract

The extreme operating environment of the combined heat and power (CHP) engine is likely to cause anomalies and defects, which can lead to engine failure; thus, detecting engine anomalies is essential. In this study, we propose a parallel convolutional neural network-long short-term memory (CNN-LSTM) residual blocks attention (PCLRA) anomaly detection model with engine sensor data. To our knowledge, this is the first time that parallel CNN-LSTM-based networks have been used in the field of CHP engine anomaly detection. In PCLRA, spatiotemporal features are extracted via CNN-LSTM in parallel and the information loss is compensated using the residual blocks and attention mechanism. The performance of PCLRA is compared with various hybrid models for 15 cases. First, the performances of serial and parallel models are compared. In addition, we evaluated the contributions of the residual blocks and attention mechanism to the performance of the CNN-LSTM hybrid model. The results indicate that PCLRA achieves the best performance, with a macro f1 score (mean ± standard deviation) of 0.951 ± 0.033, an anomaly f1 score of 0.903 ± 0.064, and an accuracy of 0.999 ± 0.002. We expect that the energy efficiency and safety of CHP engines can be improved by applying the PCLRA anomaly detection model.
BibTeX
@inproceedings{chung2023chp,
  author = {Won Hee Chung and Yeong Hyeon Gu and Seong Joon Yoo},
  title = {{CHP Engine Anomaly Detection Based on Parallel CNN-LSTM with Residual Blocks and Attention}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23218746},
}
Figure