Enhancing LightGBM for Industrial Fault Warning: An Innovative Hybrid Algorithm

Shuai Li, Nan Jin, Azadeh Dogani, Yang Yang, Ming Zhang, Xiangyun GuProcesses 2024 · DOI

TL;DR

This paper tackles the challenge of optimizing LightGBM for industrial fault warning by introducing a novel hybrid algorithm, SAOA, which combines Arithmetic Optimization Algorithm (AOA) with Simulated Annealing (SA) and new search strategies. This approach significantly improves LightGBM's hyperparameter tuning, leading to more accurate and efficient fault detection in industrial equipment, as demonstrated on a real-world water pump dataset.

What Problem Does This Paper Address?

Industrial equipment is critical for safety and production efficiency. While machine learning, particularly LightGBM, shows promise for fault warning, its practical application is hindered by a large number of complex hyperparameters that are difficult to tune. This leads to suboptimal performance, computational inefficiency, and reduced accuracy. The paper aims to overcome these limitations by developing a more effective method for optimizing LightGBM's hyperparameters specifically for industrial fault warning tasks.

Methods

The core of this paper is the development of a novel hybrid algorithm called SAOA (Simulated Annealing-based Arithmetic Optimization Algorithm). This algorithm integrates:

  • Arithmetic Optimization Algorithm (AOA): A metaheuristic algorithm known for its exploration and exploitation capabilities.
  • Simulated Annealing (SA): Incorporated to prevent premature convergence to local optima and enhance global search by allowing the retention of inferior solutions.
  • New Search Strategies: The authors introduce a chaotic mapping and dynamic reverse learning strategy for improved population initialization, aiming for better initial solutions and algorithm robustness. They also propose a modified Mathematical Optimizer Acceleration (MOA) function and a nonlinear inertia weight factor to better balance exploration and exploitation phases.

This SAOA algorithm is used to optimize the hyperparameters of LightGBM, with the Root Mean Square Error (RMSE) of the LightGBM's prediction on a test set serving as the fitness function. The optimized LightGBM model is then used for fault warning.

Datasets

The study utilizes a dataset collected from a real-world water pump in a power plant. This dataset consists of 1500 samples from eleven critical measurement points (e.g., bearing temperatures, vibrations, stator winding temperatures, inlet filter pressure drop) and one output measurement point (entrance flow). The dataset was partitioned into training (70%) and testing (30%) sets.

Key Insights and Ideas

  • Hybridization is Key: Combining metaheuristic algorithms like AOA and SA with machine learning models like LightGBM is a powerful strategy for tackling complex optimization problems in fault warning.
  • Beyond Standard AOA: The proposed SAOA algorithm addresses limitations of the standard AOA by incorporating chaotic mapping, reverse learning, and a modified MOA function, leading to more robust and effective hyperparameter optimization.
  • SA's Role in Escaping Local Optima: The inclusion of Simulated Annealing is crucial for preventing the optimization process from getting stuck in suboptimal solutions, which is a common issue with many metaheuristic algorithms.
  • Tailored Optimization for LightGBM: The paper emphasizes that optimizing LightGBM's specific hyperparameters is critical for its performance in fault warning, and their proposed method achieves this effectively.

Key Results

The proposed SAOA-LightGBM algorithm demonstrated superior performance compared to other state-of-the-art methods:

  • Accuracy: Achieved a fault warning accuracy rate of 90% on the water pump application.
  • RMSE Reduction: Decreased Root Mean Square Error (RMSE) by 7.14% compared to ISEO-BP, 17.84% compared to LSTM-CNN, and 13.16% compared to GWO-LightGBM.
  • R-Squared Improvement: Increased R-Squared value by 2.15% compared to ISEO-BP, 7.02% compared to LSTM-CNN, and 3.73% compared to GWO-LightGBM.
  • Fault Warning Success Rate: The method led in the success rate of water pump fault warnings, particularly for the high inlet filter pressure drop fault (F4).
  • Robustness: The 95% confidence interval analysis indicated the algorithm's stability and robustness.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Data Quality and Feature Selection: The algorithm's performance is dependent on data quality and feature selection, which requires further research in areas like data preprocessing and feature engineering.
  • Computational Accuracy Improvement: There is room for improving the computational accuracy of the hybrid algorithm, potentially by exploring more effective optimization strategies and integrating more hybrid methods.
  • Deep Learning Integration: Future work could explore combining deep learning techniques with LightGBM to further enhance the efficiency of fault early warning systems.
  • Broader Applicability: The SAOA optimization method could be applied to other machine learning models requiring extensive parameter tuning, and the fault warning system has potential applications in transportation, energy, and finance sectors.

Summary updated

Abstract

The reliable operation of industrial equipment is imperative for ensuring both safety and enhanced production efficiency. Machine learning technology, particularly the Light Gradient Boosting Machine (LightGBM), has emerged as a valuable tool for achieving effective fault warning in industrial settings. Despite its success, the practical application of LightGBM encounters challenges in diverse scenarios, primarily stemming from the multitude of parameters that are intricate and challenging to ascertain, thus constraining computational efficiency and accuracy. In response to these challenges, we propose a novel innovative hybrid algorithm that integrates an Arithmetic Optimization Algorithm (AOA), Simulated Annealing (SA), and new search strategies. This amalgamation is designed to optimize LightGBM hyperparameters more effectively. Subsequently, we seamlessly integrate this hybrid algorithm with LightGBM to formulate a sophisticated fault warning system. Validation through industrial case studies demonstrates that our proposed algorithm consistently outperforms advanced methods in both prediction accuracy and generalization ability. In a real-world water pump application, the algorithm we proposed achieved a fault warning accuracy rate of 90%. Compared to three advanced algorithms, namely, Improved Social Engineering Optimizer-Backpropagation Network (ISEO-BP), Long Short-Term Memory-Convolutional Neural Network (LSTM-CNN), and Grey Wolf Optimizer-Light Gradient Boosting Machine (GWO-LightGBM), its Root Mean Square Error (RMSE) decreased by 7.14%, 17.84%, and 13.16%, respectively. At the same time, its R-Squared value increased by 2.15%, 7.02%, and 3.73%, respectively. Lastly, the method we proposed also holds a leading position in the success rate of a water pump fault warning. This accomplishment provides robust support for the timely detection of issues, thereby mitigating the risk of production interruptions.
BibTeX
@inproceedings{li2024enhancing,
  author = {Shuai Li and Nan Jin and Azadeh Dogani and Yang Yang and Ming Zhang and Xiangyun Gu},
  title = {{Enhancing LightGBM for Industrial Fault Warning: An Innovative Hybrid Algorithm}},
  booktitle = {Processes},
  year = {2024},
  doi = {10.3390/pr12010221},
}
Figure