LS-SVM based spectral clustering and regression for predicting maintenance of industrial machines

Rocco Langone, Carlos Alzate, Bart De Ketelaere, Jonas Vlasselaer, Wannes Meert, Johan A. K. SuykensEngineering Applications of AI 2015 · DOI

TL;DR

This paper introduces a novel approach to predictive maintenance for industrial machines by combining unsupervised spectral clustering with supervised regression using Least Squares Support Vector Machines (LS-SVMs). This dual approach effectively predicts machine degradation and maintenance needs, offering a more cost-efficient and reliable solution compared to traditional methods.

What Problem Does This Paper Address?

Modern industrial machines generate vast amounts of sensor data, making it crucial to predict potential failures to avoid costly downtime, ensure safety, and optimize operations. Traditional maintenance strategies like corrective (fix after failure) and preventive (replace at fixed intervals) are often inefficient. Predictive maintenance aims to forecast failures, but accurately modeling complex degradation processes from sensor data remains a challenge. Existing methods often struggle with the imbalance of maintenance events (rare) versus normal operation (frequent) and can be expensive to implement. This paper tackles these issues by proposing a flexible and cost-effective LS-SVM framework for predictive maintenance.

Methods

The authors propose a two-pronged approach leveraging LS-SVMs:

  • Unsupervised Learning (Kernel Spectral Clustering - KSC): This method uses vibration sensor data to identify different operating states of the machine. KSC, formulated within the LS-SVM framework, is chosen for its ability to handle imbalanced data and its systematic model selection scheme. It effectively clusters data into "normal" and "maintenance" states, inferring degradation without explicit labels. A "soft clustering" output is also provided, offering a probability of maintenance that is more interpretable than a binary classification.
  • Supervised Learning (Nonlinear Auto-Regressive - NAR model): This method uses thermal camera data, which directly measures degradation (e.g., hot spots), to predict the future trend of this degradation. The NAR model is also cast within the LS-SVM framework, allowing for accurate time-series prediction. This approach is more direct but also more expensive due to the cost of thermal cameras.

Both methods benefit from the LS-SVM's out-of-sample extension capability, allowing for online application after offline training.

Datasets

The study utilizes data collected from a Vertical Form Fill and Seal (VFFS) machine. Three distinct datasets were used:

  • DS I: Collected using accelerometers, consisting of 771 events (processed bags) and 3 maintenance actions.
  • DS II: Also uses accelerometer data but with a different acquisition system, resulting in 11,632 events and 15 maintenance actions.
  • DS III: Combines accelerometer data with thermal camera data, capturing 3,519 events and 11 maintenance actions.

These datasets represent real-world industrial scenarios with imbalanced maintenance events.

Key Insights and Ideas

  • Dual Approach for Flexibility: The paper demonstrates that combining unsupervised clustering (for cost-effectiveness using vibration sensors) with supervised regression (for direct measurement using thermal cameras) offers a powerful and adaptable solution for predictive maintenance.
  • LS-SVMs Excel in Industrial Settings: The LS-SVM framework proves highly effective for both clustering and time-series prediction in complex industrial environments, outperforming traditional methods.
  • Soft Clustering for Interpretability: The soft clustering output from KSC provides a "probability to maintenance," offering a more nuanced and interpretable indicator of degradation than hard clustering.
  • Cost-Benefit Trade-off: The authors clearly articulate the trade-off between the cost of sensors (accelerometers vs. thermal cameras) and the reliability of the prediction, allowing companies to choose the most suitable option.
  • Unexpected Correlation: In DS III, the KSC model, trained on vibration data, surprisingly mirrored the degradation trend measured by the thermal camera, suggesting that vibration patterns can indeed capture degradation even when not explicitly trained to do so.

Key Results

  • KSC Performance: KSC effectively identified two operating regimes (normal and maintenance) and predicted maintenance actions in advance for all three datasets. In DS II, KSC outperformed k-means, which incorrectly suggested maintenance in non-maintenance regions.
  • NAR Performance: The LS-SVM NAR model demonstrated strong performance in predicting thermal camera data (hot area pixels) up to 10 steps ahead (approximately 25 seconds), with forecasts remaining good up to 1 minute ahead. It outperformed linear autoregressive models and extrapolation methods.
  • Model Selection: The Balanced Line Fit (BLF) criterion was successfully used for model selection in KSC, and cross-validation with Coupled Simulated Annealing (CSA) was effective for tuning NAR parameters.

Limitations and Future Work

  • KSC Probability Definition: The paper notes that the "probability to maintenance" from soft clustering is a subjective probability derived from the latent variable space, not a statistical probability based on large sample analysis.
  • Cost of Thermal Cameras: The more reliable thermal camera approach is acknowledged as being more expensive.
  • Future Work: The paper implicitly suggests that further exploration of advanced clustering methods (like HMMs) that naturally provide probabilistic outputs could be beneficial. The authors also highlight the potential for companies to choose between the cost-effective accelerometer-based KSC approach or the more reliable but expensive thermal camera-based NAR approach for optimizing maintenance strategies.

Summary updated

BibTeX
@article{langone2015ls,
  author = {Rocco Langone and Carlos Alzate and Bart De Ketelaere and Jonas Vlasselaer and Wannes Meert and Johan A. K. Suykens},
  title = {{LS-SVM based spectral clustering and regression for predicting maintenance of industrial machines}},
  journal = {Engineering Applications of AI},
  year = {2015},
  doi = {10.1016/j.engappai.2014.09.008},
}
Figure