Remaining Useful Life Estimation for Engineered Systems Operating under Uncertainty with Causal GraphNets

Charilaos Mylonas, Eleni N. Chatzi

TL;DR

This paper introduces GNN-tCNN, a novel deep learning approach for predicting the Remaining Useful Life (RUL) of engineered systems, even with irregularly sampled data. By leveraging Graph Neural Networks (GNNs) and Temporal Convolutional Networks (TCNs), GNN-tCNN can effectively model complex degradation processes and represent prediction uncertainty, outperforming traditional methods on both simulated and real-world bearing datasets.

What Problem Does This Paper Address?

Predicting the Remaining Useful Life (RUL) of industrial equipment is crucial for proactive maintenance and preventing costly failures. However, real-world sensor data is often sparse, non-equidistant, and subject to noise, making accurate RUL estimation challenging. Traditional methods struggle with this irregular data, often requiring imputation or making strong assumptions about the degradation process. Furthermore, accounting for the inherent uncertainty in these predictions is vital for reliable decision-making. This paper tackles the problem of RUL estimation under these challenging conditions, aiming for a data-driven approach that can handle irregular time series and quantify uncertainty.

Methods

The core of this work is the proposed GNN-tCNN architecture. It combines Graph Neural Networks (GNNs) with Temporal Convolutional Networks (TCNs).

  • Graph Neural Networks (GNNs): GNNs are used to model the causal relationships between observations, even when they are not uniformly spaced in time. This allows the model to learn an implicit state evolution of the system rather than relying on explicit time-series models. The architecture uses a message-passing mechanism where nodes (representing observations) exchange information through edges (representing temporal relationships).
  • Temporal Convolutional Networks (TCNs): TCNs are employed as feature extractors within the GNN framework. They are particularly adept at capturing patterns in time-series data, even with irregular sampling, by processing segments of the data.
  • Causal Graph Structure: Instead of a strict sequential chain, the model uses an arbitrary causal graph, allowing for more flexible modeling of dependencies.
  • Uncertainty Quantification: The model represents RUL predictions as a Gamma distribution. This is achieved by leveraging advances in reparametrization gradients, allowing for the estimation of uncertainty in a principled way.
  • Comparison Model (LSTM-tCNN): For benchmarking, the authors also implement and compare against a Long Short-Term Memory (LSTM) network with TCN feature extractors, a common approach for handling sequential data and long-term dependencies.

Datasets

The proposed GNN-tCNN model and its comparison are evaluated on two datasets:

  • Simulated Degradation Dataset: A synthetic, non-stationary degradation process was simulated to verify the method's efficacy on a controlled case where the ground truth is known. This dataset possesses characteristics of a typical RUL problem, including non-equidistant observations and stochastic evolution.
  • PRONOSTIA Bearing Dataset: This is a real-world, run-to-failure dataset for ball bearings, collected under accelerated fatigue testing. It features 17 bearings loaded under different rotational and lateral conditions, with 2-axis acceleration measurements. This dataset is used to test the model's generalization capabilities on unseen experiments.

Key Insights and Ideas

  • Implicit State Evolution: The GNN-tCNN learns an implicit model of the system's degradation, which is more robust to irregular data than methods requiring explicit time-series imputation or models.
  • Causal Graph for Irregular Data: Using a causal graph structure, rather than a strict sequential one, allows the model to effectively incorporate the temporal ordering of observations without assuming equidistant sampling. This is a key advantage for real-world data.
  • Uncertainty as a Gamma Distribution: Representing RUL predictions as a Gamma distribution provides a natural way to quantify uncertainty, especially since RUL values are inherently positive. This is a more informative output than point estimates.
  • Complementarity of GNN and LSTM: The study reveals that GNN-tCNN and LSTM-tCNN perform well on different subsets of experiments. This suggests that gradual degradation might favor GNNs (better at long-term dependencies), while abrupt failures might favor LSTMs (better at recent observations). This complementarity hints at future hybrid architectures.
  • Feature Generalization: t-SNE visualizations show that the learned features from the TCNs generalize well from training to test sets, indicating the robustness of the feature extraction process.

Key Results

  • Simulated Dataset: Both GNN-tCNN and LSTM-tCNN performed comparably on the simulated dataset, demonstrating the competitiveness of the GNN-tCNN approach.
  • Bearing Dataset:
    • GNN-tCNN generally outperformed LSTM-tCNN in terms of Mean Absolute Percentage Error (MAPE), especially for longer experiments.
  • LSTM-tCNN showed better performance in Negative Log Likelihood (NLL) for shorter experiments.
  • The simple MLP baseline model performed poorly, highlighting the limitations of hand-engineered features and single-segment analysis.
  • Predictions from both deep learning models showed reduced uncertainty (narrower prediction intervals) as the RUL approached zero.
  • Computational Efficiency: The GNN-tCNN architecture is more easily parallelizable than RNN-based models, offering a computational advantage despite potentially higher memory costs.

Limitations and Future Work

  • Failure Mechanism Ambiguity: The PRONOSTIA dataset lacks explicit information on failure mechanisms. This makes it difficult to definitively confirm hypotheses about why GNN-tCNN performs better on gradual degradation and LSTM on abrupt failures. Future work could explore datasets with detailed failure mode information.
  • Hybrid Architectures: The observed complementarity between GNN-tCNN and LSTM-tCNN suggests that combining their strengths could lead to even more robust RUL prediction models.
  • Broader Applicability: The authors suggest that the non-sequential causal approach to handling long-term dependencies could be beneficial for other domains with irregularly sampled time series data, such as electronic health records.
  • Interpretability: While t-SNE visualizations offer insights, further work on interpreting the learned representations and failure modes would be valuable.

Summary updated

BibTeX
@inproceedings{mylonas2021remaining,
  author = {Charilaos Mylonas and Eleni N. Chatzi},
  title = {{Remaining Useful Life Estimation for Engineered Systems Operating under Uncertainty with Causal GraphNets}},
  booktitle = {Sensors},
  year = {2021},
  url = {https://openreview.net/forum?id=jaP1DYhAoU},
}
Figure