Automatic Remaining Useful Life Estimation Framework with Embedded Convolutional LSTM as the Backbone

Yexu Zhou, Michael Hefenbrock, Yiran Huang, Till Riedel, Michael Beigl

Introduction

Predictive maintenance is a cornerstone of modern industrial operations, with Remaining Useful Life (RUL) estimation serving as its most critical metric. Accurately predicting when a machine will fail allows for optimized maintenance schedules, reduced downtime, and enhanced safety. Most state-of-the-art approaches utilize multivariate sensor data processed through a sliding window mechanism.

However, traditional deep learning architectures face significant hurdles. Convolutional Neural Networks (CNNs) excel at extracting local features but often fail to capture long-term temporal dependencies. Conversely, standard Long Short-Term Memory (LSTM) networks, while designed for sequences, typically require the input window to be "flattened" into a one-dimensional vector. This flattening process is inherently destructive, as it discards the spatial relationships between different sensors and the fine-grained temporal structure within the window itself.

The structural penalty of flattening: standard LSTMs often destroy spatial and temporal relationships by converting 2D sensor windows into 1D vectors.

The Embedded Convolutional LSTM (ECLSTM)

To address these limitations, researchers have introduced the Embedded Convolutional LSTM (ECLSTM). Unlike hybrid models that stack CNN layers before an LSTM, the ECLSTM integrates 1D convolutions directly into the LSTM cell's internal gating mechanisms (input, forget, and output gates).

Structural Preservation and Efficiency

By replacing standard fully connected layers with convolutional ones, the ECLSTM treats the input as a multi-dimensional tensor rather than a flat vector. This allows the model's hidden states to preserve and "remember" the internal structure of the data window. Furthermore, the model's parameter count becomes tied to the convolutional kernel size rather than the total window length, preventing parameter explosion and improving computational efficiency.

Data Fusion Strategies

The framework explores three distinct methods for integrating data from multiple sensors:

  1. Early Fusion: Sensors are treated as multiple channels and processed jointly from the first layer.
  2. Late Fusion: Each sensor is processed by its own independent convolutional path before being merged.
  3. Hybrid Fusion: Sensors are processed separately, but the model shares weights across sensors to capture common degradation patterns while maintaining individual sensor identities.
Comparison of Early, Late, and Hybrid fusion strategies for integrating multivariate sensor data within the ECLSTM framework.

Automated Hyperparameter Optimization

Designing deep learning models for specific industrial tasks is often a manual, expert-driven process. To democratize this technology, the authors wrapped the ECLSTM in an automated framework using BOHB (Bayesian Optimization and Hyperband). This system automatically optimizes preprocessing steps, the number of ECLSTM layers, and the configuration of the prediction head, making the solution "plug-and-play" for various industrial datasets.

Experimental Results

The ECLSTM framework was rigorously tested against several industry-standard benchmarks:

  • C-MAPSS (NASA Turbofan): The model achieved state-of-the-art (SOTA) performance across all four sub-datasets. Notably, on the challenging FD002 set, it reduced the Root Mean Squared Error (RMSE) from 19.29 to 15.95.
  • PHM 2008: The framework outperformed the top four original competition winners, demonstrating its superior predictive power.
  • FEMTO-ST Bearing: The model showed robust performance on real-world accelerated degradation data.
  • UCI-HAR: To test generalization, the model was applied to human activity recognition, achieving 94.69% accuracy and proving its effectiveness for general time-series tasks.

Key Insights

A critical finding of the study is the "flattening penalty." Ablation studies showed that while standard LSTMs' performance degrades as window sizes increase (due to the loss of structure during flattening), the ECLSTM's performance actually improves. This suggests that the ECLSTM successfully leverages the richer information available in larger temporal windows. Additionally, the use of shared convolutional weights makes the model more resistant to overfitting compared to traditional dense LSTMs.

Conclusion and Future Work

The ECLSTM framework represents a significant step forward in RUL estimation, combining the temporal strengths of LSTMs with the spatial feature extraction of CNNs without the destructive effects of flattening. Future research directions include exploring bidirectional ECLSTMs to capture future-to-past dependencies and integrating attention mechanisms to highlight critical failure-signaling events within the sensor streams.

Summary updated

BibTeX
@inproceedings{zhou2020automatic,
  author = {Yexu Zhou and Michael Hefenbrock and Yiran Huang and Till Riedel and Michael Beigl},
  title = {{Automatic Remaining Useful Life Estimation Framework with Embedded Convolutional LSTM as the Backbone}},
  booktitle = {ECML/PKDD},
  year = {2020},
  url = {https://openreview.net/forum?id=nMONf0LpIV},
}
Figure