Monitoring of a Dynamic System Based on Autoencoders

Aomar Osmani, Massinissa Hamidi, Salah BouhoucheIJCAI 2019 · DOI

TL;DR

This paper introduces a novel two-level LSTM-based autoencoder architecture for continuous monitoring of industrial systems, specifically turbo-compressors. By combining a reconstruction model with a continuous learning mechanism and employing Bayesian optimization for hyperparameter tuning, the approach effectively detects anomalies and adapts to system evolution, promising improved predictive maintenance.

What Problem Does This Paper Address?

The increasing complexity of industrial infrastructure, driven by Industry 4.0, necessitates advanced condition monitoring solutions that can operate in real-time and adapt to continuous system changes. Traditional approaches, like model-based or data-driven methods, have limitations: model-based methods require explicit mathematical models which are costly and difficult to obtain for complex systems, while data-driven methods can degrade as data distributions shift over time. Experience-based methods are limited by data availability. This paper aims to bridge this gap by proposing a robust, adaptive monitoring system that learns the nominal behavior of industrial equipment and can detect deviations indicative of potential failures.

Methods

The core of the proposed approach is a two-level LSTM-based autoencoder architecture.

  • Reconstruction Model: This level uses a Long Short-Term Memory (LSTM) autoencoder to learn a continuous nominal vibration model from sensor data. LSTMs are chosen for their ability to handle long-term dependencies in sequential data. The autoencoder consists of an encoder that compresses the input sequence into a "context vector" (latent representation) and a decoder that reconstructs the original signal from this vector. The model is trained to minimize the reconstruction error between the original and reconstructed signals.
  • Continuous Learning Model: This higher level ensures the monitoring system adapts to the evolving behavior of the industrial equipment. It operates with a "current learned model" (Mc) that monitors the real system. Data validated by Mc is used to train a "learning model" (M1). M1 then becomes the new Mc, allowing the system to continuously update its understanding of "normal" behavior. This prevents the model from becoming outdated due to gradual physical degradation or environmental changes.
  • Bayesian Optimization: To fine-tune the numerous hyperparameters of the LSTM autoencoder and the overall system (e.g., learning rate, window size, number of hidden units, sparsity parameter), the authors employ Bayesian optimization. This technique uses a probabilistic surrogate model (Gaussian Process) to efficiently search for optimal hyperparameter configurations, aiming to maximize generalization performance.
  • Regularization: A sparsity constraint is applied to the autoencoder's latent representations. This encourages the activation of hidden units to be low most of the time, promoting a more generalizable encoding and preventing overfitting. The Kullback-Leibler (KL) divergence is used to enforce this sparsity.

Datasets

The experiments were conducted using data collected from a real-world industrial turbo-compressor system. This system is equipped with 10 sensors that continuously monitor axial and radial displacements, as well as rotation speed. The data was acquired at a sampling rate of 1 Hz. The paper does not introduce a new benchmark dataset but uses real operational data from an industrial setting.

Key Insights and Ideas

  • Adaptive Monitoring is Crucial: The paper strongly emphasizes that industrial systems are not static. Continuous learning is essential to maintain accurate monitoring as systems naturally degrade or adapt over time.
  • LSTM Autoencoders for Vibration: LSTM autoencoders prove effective in capturing the complex temporal dynamics of vibration signals in industrial machinery.
  • Bayesian Optimization for Hyperparameter Tuning: The sheer number of hyperparameters in deep learning models makes manual tuning impractical. Bayesian optimization offers a systematic and efficient way to find configurations that yield optimal performance.
  • Sparsity as a Regularizer: Enforcing sparsity on the latent representations helps the autoencoder learn more generalizable features, preventing it from simply memorizing the training data.
  • Two-Level Architecture for Robustness: The combination of a reconstruction model and a continuous learning model creates a robust system that can both detect immediate anomalies and adapt to long-term system evolution.

Key Results

  • The proposed LSTM-based autoencoder model achieved a negligible reconstruction error after approximately 300 training iterations, indicating its ability to accurately model the nominal vibration behavior.
  • The continuous learning model demonstrated its ability to maintain a negligible discrepancy between predicted and real system outputs over time, even with varying nominal training periods.
  • The number of alarms triggered and model replacements remained stable across different nominal training period lengths, suggesting the robustness of the continuous learning mechanism.
  • Bayesian optimization identified key hyperparameters influencing performance, with the "number of hidden units" having the most significant impact (marginal importance of 0.20185), followed by "teacher-forcing temperature" and "sparsity parameter."
  • The system exhibited good reactivity, following abrupt changes in the system's output within approximately two time-steps.

Limitations and Future Work

  • Overfitting Concerns: The authors acknowledge that the relatively low importance of most hyperparameters compared to the number of hidden units could indicate a risk of overfitting. However, they state that out-of-sample validation using a walk-forward approach suggests the model does not suffer from overfitting, even with a large number of hidden units.
  • Latency in Reactivity: While the system is reactive, there is a slight latency (around two time-steps) in following abrupt changes. The authors suggest that better tuning of the regulation function (Equation 7) could further reduce this latency and discrepancy.
  • Future Work: The paper highlights that the success of this approach motivates future work on integrating it with diagnostics capabilities. This would enable more advanced predictive maintenance by not only detecting anomalies but also diagnosing their root causes.

Summary updated

BibTeX
@inproceedings{osmani2019monitoring,
  author = {Aomar Osmani and Massinissa Hamidi and Salah Bouhouche},
  title = {{Monitoring of a Dynamic System Based on Autoencoders}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/254},
}
Figure