Does Your Model Think Like an Engineer? Explainable AI for Bearing Fault Detection with Deep Learning

Thomas Decker, Michael Lebacher, Volker Tresp

Deep learning has become a powerhouse for industrial predictive maintenance, but for the engineers on the factory floor, these models remain frustrating "black boxes." When a model flags a bearing failure, an engineer doesn't just want a "yes/no" answer—they want to know if the model is looking at the same physical signatures they’ve used for decades.

In their paper, "Does Your Model Think Like an Engineer? Explainable AI for Bearing Fault Detection with Deep Learning," Thomas Decker and his colleagues from LMU Munich and Siemens AG bridge the gap between high-performance AI and classical mechanical engineering.

TL;DR

The authors propose a framework that translates "black-box" deep learning explanations into the envelope spectrum—the standard language of vibration engineers. By quantifying how much a model relies on known physical fault frequencies, they can predict which models will actually generalize to new machinery and which are just "getting lucky" on training data.

What Problem Does This Paper Address?

In industrial settings, trust is the primary barrier to AI adoption. For rolling element bearings, engineers traditionally use Envelope Analysis to find periodic impacts caused by defects (like a crack on an outer ring). These impacts occur at specific, mathematically predictable frequencies (e.g., the Ball Passing Frequency Outer ring, or BPFO).

Standard Explainable AI (XAI) tools like LIME or Integrated Gradients usually provide "attributions" in the raw input domain. For vibration data, this means highlighting specific points in a messy time-series signal. To an engineer, these highlights are often incomprehensible. This paper addresses the interpretability gap: how do we transform abstract AI explanations into domain-specific insights that prove the model understands the underlying physics?

Methods

The core innovation is a Domain Mapping Framework. Instead of trying to explain the model's decision in the raw time domain, the authors mathematically "wrap" the model to allow explanations to be calculated in the frequency domain.

  1. Augmented Model: They define a mapping using the Hilbert Transform (to get the signal envelope) and the Fourier Transform. This allows them to treat the transformation as a linear input layer, enabling standard XAI methods (like Integrated Gradients or SHAP) to attribute importance directly to specific frequencies.
  2. Alignment Metrics: To move beyond visual "heatmaps," they introduce two quantitative metrics:
  3. Informative Attribution Share (IAS): The percentage of the model's "attention" that falls on the BPFO and its harmonics.
  4. Informative Signal to Noise Ratio (ISNR): A decibel-scale ratio comparing the importance of "correct" physical frequencies versus "noise" frequencies.

Datasets

The study utilizes the Paderborn University (Kat-DataCenter) dataset, a gold standard in the field. It contains vibration recordings from:

  • 6 undamaged bearings.
  • 6 bearings with artificially induced faults.
  • 5 bearings with "genuine" faults from accelerated lifetime tests.
  • Data across different operating conditions (e.g., 1500 RPM vs. 900 RPM) to test generalization.

Key Insights and Ideas

  • Alignment Equals Robustness: The authors found a direct correlation between "thinking like an engineer" and performance. Models that focused their attribution on the BPFO (high IAS) were significantly better at generalizing to new rotation speeds and "genuine" (non-artificial) faults.
  • The Danger of "Clever Hans" Models: Some architectures, like Fully-Connected Networks (FCNs), achieved high accuracy on training data but had very low alignment with domain knowledge. These models were essentially "cheating" by picking up on spurious correlations in the data that didn't exist in real-world scenarios.
  • CNNs vs. LSTMs: Convolutional Neural Networks (CNNs) showed the highest alignment with engineering logic, followed by LSTMs. This suggests that the internal filters of a CNN are naturally better at capturing the periodic, frequency-based nature of mechanical faults.

Key Results

The researchers compared a CNN, an LSTM, and a Fully-Connected Network (FCN) against a "BASE" physics-only classifier:

  • Generalization Gap: While all models performed well on the training set, the FCN’s precision plummeted to 0.487 when tested on a new rotation speed. In contrast, the CNN maintained a precision of 0.994.
  • Quantifying Trust: The CNN showed the highest ISNR, meaning its decisions were most heavily rooted in the characteristic fault frequencies.
  • Visual Proof: When visualizing attributions in the envelope spectrum, the CNN clearly "peaked" at the BPFO, whereas the FCN's attention was scattered across irrelevant frequencies, explaining why it failed in the real-world test.

Limitations and Future Work

While the framework is a major step forward, the authors note a few areas for growth:

  • Scope: The study focused primarily on outer ring faults. Future work is needed to see if the same alignment holds for more complex inner ring or cage faults, where signal modulation makes the physics trickier.
  • Noise Environments: The dataset, while realistic, may not capture the extreme background noise of a heavy industrial plant. Testing the "ISNR" metric in high-noise environments will be a crucial next step.
  • Automation: The current framework requires knowing the BPFO in advance. Future iterations could potentially use XAI to discover new informative frequencies in complex systems where the physics isn't yet fully modeled.

Summary updated

Abstract

Deep Learning has already been successfully applied to analyze industrial sensor data in a variety of relevant use cases. However, the opaque nature of many well-performing methods poses a major obstacle for real-world deployment. Explainable AI (XAI) and especially feature attribution techniques promise to enable insights about how such models form their decision. But the plain application of such methods often fails to provide truly informative and problem-specific insights to domain experts. In this work, we focus on the specific task of detecting faults in rolling element bearings from vibration signals. We propose a novel and domain-specific feature attribution framework that allows us to evaluate how well the underlying logic of a model corresponds with expert reasoning. Utilizing the framework we are able to validate the trustworthiness and to successfully anticipate the generalization ability of different wellperforming deep learning models. Our methodology demonstrates how signal processing tools can effectively be used to enhance Explainable AI techniques and acts as a template for similar problems.
BibTeX
@inproceedings{decker2023does,
  author = {Thomas Decker and Michael Lebacher and Volker Tresp},
  title = {{Does Your Model Think Like an Engineer? Explainable AI for Bearing Fault Detection with Deep Learning}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  url = {https://openreview.net/forum?id=7adLZUJtPi},
}

Loading related content...

Figure