Explaining Deep Neural Networks for Bearing Fault Detection with Vibration Concepts

Thomas Decker, Michael Lebacher, Volker Tresp

Abstract

Concept-based explanation methods, such as Concept Activation Vectors (TCAV), are potent means to quantify how abstract or high-level characteristics of input data influence the predictions of complex deep neural networks. However, applying them to industrial prediction problems is challenging as it is not immediately clear how to define and access appropriate concepts for individual use cases and specific data types. In this work, we investigate how to leverage established concept-based explanation techniques in the context of bearing fault detection with deep neural networks trained on vibration signals. Since bearings are prevalent in almost every rotating equipment, ensuring the reliability of intransparent fault detection models is crucial to prevent costly repairs and downtimes of industrial machinery. Our evaluations demonstrate that explaining opaque models in terms of vibration concepts enables human-comprehensible and intuitive insights about their inner workings, but the underlying assumptions need to be carefully validated first.

Introduction

Deep Neural Networks (DNNs) have demonstrated exceptional performance in detecting faults in rotating machinery using raw vibration data. Despite their accuracy, these models often operate as "black boxes," providing little insight into the reasoning behind a specific diagnosis. In industrial settings, where maintenance decisions involve high stakes, engineers require more than just a classification; they need to understand the physical basis of the model's decision.

Traditional Explainable AI (XAI) tools, such as LIME or SHAP, typically highlight specific input features—for instance, identifying a particular millisecond in a time-series signal as "important." However, for a vibration engineer, a single data point at a specific timestamp is largely meaningless. Diagnostic value lies in high-level physical concepts: frequencies, resonances, and harmonics. This paper addresses the "semantic gap" between raw signal values and the domain-specific concepts used in mechanical engineering by applying Testing with Concept Activation Vectors (TCAV) to the domain of bearing fault detection.

Methodology: Bridging Physics and Deep Learning

The core of this research involves quantifying whether a DNN "understands" the underlying physics of a machine or is merely memorizing statistical noise. The authors utilize the TCAV framework through a three-step process:

  1. Defining Vibration Concepts: Rather than using human-labeled visual concepts, the researchers simulate physical vibration concepts using domain expertise. They employ a physical signal model to generate synthetic high-frequency resonances modulated by characteristic fault frequencies, such as the Ball Passing Frequency Outer ring (BPFO).
  2. Training Concept Classifiers: Using a pre-trained fault detection model, the researchers examine its internal activations (the latent space). They train a linear classifier to determine if the model's internal state can distinguish between these simulated physical concepts and random noise.
  3. Calculating TCAV Scores: If the linear classifier successfully separates the concepts, the vector perpendicular to the decision boundary is defined as the Concept Activation Vector. The TCAV score then measures the sensitivity of the model's final prediction to this vector. A high score indicates that the model's prediction is strongly influenced by that specific physical concept.

Experimental Setup and Datasets

The methodology was evaluated using two prominent benchmarks in the field of predictive maintenance:

  • CWRU (Case Western Reserve University): A standard dataset featuring artificially induced faults created via electro-discharge machining.
  • Paderborn (PB): A more rigorous dataset containing both artificially induced faults and "realistic" faults generated through accelerated lifetime tests, which better reflect actual wear and tear in industrial environments.

Key Insights

The Separability Requirement

A critical finding of the study is that not all neural architectures are equally amenable to concept-based explanations. The researchers discovered that vibration concepts were only linearly separable in the latent spaces of DenseNet and ResNet. In contrast, simpler models like LeNet or more complex architectures like Vision Transformers (ViT) did not represent these physical concepts in a linearly organized manner. This suggests that the "explainability" of a model is deeply tied to how its architecture structures internal representations.

Validation of Domain Knowledge

On the CWRU dataset, the models exhibited high sensitivity to the correct physical concepts. This provides quantitative evidence that the DNNs had successfully learned to identify the same patterns that a human expert would use for diagnosis, validating the model's alignment with physical reality.

Detecting Shortcut Learning

The analysis of the Paderborn "realistic" dataset revealed a significant "model deficiency." While the DenseNet model achieved high accuracy, it showed low TCAV scores for inner ring faults. This discrepancy suggests shortcut learning: the network was likely relying on secondary signal patterns or artifacts rather than the actual physical characteristics of the fault. Such models may perform well on benchmarks but could fail unexpectedly when deployed in real-world scenarios where those specific shortcuts are absent.

Results and Performance

  • Accuracy: All tested models (LeNet, AlexNet, ResNet, DenseNet, ViT) achieved near-perfect accuracy (>97%) on the CWRU dataset. On the more challenging Paderborn realistic set, performance varied, with DenseNet leading at 96%.
  • Faithfulness: Statistical significance tests for concept separability were only passed by DenseNet and ResNet, meaning only their explanations can be considered reliable under the TCAV framework.
  • The PB Realistic Gap: On the Paderborn realistic dataset, DenseNet correctly identified 89% of outer ring faults (correlating with a high TCAV score) but only 60% of inner ring faults (correlating with a low TCAV score). This alignment between accuracy and concept sensitivity demonstrates that TCAV can effectively highlight where a model fails to apply domain-consistent logic.

Conclusion and Future Work

This research demonstrates that explaining opaque models through vibration concepts provides intuitive, human-comprehensible insights into industrial AI. However, the study also highlights that the underlying assumptions of XAI methods—such as the linearity of concepts—must be carefully validated.

Future research will focus on defining more nuanced signal properties to capture the complexity of real-world vibrations, which are often noisier than idealized simulations. Additionally, further investigation is needed to understand why specific architectures, particularly those with skip connections like ResNet and DenseNet, tend to organize physical concepts more effectively than others.

Summary updated

Abstract

Concept-based explanation methods, such as Concept Activation Vectors, are potent means to quantify how abstract or high-level characteristics of input data influence the predictions of complex deep neural networks. However, applying them to industrial prediction problems is challenging as it is not immediately clear how to define and access appropriate concepts for individual use cases and specific data types. In this work, we investigate how to leverage established concept-based explanation techniques in the context of bearing fault detection with deep neural networks trained on vibration signals. Since bearings are prevalent in almost every rotating equipment, ensuring the reliability of intransparent fault detection models is crucial to prevent costly repairs and downtimes of industrial machinery. Our evaluations demonstrate that explaining opaque models in terms of vibration concepts enables human-comprehensible and intuitive insights about their inner workings, but the underlying assumptions need to be carefully validated first.
BibTeX
@inproceedings{decker2023explaining,
  author = {Thomas Decker and Michael Lebacher and Volker Tresp},
  title = {{Explaining Deep Neural Networks for Bearing Fault Detection with Vibration Concepts}},
  booktitle = {INDIN},
  year = {2023},
  url = {https://openreview.net/forum?id=8f8t4hCuHm},
}
Figure