A Novel Study on a Generalized Model Based on Self-Supervised Learning and Sparse Filtering for Intelligent Bearing Fault Diagnosis

Guocai Nie, Zhongwei Zhang, Mingyu Shao, Zonghao Jiao, Youjia Li, Lei LiSensors 2023 · DOI

TL;DR

This paper introduces a novel generalized model (GSLSF) for intelligent bearing fault diagnosis that leverages self-supervised learning and sparse filtering. By pre-training on unlabeled data with custom "pretext tasks," the model learns robust feature representations, significantly improving diagnostic accuracy and generalization, especially when labeled data is scarce.

What Problem Does This Paper Address?

Modern mechanical equipment, like bearings, is increasingly complex, making fault diagnosis crucial for safety and efficiency. Traditional deep learning methods for fault diagnosis often require vast amounts of labeled data. However, in real-world scenarios, obtaining such extensive labeled datasets for bearing failures is challenging and expensive. This data scarcity leads to poor generalization, meaning models trained on one set of conditions may perform poorly on others. The paper aims to bridge this gap by developing a method that can achieve high diagnostic performance and generalization with minimal labeled training data.

Methods

The proposed Generalized Self-Supervised Learning and Sparse Filtering (GSLSF) model operates in two stages:

  1. Self-Supervised Pre-training: This stage focuses on learning generalizable features from unlabeled vibration data.
  2. Pretext Tasks: The authors design six novel "pretext tasks" based on signal transformations (e.g., value increase/decrease, random zeroing, disordered splicing, adding white noise). The model is trained to identify these transformations. The core idea is that by learning to distinguish these manipulated signals, the model implicitly learns the underlying characteristics of the original vibration signals.
  3. Pseudo-Labels: These signal transformations are automatically assigned pseudo-labels, eliminating the need for manual annotation.
  4. Sparse Filtering: A sparse filtering model is used for feature extraction during this pre-training phase. This model learns a weight matrix that captures essential signal features.
  5. Knowledge Transfer and Target Task:
    • Knowledge Transfer: The learned weight matrix from the pre-training stage is transferred to the target task (actual fault diagnosis). This is a form of transfer learning, where knowledge gained from the auxiliary task (identifying signal transformations) is applied to the primary task (fault classification).
  6. Sparse Filtering Feature Extraction: The transferred sparse filtering model is used to extract features from the labeled (though limited) bearing fault data.
  7. Softmax Regression: A softmax regression classifier is then employed to distinguish between different bearing fault types based on the extracted features.

Datasets

The proposed GSLSF model was validated using two publicly available bearing datasets:

  1. Case Western Reserve University (CWRU) Dataset: This dataset contains vibration signals from rolling bearings under various fault conditions (normal, inner race, outer race, ball faults) and severities, collected at different load conditions (0, 1, 2, 3 hp).
  2. Bearing Data from a Special Test Bench: This dataset was collected from an experimental platform simulating bearing failures, including normal (NC), outer ring fault (OF), inner ring fault (IF), and ball fault (RF) conditions, with varying fault severities.

Key Insights and Ideas

  • Leveraging Unlabeled Data: The core innovation is the effective use of self-supervised learning with custom pretext tasks to extract meaningful features from unlabeled vibration data. This bypasses the bottleneck of manual labeling.
  • Signal Transformations as Pretext Tasks: The idea of using signal transformations (like adding noise or splicing) as pretext tasks is clever. It forces the model to learn robust representations that are invariant to these common signal distortions, which can be analogous to learning invariant features for fault diagnosis.
  • Sparse Filtering for Feature Extraction: Combining sparse filtering with self-supervised learning provides a powerful mechanism for extracting discriminative features, especially when dealing with noisy or limited data.
  • Knowledge Transfer for Generalization: The explicit knowledge transfer step from the pre-training to the target task is crucial for improving the model's generalization ability, allowing it to perform well on unseen fault conditions or variations.

Key Results

  • High Accuracy: The GSLSF model achieved high diagnostic accuracy across both datasets. For the CWRU dataset, it reached an average accuracy of approximately 97.26%. For the second dataset, the average accuracy was around 97.02%.
  • Improved Generalization: The proposed method demonstrated superior generalization ability compared to traditional methods like Sparse Filtering (SF), Stacked Autoencoder (SAE), and Sparse Representation Classification (SRC), especially when dealing with limited labeled data.
  • Robustness: The standard deviation of the accuracy was consistently lower for GSLSF, indicating better robustness and stability of the model.
  • Efficiency: While the pre-training adds computational overhead, the overall running time of the GSLSF model was comparable to or better than some of the compared methods, particularly SRC, which was significantly slower.

Limitations and Future Work

  • Computational Overhead: The self-supervised pre-training process adds computational cost and slightly increases the overall running time compared to methods that don't require pre-training.
  • Labeled Data for Optimization: While the method excels with minimal labeled data, it still requires some labeled data to optimize the decision boundary for the final classification.
  • Future Work: The authors suggest several avenues for future research:
  • Developing more diverse and effective pretext tasks.
  • Exploring the application of this self-supervised learning framework to other fields beyond fault diagnosis.
  • Investigating how to further improve computational efficiency.

Summary updated

Abstract

Recently, deep learning has become more and more extensive in the field of fault diagnosis. However, most deep learning methods rely on large amounts of labeled data to train the model, which leads to their poor generalized ability in the application of different scenarios. To overcome this deficiency, this paper proposes a novel generalized model based on self-supervised learning and sparse filtering (GSLSF). The proposed method includes two stages. Firstly (1), considering the representation of samples on fault and working condition information, designing self-supervised learning pretext tasks and pseudo-labels, and establishing a pre-trained model based on sparse filtering. Secondly (2), a knowledge transfer mechanism from the pre-training model to the target task is established, the fault features of the deep representation are extracted based on the sparse filtering model, and softmax regression is applied to distinguish the type of failure. This method can observably enhance the model's diagnostic performance and generalization ability with limited training data. The validity of the method is proved by the fault diagnosis results of two bearing datasets.
BibTeX
@inproceedings{nie2023novel,
  author = {Guocai Nie and Zhongwei Zhang and Mingyu Shao and Zonghao Jiao and Youjia Li and Lei Li},
  title = {{A Novel Study on a Generalized Model Based on Self-Supervised Learning and Sparse Filtering for Intelligent Bearing Fault Diagnosis}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23041858},
}
Figure