TL;DR
This paper introduces a novel lightweight transfer learning framework for diagnosing faults in rolling bearings, especially when data distributions differ significantly between training and testing devices. By combining knowledge distillation with domain adaptation, the method effectively transfers diagnostic capabilities from a powerful "teacher" model to a more efficient "student" model, achieving high accuracy even with limited data across different bearing types.
What Problem Does This Paper Address?
Industrial equipment, particularly rotating components like bearings, is prone to failures that can lead to costly downtime and safety hazards. While deep learning, especially Convolutional Neural Networks (CNNs), has shown promise in fault diagnosis, its application faces challenges:
- Data Distribution Shifts: Models trained on data from one device or operating condition often perform poorly on data from different devices (cross-device scenarios) due to variations in materials, configurations, and operating environments. This is a more pronounced challenge than cross-condition shifts.
- Computational Demands: Deep learning models, while powerful, can be computationally expensive and require significant memory, making them difficult to deploy in resource-constrained industrial settings.
- Imbalanced Data: Real-world fault data is often imbalanced, with many more healthy samples than faulty ones, further complicating model training and generalization.
This paper specifically tackles the problem of fault diagnosis across different bearing devices, where data distribution differences are substantial and inter-class data can be incomplete.
Methods
The proposed framework, KD-ResNet-DA (Knowledge Distillation-based Residual Network with Domain Adaptation), integrates two key techniques:
- Lightweight Knowledge Distillation:
- A large, complex teacher model (a variable-scale ResNet) is trained on the source domain data to learn robust, domain-invariant fault features.
- This teacher model then generates "soft labels" (probability distributions over classes) for the training data.
- A smaller, more efficient student model (a shallow ResNet) is trained to mimic these soft labels from the teacher, in addition to learning from the true "hard labels." This process transfers the teacher's knowledge to the student, enabling the student to achieve high performance with fewer parameters and less computation.
- A temperature factor (T) is used in the Softmax function of the teacher model to smooth the output probabilities, capturing more nuanced class similarities.
- Feature-Level Domain Adaptation:
- To bridge the gap between source and target domain data distributions, a Maximum Mean Discrepancy (MMD) loss is employed at the feature level.
- MMD measures the distance between the feature distributions of the source and target domains in a Reproducing Kernel Hilbert Space (RKHS).
- By minimizing this MMD loss, the model is encouraged to learn domain-invariant features, ensuring that the learned representations are transferable across different devices.
- A multi-kernel approach is used for MMD to enhance adaptability to diverse feature distributions.
The overall objective function combines three components: distillation error (between student's soft predictions and teacher's soft labels), classification error (between student's hard predictions and true labels), and the MMD loss for domain adaptation.
Datasets
The paper uses two datasets for experimental validation:
- Source Domain Dataset A: Publicly available bearing fault dataset from Case Western Reserve University (CWRU). Features SKF6205 ball bearings, operating at 1772 r/min, sampled at 12 kHz.
- Target Domain Dataset B: A dataset from Qilu Normal University (QLNU). Features UCPH206 ball bearings, operating at 900 r/min, sampled at 51.2 kHz.
The experiments are designed to simulate cross-device scenarios with incomplete inter-class data, meaning only a few fault samples are available for certain fault types in the target domain.
Key Insights and Ideas
- Knowledge Distillation for Efficiency: Using knowledge distillation is a powerful way to compress large, accurate models into smaller, deployable ones without a significant loss in performance, which is crucial for industrial applications.
- Domain Adaptation is Key for Cross-Device: Simply using knowledge distillation isn't enough when device distributions differ drastically. Explicitly minimizing the feature distribution distance (MMD) is essential for successful transfer learning across different bearing types.
- Teacher's Soft Labels Provide Crucial Information: The ablation studies suggest that the teacher model's soft labels are vital for providing stable and essential fault information, leading to better student model performance compared to methods that omit distillation.
- Multi-Kernel MMD Enhances Robustness: Employing multiple Gaussian kernels for MMD allows the model to better adapt to the varied feature distributions encountered in different cross-device scenarios.
- Synergy of Techniques: The paper demonstrates that the combination of knowledge distillation and feature-level domain adaptation is more effective than either technique alone for challenging cross-device fault diagnosis tasks.
Key Results
- High Accuracy: The proposed KD-ResNet-DA method achieved an accuracy of 99.50% in the first case study (CWRU to QLNU) and 97.60% in the second case study (laboratory to real-world bearings).
- Superiority over Baselines: In ablation studies, KD-ResNet-DA outperformed other methods like ResNet-DA (without distillation), KD-ResNet (without domain adaptation), and KD-CNN-DA (modified CNN structure), achieving accuracies of 97.60% compared to 87.40%, 86.00%, and 90.80%, respectively.
- Comparison with Classical Algorithms: KD-ResNet-DA achieved an average accuracy of 96.25%, significantly outperforming methods like SMOTE (64.18%), GAN (76.62%), CORAL (82.84%), and DANN (89.38%) in cross-device scenarios with incomplete inter-class data.
- Model Compression: The student model is significantly smaller than the teacher model, reducing computational complexity by one-sixth and parameter count by one-eighth, making it suitable for deployment.
- Convergence: The KD-ResNet-DA method showed the best convergence speed and stability during training.
Limitations and Future Work
- Identical Sample Labels Assumption: The current study assumes identical sample labels between source and target domains. This is a limitation for real-world scenarios where fault types might not perfectly align.
- Future Work: The authors plan to address open-set fault diagnosis issues, where the target domain may contain fault types not present in the source domain. This would further enhance the applicability and generality of the method.
Summary updated