Abstract
Intelligent Fault Diagnosis (IFD) based on deep learning has proven to be an effective and flexible solution, attracting extensive research. Deep neural networks can learn rich representations from vast amounts of representative labeled data for various applications. In IFD, they achieve high classification performance from signals in an end-to-end manner, without requiring extensive domain knowledge. However, deep learning models usually only perform well on the data distribution they have been trained on. When applied to a different distribution, they may experience performance drops. This is also observed in IFD, where assets are often operated in working conditions different from those in which labeled data have been collected. Unsupervised domain adaptation (UDA) deals with the scenario where labeled data are available in a source domain, and only unlabeled data are available in a target domain, where domains may correspond to operating conditions. Recent methods rely on training with confident pseudo-labels for target samples. However, the confidence-based selection of pseudo-labels is hindered by poorly calibrated confidence estimates in the target domain, primarily due to over-confident predictions, which limits the quality of pseudo-labels and leads to error accumulation. In this paper, we propose a novel UDA method called Calibrated Adaptive Teacher (CAT), where we propose to calibrate the predictions of the teacher network throughout the self-training process, leveraging post-hoc calibration techniques. We evaluate CAT on domain-adaptive IFD and perform extensive experiments on the Paderborn benchmark for bearing fault diagnosis under varying operating conditions. Our proposed method achieves state-of-the-art performance on most transfer tasks.
Introduction: The Challenge of Domain Shift in IFD
Deep learning models for machine fault diagnosis often fail when operating conditions—such as rotational speed or load—change. This phenomenon, known as domain shift, causes models to become overconfident in their own mistakes. While Unsupervised Domain Adaptation (UDA) aims to bridge this gap using labeled source data and unlabeled target data, standard self-training methods often fall into a "vicious cycle." The model generates "confident" but incorrect pseudo-labels for the target domain, leading to error accumulation that degrades performance.
The Calibrated Adaptive Teacher (CAT) Framework
To address the issue of miscalibration under domain shift, we introduce Calibrated Adaptive Teacher (CAT). This framework integrates post-hoc calibration directly into the self-training loop to ensure the model's confidence accurately reflects its predictive accuracy.
The CAT architecture consists of several key components:
- Teacher-Student Architecture: A "Student" network learns from the data, while a "Teacher" network—maintained as an exponential moving average (EMA) of the student's weights—provides stable pseudo-labels.
- Post-hoc Calibration: Before generating pseudo-labels, the Teacher's predictions are calibrated. Among various techniques, Temperature Scaling proved most effective, using a single learned parameter to "soften" output probabilities and mitigate overconfidence.
- Adaptive Thresholding: Rather than a fixed confidence cutoff, CAT employs a curriculum-based threshold that adjusts per class based on the model's learning progress for specific faults.
- Feature Alignment: Using Domain-Adversarial Neural Networks (DANN), the framework ensures that internal feature representations remain consistent across both source and target domains.
Comparative Analysis of Methodologies
CAT distinguishes itself from existing UDA methods by focusing on the reliability of confidence estimates. As shown in the table below, while other methods utilize various feature alignment and self-training strategies, CAT is unique in its combination of DANN-based alignment and calibrated pseudo-label filtering.
Experimental Evaluation
We evaluated CAT on the Paderborn University (PU) bearing dataset, a rigorous benchmark involving motor current signals for 13-class fault classification. The experiments covered 12 transfer tasks across four distinct operating conditions.
Key Results
- Performance Gains: CAT with Temperature Scaling achieved an average accuracy of 54.5% on time-domain tasks, a 7.5% absolute improvement over the DANN baseline. In the frequency domain, it reached 64.8%, setting a new state-of-the-art.
- Calibration Reliability: The method reduced the Expected Calibration Error (ECE) by a factor of four compared to standard DANN. This ensures that when the model reports high confidence, it is statistically more likely to be correct.
- Robustness of Simplicity: Interestingly, simpler calibration methods like Temperature Scaling outperformed more complex variants (e.g., Matrix Scaling), which tended to overfit the source domain and fail to generalize to the target.
Conclusion and Future Directions
The CAT framework demonstrates that calibration is a critical, often overlooked component of robust domain adaptation. By fixing the underlying confidence metrics, we can build more reliable self-training loops for industrial applications. Future work will explore the integration of domain-specific data augmentations and the application of CAT to multi-sensor environments with varying noise profiles.
Summary updated