TL;DR
This paper introduces SCFSL, a novel supervised contrastive learning framework designed for high-frequency time series, particularly in industrial settings where labeled data is scarce. It tackles the challenge of learning robust representations with limited labels by proposing a modified contrastive loss that encourages intra-class spread and inter-class separation, outperforming existing methods in few-shot learning scenarios.
What Problem Does This Paper Address?
Representation learning for time series has seen significant advancements, largely driven by self-supervised contrastive learning. However, applying these methods to time series with less intuitive semantic meaning, like high-frequency vibration data from industrial machinery, is challenging. Existing self-supervised approaches can suffer from sampling bias, while supervised contrastive learning, though powerful, can lead to "class collapse" where different classes become indistinguishable. In real-world industrial scenarios, obtaining large labeled datasets is often costly and difficult, making few-shot learning a critical consideration. This paper addresses the need for effective representation learning for high-frequency time series under these constraints, aiming to achieve high classification accuracy with minimal labeled data.
Methods
The proposed framework, SCFSL (Supervised Contrastive Few-Shot Learning), is built upon several key components:
- Data Preprocessing: High-frequency time series are segmented, and Fast-Fourier Transformation (FFT) is applied to convert them into frequency spectrums. This step is crucial for capturing relevant features in vibration data. Data augmentation techniques are also employed to create diverse views of the data.
- Encoder Network: A lightweight 1D convolutional neural network (CNN) backbone is used, followed by a Multi-Layer Perceptron (MLP) projector. This architecture is designed to be efficient and effective for vibration signal processing.
- Modified Supervised Contrastive Loss: This is the core innovation. The loss function is designed to:
- Leverage Label Information: It uses supervised signals to guide the learning process, avoiding the sampling bias inherent in purely self-supervised methods.
- Handle Multiple Augmentations: It can effectively incorporate multiple augmented versions of a sample.
- Induce Spread: A novel "spread loss" term is introduced to prevent class collapse by explicitly encouraging representations of different classes to be further apart, while keeping samples within the same class closer. This is achieved by considering both global pairs (original samples and their augmentations) and class-conditional pairs (augmentations of a sample vs. other samples in the same class).
- Downstream Classification: After training the encoder, representations are used for classification. Two approaches are explored: similarity measurement (using dot product) and classical machine learning models (Logistic Regression, SVM).
Datasets
The paper evaluates SCFSL on four publicly available datasets commonly used for fault diagnosis of rotating machinery:
- MFPT: Provided by the Society of Machinery Failure Prevention Technology.
- PU: Bearing dataset from Paderborn University.
- CWRU: Bearing data from Case Western Reserve University Bearing Data Center.
- SEU: Gearbox and bearing vibration dataset from Southeast University.
These datasets represent various working conditions, damage types, and sampling frequencies, providing a comprehensive evaluation environment.
Key Insights and Ideas
- FFT as a Preprocessing Step is Crucial: Transforming vibration data into the frequency domain (using FFT) significantly aids in feature extraction for the encoder, leading to better performance compared to using raw time series directly.
- Supervised Contrastive Learning with Spread Control is Key for Few-Shot: The proposed modified contrastive loss, which explicitly encourages intra-class spread, is vital for preventing class collapse and achieving high accuracy in low-data regimes. This is a significant improvement over standard supervised contrastive learning.
- Augmentations are Beneficial but Require Careful Selection: While augmentations generally improve performance, the paper shows that simply increasing the number of augmentations doesn't always lead to better results. There's an optimal range, and certain augmentation types might be more effective than others.
- Lightweight Architecture is Sufficient: The proposed lightweight CNN encoder, combined with the novel loss, achieves state-of-the-art performance, suggesting that complex, massive models are not always necessary for this task.
Key Results
- Superior Few-Shot Performance: SCFSL consistently outperforms Direct Supervised Learning (DSL) and Self-Supervised Learning (SSL) methods (specifically TS2Vec) across all four datasets in few-shot scenarios (e.g., 4 or 8 labeled samples per class). For instance, on the MFPT dataset with 4 samples per class, SCFSL achieves an accuracy of 0.938±0.008, significantly higher than DSL (0.169±0.042) and SSL (0.737±0.024).
- Robustness and Stability: SCFSL demonstrates lower standard deviations in accuracy compared to other methods, indicating greater stability.
- Effectiveness of Components: Ablation studies confirm that each component of SCFSL (projection layer, spread loss, augmentations, FFT) contributes positively to the overall performance. Removing the spread loss, for example, leads to a notable drop in accuracy.
- Encoder Efficiency: The proposed encoder network is lightweight (4.66M parameters) and outperforms larger, more complex SOTA models when used within the SCFSL framework.
Limitations and Future Work
- Augmentation Sensitivity: While augmentations are beneficial, the paper notes that increasing their number can sometimes compromise performance, suggesting that careful selection and tuning are necessary. Future work could explore more sophisticated, domain-aware augmentation strategies.
- Generalizability: The paper focuses on high-frequency time series. The authors suggest that the proposed SCFSL framework can be extended to more general time series representation tasks by replacing the encoder with suitable backbones.
- Parameter Tuning: Sensitivity analysis highlights the importance of tuning parameters like temperature ($\tau$) and the spread weight ($\alpha$). Further research could explore automated or adaptive methods for parameter optimization.
Summary updated