Joint-Label Learning by Dual Augmentation for Time Series Classification

Qianli Ma, Zhenjing Zheng, Jiawei Zheng, Sen Li, Wanqing Zhuang, Garrison W. Cottrell

Introduction

Deep Neural Networks (DNNs) have revolutionized Time Series Classification (TSC), yet their performance remains heavily dependent on the availability of large, labeled datasets. In scenarios where data is scarce, Data Augmentation (DA)—the process of creating synthetic samples through transformations like jittering or scaling—is the standard remedy. However, traditional DA techniques suffer from a fundamental limitation: they assign the same label to the augmented sample as the original. If a transformation significantly alters the sample, it may drift into the distribution of a different class, blurring decision boundaries and hindering the model's ability to generalize.

The JobDA Framework

To address this "distribution expansion" problem, we introduce JobDA (Joint-label learning by Dual Augmentation). This framework redefines how models interact with augmented data by treating transformations not just as more samples, but as distinct new categories.

The JobDA training and testing pipeline compared to standard data augmentation.

The framework consists of two primary components:

  1. Time-Series Warping (TSW): A novel augmentation technique that divides a signal into segments and applies alternating upsampling and downsampling. This creates a "warped" version of the signal that preserves the overall length while distorting local features, effectively acting as a "fun-house mirror" for time series data.
  2. Joint-Label Learning: Instead of a simple $C$-class classification task, the model is trained on a joint objective. Given $C$ original classes and $M$ transformation types, the model learns to classify samples into $C \times M$ distinct categories. For instance, the model must distinguish between a "Normal Class A" sample and a "Warped Class A" sample.

During the inference phase, the model's joint-label predictions are aggregated. The final probability for an original class is calculated by summing the predicted probabilities of all its associated joint labels (e.g., $P(\text{Class A}) = P(\text{Normal A}) + P(\text{Warped A})$).

Feature Space and Generalization

A key advantage of JobDA is its impact on the model's internal representations. By forcing the network to identify specific transformations, the approach encourages the formation of more nuanced and compact feature clusters.

Visualization of feature maps showing the effect of joint-label learning on cluster separation.

As shown in the feature map visualizations, standard DA often results in large, overlapping clouds of data points. In contrast, JobDA produces multiple tight, well-separated clusters. This separation leads to sharper decision boundaries and helps the model converge to "flat minima" in the loss landscape—regions where the model's performance is less sensitive to small weight perturbations, a hallmark of strong generalization.

Experimental Results

The JobDA framework was rigorously tested using the UCR Time Series Classification Archive, encompassing 85 diverse datasets. Using a ResNet backbone, the joint-label variant (ResNet_JL) demonstrated significant improvements:

  • Superior Ranking: ResNet_JL achieved an average rank of 2.653 across the archive, outperforming standard ResNet (3.053) and FCN (4.000).
  • Small Data Robustness: On datasets like PhalangesOutlinesCorrect, ResNet_JL maintained high accuracy even when training data was reduced to 10-20% of its original size, whereas standard models experienced sharp declines.
  • Imbalance Handling: The creation of distinct sub-categories through warping acts as a sophisticated oversampling mechanism, improving performance on imbalanced datasets such as Earthquakes and ECG5000.

Limitations and Future Directions

While effective, JobDA faces challenges when the number of original classes ($C$) is very large and classes are highly similar. In such cases, the $C \times M$ expansion can lead to "category proliferation," where the joint labels overlap excessively, potentially degrading performance. Future research will explore extending this dual augmentation logic to unsupervised learning settings, where models could learn robust features by predicting transformations without any human-provided labels.

Summary updated

Abstract

Recently, deep neural networks (DNNs) have achieved excellent performance on time series classification. However, DNNs require large amounts of labeled data for supervised training. Although data augmentation can alleviate this problem, the standard approach assigns the same label to all augmented samples from the same source. This leads to the expansion of the data distribution such that the classification boundaries may be even harder to determine. In this paper, we propose Joint-label learning by Dual Augmentation (JobDA), which can enrich the training samples without expanding the distribution of the original data. Instead, we apply simple transformations to the time series and give these modified time series new labels, so that the model has to distinguish between these and the original data, as well as separating the original classes. This approach sharpens the boundaries around the original time series, and results in superior classification performance. We use Time Series Warping for our transformations: We shrink and stretch different regions of the original time series, like a fun-house mirror. Experiments conducted on extensive time-series datasets show that JobDA can improve the model performance on small datasets. Moreover, we verify that JobDA has better generalization ability compared with conventional data augmentation, and the visualization analysis further demonstrates that JobDA can learn more compact clusters.
BibTeX
@inproceedings{ma2021joint,
  author = {Qianli Ma and Zhenjing Zheng and Jiawei Zheng and Sen Li and Wanqing Zhuang and Garrison W. Cottrell},
  title = {{Joint-Label Learning by Dual Augmentation for Time Series Classification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i10.17071},
}
Figure