Adaptive Transfer Learning of Multi-View Time Series Classification

Donglin Zhan, Shiyu Yi, Dongli Xu, Xiao Yu, Denglin Jiang, Siqi Yu, Haoting Zhang, Wenfang Shangguan, Weihua ZhangarXiv 2019 · arXiv

Introduction

Time Series Classification (TSC) is a critical task in domains ranging from medical diagnostics (e.g., ECG analysis) to human activity recognition via wearable sensors. While Deep Neural Networks (DNNs) have significantly advanced the state-of-the-art, they often suffer from severe overfitting when labeled training data is scarce. Furthermore, modern time series data is frequently "multi-view," where an object is observed through multiple sensors or perspectives. Standard methods often fail to leverage the varying importance of these different views. This paper proposes an adaptive transfer learning framework designed to intelligently transfer knowledge from source views to a target view, mitigating data scarcity issues and improving classification accuracy.

Methodology

The proposed framework moves beyond simple weight copying by adaptively weighing the relevance of source data. The process involves several key stages:

  1. Similarity Measurement: Multivariate time series are decomposed into univariate ones. Distances between source and target views are calculated using Dynamic Time Warping (DTW) and Bag of SFA Symbols (BOSS).
  2. Latent Space Mapping: Similarity scores are projected into a latent space to facilitate importance estimation.
  3. Density Estimation: To account for uncertainty and noise, the framework uses Normalizing Flows or Kernel Density Estimation (KDE) to approximate the posterior distribution of "importance."
  4. Importance Sampling: Sampling from this distribution yields a matrix norm ($g_t$), representing the "transfer degree."
  5. Adaptive Pre-training: The $g_t$ value determines the number of pre-training epochs for a specific source view. Views more similar to the target receive more training time.
  6. Fine-Tuning: The model is finally fine-tuned on the target view data.
Overview of the Adaptive Transfer Learning framework showing the flow from source views to the transferred model.

This architecture-agnostic approach allows for integration with various deep learning models, including Fully Convolutional Networks (FCNs), Long Short-Term Memory (LSTM) networks, and Multi-Layer Perceptrons (MLPs).

Datasets and Evaluation

The framework was validated on three distinct benchmarks:

  • UCI Daily and Sports Activity: Motion data from 19 activities with 5 body-part views.
  • Indoor User Movement (RSS): Binary classification of movement patterns based on Radio Signal Strength.
  • Self-Regulation of SCPs: EEG data for cursor control, treated as a 6-view dataset.

Key Results

Experimental results demonstrate that the adaptive framework consistently outperforms both training-from-scratch baselines and standard transfer learning techniques.

Performance comparison of various models (FCN, LSTM, MLP) and their adaptive variants (DTW, BOSS) across three benchmark datasets.

As shown in the table above, the adaptive variants (DTW and BOSS) provide a significant accuracy boost, typically ranging from 2% to 5%. For instance, on the 'Self-Regulation of SCPs' dataset, the BOSS-FCN variant achieved an accuracy of 0.79522, a substantial improvement over the baseline FCN's 0.70307. The results highlight that leveraging classical similarity metrics like DTW to guide modern deep learning training is a highly effective strategy for multi-view TSC.

Conclusion

This research introduces a robust method for multi-view time series classification that addresses the limitations of small-sample learning. By treating view importance as a dynamic probability distribution, the model effectively filters noise and prioritizes relevant knowledge transfer. Future work may focus on reducing the computational overhead of similarity calculations and automating the selection of optimal target views.

Summary updated

Abstract

Time Series Classification (TSC) has been an important and challenging task in data mining, especially on multivariate time series and multi-view time series data sets. Meanwhile, transfer learning has been widely applied in computer vision and natural language processing applications to improve deep neural network's generalization capabilities. However, very few previous works applied transfer learning framework to time series mining problems. Particularly, the technique of measuring similarities between source domain and target domain based on dynamic representation such as density estimation with importance sampling has never been combined with transfer learning framework. In this paper, we first proposed a general adaptive transfer learning framework for multi-view time series data, which shows strong ability in storing inter-view importance value in the process of knowledge transfer. Next, we represented inter-view importance through some time series similarity measurements and approximated the posterior distribution in latent space for the importance sampling via density estimation techniques. We then computed the matrix norm of sampled importance value, which controls the degree of knowledge transfer in pre-training process. We further evaluated our work, applied it to many other time series classification tasks, and observed that our architecture maintained desirable generalization ability. Finally, we concluded that our framework could be adapted with deep learning techniques to receive significant model performance improvements.
BibTeX
@misc{zhan2019adaptive,
  author = {Donglin Zhan and Shiyu Yi and Dongli Xu and Xiao Yu and Denglin Jiang and Siqi Yu and Haoting Zhang and Wenfang Shangguan and Weihua Zhang},
  title = {{Adaptive Transfer Learning of Multi-View Time Series Classification}},
  year = {2019},
  eprint = {1910.07632},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1910.07632},
}
Figure