Deep Unsupervised Domain Adaptation with Time Series Sensor Data: A Survey

Yongjie Shi, Xianghua Ying, Jinfa Yang

TL;DR

Deep learning models trained on sensor data often fail when applied to new, unseen environments due to "domain gaps." This survey provides a comprehensive overview of Unsupervised Domain Adaptation (UDA) techniques specifically for time-series sensor data, categorizing methods by their adaptation strategy and settings, and highlighting applications across various industries.

What Problem Does This Paper Address?

Sensors are ubiquitous, collecting vital time-series data for everything from industrial machinery monitoring to healthcare and human activity recognition. Deep learning models excel at analyzing this data, but they typically assume that training and testing data come from the same distribution (domain). In reality, sensor data often exhibits "domain shifts" due to variations in operating conditions, sensor placement, equipment differences, or even different users. A model trained in one scenario (source domain) might perform poorly on data from another (target domain). Retraining models for every new domain is resource-intensive. Unsupervised Domain Adaptation (UDA) offers a solution by leveraging labeled source data and unlabeled target data to adapt models, bridging this domain gap without requiring extensive target-domain labeling. This paper surveys the landscape of UDA for time-series sensor data, organizing existing research by application, method, and adaptation setting.

Methods

This survey categorizes UDA methods for time-series sensor data into four main types:

  • Input Space Adaptation: Aims to make source domain data more similar to target domain data, often using generative adversarial networks (GANs) or prior knowledge to synthesize source samples.
  • Feature Space Adaptation: Focuses on learning domain-invariant feature representations. This is achieved through:
  • Mapping-based methods: Using techniques like Maximum Mean Discrepancy (MMD) to minimize the distance between source and target feature distributions.
  • Adversarial-based methods: Employing a discriminator network to distinguish between source and target features, while the feature extractor tries to "fool" the discriminator, thus learning domain-invariant features.
  • Output Space Adaptation: Primarily uses pseudo-labeling, where high-confidence predictions on target data are used as labels for further training.
  • Model-Based Adaptation: Imposes constraints on model parameters to align domain distributions, often using techniques like adaptive batch normalization.

The paper also discusses different adaptation settings: single-source single-target, multi-source single-target, and multi-target domain adaptation, analyzing their advantages and challenges.

Datasets

The paper highlights several publicly available datasets used across various applications:

  • Industry:
    • CWRU (Case Western Reserve University) Dataset for rolling bearing fault diagnosis.
  • PU (Paderborn University) Dataset for bearing fault diagnosis.
  • IMS Dataset for bearing run-to-failure tests.
  • NASA Battery Dataset and CALCE Battery Dataset for lithium-ion battery capacity estimation.
  • C-MAPPS (Commercial Modular Aero-Propulsion System Simulation) datasets for turbofan engine RUL estimation.
  • IEEE PHM Challenge 2012 bearing dataset.
  • Biosignal:
    • SEED and DEAP datasets for EEG-based emotion recognition.
  • BCI Competition IV-IIa and IV-IIb datasets for EEG-based motor imagery.
  • Opportunity, HHAR, PAMAP2, and RealWorld datasets for human activity recognition.
  • CapgMyo, NinaPro, and CSL-HDEMG datasets for EMG-based muscle-computer interfaces.
  • Daphnet, OU-ISIR, CASIA-B, and CASIA-C datasets for gait analysis.

No new benchmarks were introduced in this survey.

Key Insights and Ideas

  • Ubiquity of Domain Gaps: The paper strongly emphasizes that domain shifts are a pervasive issue across nearly all sensor-based applications, stemming from diverse factors like operating conditions, sensor placement, and individual user differences.
  • UDA as a Crucial Solution: UDA is presented not just as an academic curiosity but as a practical necessity for deploying robust deep learning models in real-world sensor applications, saving significant effort in data labeling.
  • Methodological Diversity: The survey reveals a rich and evolving set of UDA techniques, moving beyond simple feature alignment to more sophisticated adversarial and model-based approaches.
  • Application-Specific Challenges: Different application domains (industry, transportation, biosignal) present unique sources of domain gaps, requiring tailored UDA strategies. For instance, user variability is a major challenge in biosignal applications, while operating conditions dominate in industrial fault diagnosis.
  • Setting Matters: The choice of adaptation setting (single-source/target vs. multi-source/target) significantly impacts the complexity and potential effectiveness of UDA, with multi-source/target settings offering greater flexibility but also introducing new challenges like negative adaptation.

Key Results

As this is a survey paper, it does not present novel experimental results. Instead, it synthesizes findings from numerous existing studies. The key "results" are the comprehensive classification and comparison of:

  • Applications: Identifying common sensor types, domain gap sources, and datasets across industry, transportation, and biosignal domains.
  • Methods: Categorizing and describing various UDA techniques (input, feature, output, model-based) and their underlying principles.
  • Settings: Analyzing the advantages and references for single-source single-target, multi-source, and multi-target UDA.

The paper effectively demonstrates the broad applicability and ongoing development of UDA for time-series sensor data.

Limitations and Future Work

Limitations:

  • Negative Adaptation: In multi-source domain adaptation, some source domains might negatively impact performance on the target domain.
  • Limited Research in Multi-Target UDA: The paper notes that research on multi-target UDA for time-series sensor data is still limited.
  • Generalizability of Deep Models: While UDA aims to improve generalization, the inherent complexity of deep models can still pose challenges.

Future Work:

  • Exploring Transformer Architectures: Investigating the use of Transformer models as backbone networks for UDA in time-series sensor data.
  • Knowledge Distillation: Leveraging knowledge distillation with multiple teachers for multi-source UDA.
  • Contrastive Learning: Integrating contrastive learning to enhance feature extraction from unlabeled data before applying UDA.
  • Multi-Source Multi-Target UDA: Developing methods for scenarios with multiple source and multiple target domains, aiming for generalized models that perform well across various conditions.
  • Wearable Sensor Data: Addressing domain shifts in applications like heart rate and ECG recognition from smart wearable devices, where individual differences are significant.

Summary updated

Abstract

Sensors are devices that output signals for sensing physical phenomena and are widely used in all aspects of our social production activities. The continuous recording of physical parameters allows effective analysis of the operational status of the monitored system and prediction of unknown risks. Thanks to the development of deep learning, the ability to analyze temporal signals collected by sensors has been greatly improved. However, models trained in the source domain do not perform well in the target domain due to the presence of domain gaps. In recent years, many researchers have used deep unsupervised domain adaptation techniques to address the domain gap between signals collected by sensors in different scenarios, i.e., using labeled data in the source domain and unlabeled data in the target domain to improve the performance of models in the target domain. This survey first summarizes the background of recent research on unsupervised domain adaptation with time series sensor data, the types of sensors used, the domain gap between the source and target domains, and commonly used datasets. Then, the paper classifies and compares different unsupervised domain adaptation methods according to the way of adaptation and summarizes different adaptation settings based on the number of source and target domains. Finally, this survey discusses the challenges of the current research and provides an outlook on future work. This survey systematically reviews and summarizes recent research on unsupervised domain adaptation for time series sensor data to provide the reader with a systematic understanding of the field.
BibTeX
@inproceedings{shi2022deep,
  author = {Yongjie Shi and Xianghua Ying and Jinfa Yang},
  title = {{Deep Unsupervised Domain Adaptation with Time Series Sensor Data: A Survey}},
  booktitle = {Sensors},
  year = {2022},
  url = {https://openreview.net/forum?id=O87XVFpiuf},
}
Figure