Model-Agnostic Structural Transfer Learning for Cross-Domain Autonomous Activity Recognition

Parastoo Alinia, Asiful Arefeen, Zhila Esna Ashari, Seyed-Iman Mirzadeh, Hassan Ghasemzadeh2023 · DOI

TL;DR

This paper introduces ActiLabel, a novel framework for activity recognition that can learn from new sensor data without requiring manual labeling. It achieves this by identifying structural similarities between activity patterns in different domains, enabling effective knowledge transfer even when sensor modalities or locations change. This is crucial for making activity recognition systems more adaptable and scalable in real-world applications.

What Problem Does This Paper Address?

Activity recognition, powered by sensors in wearables and mobile devices, is vital for applications like health monitoring and personalized medicine. However, current models are often trained for specific sensor types, locations, or users, leading to poor performance when deployed in new settings. This "domain shift" problem necessitates extensive, time-consuming, and expensive manual labeling of new data. The paper tackles the challenge of building activity recognition models for new domains (e.g., different sensor types) without needing any labeled data from that new domain.

Methods

ActiLabel employs a "structural transfer learning" approach. The core idea is to represent activities as structural relationships rather than just raw sensor data. The framework involves several key steps:

  • Network Graph Construction: Sensor observations are transformed into a graph where nodes represent data points and edges represent their similarity. Cosine similarity is used to measure affinity between sensor observations.
  • Core Cluster Identification: This step identifies groups of similar activity patterns (core clusters) within the target domain's network graph using a graph-based clustering algorithm. For the source domain, these clusters correspond to the known activity classes.
  • Dependency Graph Construction: A higher-level graph is built where nodes represent the identified core clusters. This "dependency graph" captures the structural relationships and similarities between these clusters.
  • Optimal Label Learning: This is the core of the transfer learning. ActiLabel constructs bipartite graphs to map the dependency graph of the target domain to that of the source domain. It uses a min-cost mapping approach, leveraging the Hungarian algorithm, to assign labels from the source domain to the target domain's clusters. This process effectively creates a labeled dataset for the target domain.
  • Model-Agnostic: A key feature is that ActiLabel is model-agnostic. It generates a labeled dataset, which can then be used to train any chosen activity recognition classifier (e.g., k-NN, SVM, deep learning models).

Datasets

The authors evaluated ActiLabel on three publicly available datasets:

  • PAMAP2: A physical activity monitoring dataset with 9 participants performing 24 activities, using IMUs (accelerometer, gyroscope, orientation, temperature) and a heart rate monitor.
  • DAS: A daily and sport activity dataset with 8 participants performing 19 sports activities, using accelerometer, gyroscope, and magnetometer sensors.
  • Smartsock: A dataset containing ankle-worn sensor data (accelerometer, stretch sensor) from 12 participants performing 12 physical tasks.

Key Insights and Ideas

  • Structure over Features: The paper's central insight is that focusing on the structural relationships between activity patterns (represented by dependency graphs) is more effective for cross-domain transfer than relying solely on low-level features, especially when sensor modalities differ significantly.
  • Autonomous Labeling: The ability to generate labels for a new domain without manual annotation is a significant step towards truly adaptive and deployable activity recognition systems.
  • Model Agnosticism: By decoupling the labeling process from the final classifier, ActiLabel offers flexibility, allowing users to employ their preferred machine learning models.
  • Community Detection's Power: The research highlights the effectiveness of community detection algorithms for unsupervised clustering of sensor data, outperforming traditional clustering methods in this context.

Key Results

ActiLabel demonstrated strong performance across various transfer learning scenarios:

  • Cross-Modality Transfer: ActiLabel achieved an average F1-score of 59.2%, outperforming competing methods by a significant margin (e.g., 36.3% better than baseline).
  • Cross-Location Transfer: Achieved an average F1-score of 70.8%, outperforming others by 32.7%.
  • Cross-Subject Transfer: Achieved an average F1-score of 82.7%, outperforming others by 9.1%.
  • Labeling Accuracy: The framework achieved high labeling accuracy (70.2–88.0% for cross-modality, and >70.3% for cross-location) in generating the target domain's labeled dataset.
  • Core Cluster Identification: The proposed Core Cluster Identification (CCI) method outperformed standard clustering and community detection algorithms in terms of Normalized Mutual Information (NMI) and purity.

Limitations and Future Work

  • Subset Assumption: ActiLabel currently assumes that the target domain's activity labels are a subset of the source domain's labels. The authors acknowledge that in real-world scenarios, target activities might be entirely unknown to the source. They are exploring solutions for more complex many-to-many mapping scenarios.
  • Scalability: Graph-based algorithms like ActiLabel can face scalability challenges with very large datasets. Future work includes investigating less computationally expensive graph structures (e.g., kd-graphs, minimum spanning trees) to improve scalability.
  • Computational Efficiency: Optimizing various computational components for time, power, and memory efficiency is a direction for future work, especially for real-world deployment.
  • Generalizability: While the core principles of ActiLabel are applicable to other domains, further research is needed to explore its effectiveness beyond activity recognition.

Summary updated

BibTeX
@misc{alinia2023model,
  author = {Parastoo Alinia and Asiful Arefeen and Zhila Esna Ashari and Seyed-Iman Mirzadeh and Hassan Ghasemzadeh},
  title = {{Model-Agnostic Structural Transfer Learning for Cross-Domain Autonomous Activity Recognition}},
  year = {2023},
  doi = {10.3390/s23146337},
}
Figure