ADL-GAN: Data Augmentation to Improve In-the-Wild ADL Recognition Using GANs

Apiwat Ditthapron, Adam C. Lammert, Emmanuel O. AguIEEE Access 2023 · DOI

Introduction

Recognizing Activities of Daily Living (ADLs) using smartphone sensors is a cornerstone of modern mobile health monitoring. However, researchers face a major hurdle: class imbalance. In "in-the-wild" datasets—collected during a user's normal life—people spend hours sleeping or sitting but perhaps only minutes jogging or exercising.

Extreme class imbalance in the WASH "in-the-wild" dataset, where sedentary activities like sleeping and sitting dominate the recorded hours.

Machine learning models trained on such data become biased toward majority classes and fail to recognize rarer, health-critical activities. Traditional fixes like oversampling or SMOTE often lead to overfitting or generate unrealistic sensor signals that fail to capture the complex, rhythmic nature of human movement. This paper proposes ADL-GAN, a framework that uses translation-based Generative Adversarial Networks (GANs) to generate realistic synthetic data.

Methodology: Translation-Based Augmentation

ADL-GAN moves away from generating data from random noise, instead adapting image-to-image translation concepts (like StarGAN and CycleGAN) for 1D sensor data (accelerometer, gyroscope, and audio MFCCs). The authors introduced two distinct strategies:

  1. Context-transfer ADL-GAN: This model takes a real sample of a common activity (e.g., walking) and "translates" it into a minority activity (e.g., jogging) for the same subject using one-hot encoding to guide the generator.
  2. Subject-transfer ADL-GAN: This model takes a minority activity from one subject and "translates" it to match the unique movement "style" of another. To capture this personal style, the authors used d-vector embeddings trained with a contrastive loss (GE2E) to learn individual movement signatures.

The architecture utilizes a CNN-based generator with self-attention layers to capture long-range dependencies in the sensor signals, ensuring the synthetic data maintains realistic temporal patterns.

Datasets and Evaluation

The study evaluated ADL-GAN across three diverse datasets representing different levels of complexity:

  • WASH (Unscripted): A highly imbalanced "in-the-wild" dataset from 100 subjects.
  • ExtraSensory (In-the-wild): Data from 60 subjects using smartphones and smartwatches in natural settings.
  • MobiAct (Scripted): A controlled dataset where 50 subjects performed specific sequences of activities.
Overview of activity distributions and subject-specific variability, highlighting the diverse data landscapes ADL-GAN must navigate.

Key Results

  • Accuracy Boost: On the "in-the-wild" WASH dataset, Subject-transfer ADL-GAN improved the Balanced Accuracy (BA) by 27.9% over the baseline, significantly outperforming SMOTE.
  • Scripted Success: On the controlled MobiAct dataset, Context-transfer performed best, improving BA by 9.58%.
  • Efficiency: Unlike CycleGAN, which requires a separate model for every pair of activities, ADL-GAN uses a single unified generator to handle all classes, making it significantly more computationally efficient.
  • Realistic Diversity: Quantitative metrics (Inception Score and FID) showed that ADL-GAN produces samples that are not only more realistic but also more diverse than standard conditional GANs, which often suffer from "mode collapse."

Conclusion

ADL-GAN provides a robust solution to the class imbalance problem in ADL recognition. By treating movement as a "style" that can be transferred, the framework generates high-quality synthetic data that respects both the activity context and the individual's unique movement patterns. Future work will explore incorporating "soft" features like GPS location and meta-learning to allow the model to adapt to new subjects with even less data.

Summary updated

BibTeX
@article{ditthapron2023adl,
  author = {Apiwat Ditthapron and Adam C. Lammert and Emmanuel O. Agu},
  title = {{ADL-GAN: Data Augmentation to Improve In-the-Wild ADL Recognition Using GANs}},
  journal = {IEEE Access},
  year = {2023},
  doi = {10.1109/access.2023.3271409},
}
Figure