Leveraging Wearable Sensors for Human Daily Activity Recognition with Stacked Denoising Autoencoders

Qin Ni, Zhuo Fan, Lei Zhang, Chris Nugent, Ian Cleland, Yuping Zhang, Nan ZhouSensors 2020 · DOI

TL;DR

This paper introduces a novel framework using Stacked Denoising Autoencoders (SDAE) for human daily activity recognition, specifically focusing on the challenging "transitional" activities (like standing up from a chair). By automatically learning features and employing resampling techniques to handle imbalanced data, the SDAE model achieves a high overall accuracy of 94.88%, outperforming traditional machine learning methods.

What Problem Does This Paper Address?

Human activity recognition (HAR) is a growing field with applications in healthcare, industry, and more. While recognizing static (e.g., sleeping) and dynamic (e.g., walking) activities has seen good progress, "transitional" activities—those that bridge two other activities—are significantly harder to detect. This is primarily due to their short duration, leading to a scarcity of data and making them prone to misclassification. This imbalance in data is a major hurdle for achieving high accuracy in HAR systems, especially when these transitional activities are crucial for understanding overall human behavior and health.

Methods

The core of this work is a deep learning approach using a Stacked Denoising Autoencoder (SDAE). Unlike traditional machine learning methods that require manual feature engineering, SDAE automatically learns hierarchical feature representations from raw sensor data.

Here's a breakdown of the key methodological components:

  • Stacked Denoising Autoencoder (SDAE): This deep learning architecture consists of multiple layers of denoising autoencoders. Each autoencoder learns to reconstruct its input after it has been intentionally corrupted (e.g., by randomly dropping nodes). This process forces the autoencoder to learn robust, underlying features. Stacking these layers allows the model to learn increasingly complex and abstract representations.
  • Automatic Feature Extraction: SDAE's primary advantage is its ability to learn features directly from the data, eliminating the need for manual feature engineering, which is time-consuming and can be suboptimal.
  • Resampling Technique (Random Oversampling): To address the problem of imbalanced datasets (where transitional activities have far fewer samples than static or dynamic ones), the researchers employed random oversampling. This technique duplicates samples from minority classes to balance the dataset, thereby improving the model's ability to learn from them.
  • Data Preprocessing: This included:
  • Segmentation: Using a sliding window approach (50% overlap) to divide the continuous sensor data into meaningful segments.
  • Standardization: Normalizing the data to a 0-1 range to improve model performance.

Datasets

The study utilized two main sources of data:

  • Custom Dataset: Collected from 10 healthy adults in a controlled laboratory environment at Ulster University. This dataset included 12 daily activities categorized into static, dynamic, and transitional types. Wearable sensors (accelerometer and gyroscope) were used.
  • Three Public Datasets: To validate the generalizability of their approach, the researchers also tested their SDAE model on three publicly available datasets:
  • Smartphone dataset
    • Chest-mounted accelerometer dataset
    • UCI daily and sports dataset

Key Insights and Ideas

  • Transitional Activities are Crucial and Challenging: The paper strongly emphasizes that transitional activities, though difficult to recognize, are vital for a comprehensive understanding of human behavior and have significant implications, particularly in healthcare (e.g., fall detection).
  • Deep Learning for Automatic Feature Learning: SDAE's ability to automatically extract features is a significant advantage over traditional methods, saving time and potentially leading to more effective representations.
  • Data Imbalance is a Major Hurdle: The stark difference in sample counts between transitional and other activities significantly degrades recognition accuracy, highlighting the necessity of techniques like resampling.
  • Sensor Fusion Matters: Combining accelerometer and gyroscope data yields better overall performance than using either sensor alone, with each sensor contributing uniquely to recognizing different activity types.
  • SDAE's Robustness: The denoising aspect of the autoencoders helps mitigate overfitting, a common issue when dealing with imbalanced data and complex models.

Key Results

  • Overall Accuracy: The proposed SDAE framework achieved an impressive overall accuracy of 94.88% across static, dynamic, and transitional activities.
  • Transitional Activity Performance: The model showed significant improvement in recognizing transitional activities, which were previously a major challenge. For instance, stand-to-sit reached an accuracy of 98.92%.
  • Comparison with Traditional Methods: The SDAE model outperformed conventional machine learning algorithms like SVM, DT, and KNN, achieving higher accuracy (94.88% vs. ~90.95% for SVM).
  • Sensor Performance:
    • Accelerometer alone achieved 93.36% accuracy.
  • Gyroscope alone achieved a lower 75.76% accuracy.
  • Combining accelerometer and gyroscope (Acc+Gyro) yielded the best performance at 94.88%.
  • Public Dataset Validation: The SDAE model also demonstrated strong performance on three public datasets, indicating its broad applicability.

Limitations and Future Work

  • Sensor Limitations: The study acknowledges that the measurement accuracy of wearable sensors can impact recognition performance. While they explored accelerometer and gyroscope, future work could investigate other sensor modalities.
  • Computational Cost: The paper notes that SDAE requires longer runtime for training compared to some conventional methods, although it saves time on feature extraction.
  • Future Research Directions:
    • Investigating more specialized human activity recognition for specific fields like healthcare.
  • Exploring the use of more diverse sensor types.
  • Combining activity recognition with computer vision for a more comprehensive understanding of human behavior.
  • Adjusting model architectures (like CNN and LSTM) for specific activity types where they might currently underperform.

Summary updated

Abstract

Activity recognition has received considerable attention in many research fields, such as industrial and healthcare fields. However, many researches about activity recognition have focused on static activities and dynamic activities in current literature, while, the transitional activities, such as stand-to-sit and sit-to-stand, are more difficult to recognize than both of them. Consider that it may be important in real applications. Thus, a novel framework is proposed in this paper to recognize static activities, dynamic activities, and transitional activities by utilizing stacked denoising autoencoders (SDAE), which is able to extract features automatically as a deep learning model rather than utilize manual features extracted by conventional machine learning methods. Moreover, the resampling technique (random oversampling) is used to improve problem of unbalanced samples due to relatively short duration characteristic of transitional activity. The experiment protocol is designed to collect twelve daily activities (three types) by using wearable sensors from 10 adults in smart lab of Ulster University, the experiment results show the significant performance on transitional activity recognition and achieve the overall accuracy of 94.88% on three types of activities. The results obtained by comparing with other methods and performances on other three public datasets verify the feasibility and priority of our framework. This paper also explores the effect of multiple sensors (accelerometer and gyroscope) to determine the optimal combination for activity recognition.
BibTeX
@inproceedings{ni2020leveraging,
  author = {Qin Ni and Zhuo Fan and Lei Zhang and Chris Nugent and Ian Cleland and Yuping Zhang and Nan Zhou},
  title = {{Leveraging Wearable Sensors for Human Daily Activity Recognition with Stacked Denoising Autoencoders}},
  booktitle = {Sensors},
  year = {2020},
  doi = {10.3390/s20185114},
}
Figure