Human Activity Recognition using Recurrent Neural Networks

Deepika Singh, Erinc Merdivan, Ismini Psychoula, Johannes Kropf, Sten Hanke, Matthieu Geist, Andreas Holzinger

Abstract

Human activity recognition using smart home sensors is one of the bases of ubiquitous computing in smart environments and a topic undergoing intense research in the field of ambient assisted living. The increasingly large amount of data sets calls for machine learning methods. In this paper, we introduce a deep learning model that learns to classify human activities without using any prior knowledge. For this purpose, a Long Short Term Memory (LSTM) Recurrent Neural Network was applied to three real world smart home datasets. The results of these experiments show that the proposed approach outperforms the existing ones in terms of accuracy and performance.

Introduction

Human Activity Recognition (HAR) is a foundational technology for smart homes and Ambient Assisted Living (AAL), enabling systems to monitor and assist residents. However, data from ubiquitous sensors like motion detectors or door switches is often noisy and ambiguous. Traditional probabilistic models such as Naive Bayes, Hidden Markov Models (HMMs), and Conditional Random Fields (CRFs) have been used to classify these activities. While these models are established, they face significant hurdles: Naive Bayes ignores temporal context, while HMMs and CRFs often require extensive manual feature engineering and struggle with long-term temporal dependencies. This study proposes a deep learning approach using Long Short-Term Memory (LSTM) networks to automatically learn temporal representations directly from raw sensor data.

Methods

The research employs a Long Short-Term Memory (LSTM) network, a specialized Recurrent Neural Network (RNN) architecture designed to capture long-range dependencies in sequence data while mitigating the vanishing gradient problem.

The internal architecture of an LSTM cell, featuring input, forget, and output gates to regulate information flow.

Unlike traditional methods that rely on handcrafted features from sliding windows, the LSTM is fed sensor data directly. The study evaluates two data representations:

  1. Raw sensor data: Direct binary outputs (0 or 1) from sensors.
  2. Last-fired sensor data: A representation where a triggered sensor maintains a state of "1" until a different sensor changes state, providing a basic form of memory within the input itself.

The implementation utilized TensorFlow with a time slice of 70, a hidden state size of 300, and the Adam optimizer.

Datasets

The model was tested on three benchmark smart home datasets (Houses A, B, and C) originally published by Kasteren et al. These environments featured single users monitored over 14 to 25 days using 14 to 23 binary sensors (e.g., PIR motion detectors, reed switches, pressure sensors). Activities such as cooking, bathing, and sleeping were annotated by users to provide ground truth labels.

Results and Discussion

The LSTM demonstrated a significant advantage when processing raw, un-engineered data. It achieved accuracies of 89.8% for House A, 85.7% for House B, and 64.2% for House C. Notably, in the noisy environment of House C, the LSTM outperformed the next best model (CRF) by approximately 40% on raw data.

When evaluated on "last-fired" data, which inherently includes some temporal context, the LSTM remained highly competitive, matching the performance of top-tier traditional models like CRFs.

Comparison of activity recognition accuracy across different models for the 'last-fired' data representation.

The results indicate that LSTMs effectively eliminate the need for manual feature extraction, saving significant engineering effort. While training required approximately 30 minutes per day of data on a GPU, the resulting models are capable of real-time inference on standard hardware.

Conclusion

This work proves that LSTMs can effectively classify human activities in smart homes using raw sensor data, bypassing the need for tedious manual feature extraction. While the model excels at capturing temporal dependencies, future work should address model uncertainty. The authors suggest exploring Bayesian Neural Networks to provide confidence intervals for predictions, which is critical for healthcare applications in assisted living environments.

Summary updated

Abstract

Human activity recognition using smart home sensors is one of the bases of ubiquitous computing in smart environments and a topic undergoing intense research in the field of ambient assisted living. The increasingly large amount of data sets calls for machine learning methods. In this paper, we introduce a deep learning model that learns to classify human activities without using any prior knowledge. For this purpose, a Long Short Term Memory (LSTM) Recurrent Neural Network was applied to three real world smart home datasets. The results of these experiments show that the proposed approach outperforms the existing ones in terms of accuracy and performance.
BibTeX
@inproceedings{singh2017human,
  author = {Deepika Singh and Erinc Merdivan and Ismini Psychoula and Johannes Kropf and Sten Hanke and Matthieu Geist and Andreas Holzinger},
  title = {{Human Activity Recognition using Recurrent Neural Networks}},
  booktitle = {CD-MAKE},
  year = {2017},
  url = {https://openreview.net/forum?id=QcgTKy3KOfr},
}
Figure