Inferring Human Activity in Mobile Devices by Computing Multiple Contexts

Ruizhi Chen, Tianxing Chu, Keqiang Liu, Jingbin Liu, Yuwei Chen2015 · DOI

TL;DR

This paper introduces a novel framework for inferring human activities using mobile devices by integrating multiple contextual data sources beyond just location. By combining spatial, temporal, spatiotemporal, and user contexts, their system achieves a significantly higher activity recognition accuracy (88.8%) compared to location-only approaches (61.7%), demonstrating the power of a holistic contextual understanding.

What Problem Does This Paper Address?

Modern smartphones are powerful sensing platforms, enabling a new generation of context-aware applications. While location-aware applications are common, understanding what a user is doing requires more than just knowing where they are. For instance, both a waitress and a customer might be in a coffee shop, but their activities ("working" vs. "taking a coffee break") are distinct. Traditional location-only approaches struggle to differentiate these activities, especially when multiple activities can occur at the same location. This paper addresses the challenge of accurately inferring human activities by developing a multi-context framework that leverages a richer set of information available from mobile devices.

Methods

The core of the proposed framework is a Naïve Bayes classifier that infers human activities based on a "contextual tuple." This tuple is computed on a per-second basis and comprises:

  • Spatial Context: The user's current significant location, defined as a geofence (polygon). This is determined using a hybrid indoor/outdoor positioning solution combining GPS, motion sensors, inertial sensors, and WiFi signals via an unscented Kalman filter (UKF).
  • Temporal Context: The local time, obtained directly from the smartphone's system clock.
  • Spatiotemporal Context: The dwelling length at a significant location, calculated within a 60-second sliding window. This helps distinguish short visits from longer stays.
  • User Context: Information about the user's state, including mobility patterns (static, walking, running, driving). This is derived from accelerometer data.

The framework's activity inference engine uses these contextual tuples to compute the posterior probability of various activities. The probability model can be initialized empirically (based on general knowledge) and then refined through self-learning or trained on labeled data.

Datasets

The study utilized a large dataset collected over one week from three participants using Samsung smartphones on the Texas A&M University Corpus Christi campus. This dataset comprises 710,436 contextual tuples, with each second of activity labeled. This dataset serves as a benchmark for evaluating the proposed multi-context approach against simpler methods.

Key Insights and Ideas

  • Beyond Location: The most significant insight is that human activity inference is dramatically improved by incorporating multiple contextual dimensions beyond just spatial location. Temporal, spatiotemporal, and user mobility contexts provide crucial disambiguation.
  • Dwelling Length Matters: The spatiotemporal context, specifically how long a user stays in a location, is a strong indicator of activity. A brief stop might be incidental, while a longer duration suggests a specific activity.
  • User Mobility is Key: Differentiating user movement patterns (e.g., static vs. walking vs. driving) is essential for distinguishing activities that might occur in the same place.
  • Real-World Data is Crucial: The experiment was conducted in a realistic campus environment, with participants carrying smartphones in their pockets, which introduces challenges like signal attenuation but provides more authentic data.

Key Results

The paper demonstrates a significant performance improvement with the multi-context approach:

  • Spatial-Context-Only Solution: Achieved an average classification accuracy of 61.7%.
  • Multi-Context Solution: Achieved an average classification accuracy of 88.8%.

This represents a substantial increase in accuracy, highlighting the effectiveness of the proposed framework. The supervised learning approach also significantly outperformed the unsupervised approach, with supervised solutions achieving an average accuracy of 88.8% compared to 58.0% for unsupervised.

Limitations and Future Work

Limitations:

  • Positioning Accuracy in Pockets: Carrying smartphones in pockets, while realistic, leads to lower positioning accuracy due to signal blockage, impacting the accuracy of spatial and spatiotemporal contexts.
  • Limited User Contexts: The initial study focused primarily on user mobility contexts. Other user contexts (psychological, environmental, social) were not deeply explored.
  • Empirical Model Initialization: While an empirical model can be used as a starting point, its effectiveness depends on how well it aligns with individual user patterns.

Future Work:

  • Broader User Contexts: Incorporate more user contexts, such as audio signals, ambient light intensity, and temperature, to further enhance activity inference accuracy.
  • Advanced Machine Learning: Explore more sophisticated machine learning algorithms beyond Naïve Bayes, potentially including hidden Markov models or deep learning, for improved model training and inference.
  • Incremental Learning: Further develop the framework's ability to adapt and learn incrementally from new data and user behaviors.

Summary updated

BibTeX
@misc{chen2015inferring,
  author = {Ruizhi Chen and Tianxing Chu and Keqiang Liu and Jingbin Liu and Yuwei Chen},
  title = {{Inferring Human Activity in Mobile Devices by Computing Multiple Contexts}},
  year = {2015},
  doi = {10.3390/s150921219},
}

Loading related content...

Figure