Discovering Behavioral Predispositions in Data to Improve Human Activity Recognition

Maximilian Popko, Sebastian Bader, Stefan Lüdtke, Thomas Kirste

Abstract

The automatic, sensor-based assessment of challenging behavior of persons with dementia is an important task to support the selection of interventions. However, predicting behaviors like apathy and agitation is challenging due to the large inter- and intra-patient variability. Goal of this paper is to improve the recognition performance by making use of the observation that patients tend to show specific behaviors at certain times of the day or week. We propose to identify such segments of similar behavior via clustering the distributions of annotations of the time segments. All time segments within a cluster then consist of similar behaviors and thus indicate a behavioral predisposition (BPD). We utilize BPDs by training a classifier for each BPD. Empirically, we demonstrate that when the BPD per time segment is known, activity recognition performance can be substantially improved.

Introduction

Human Activity Recognition (HAR) in dementia care faces significant hurdles due to high inter- and intra-subject variability. Behaviors like "agitation" or "apathy" can manifest differently across individuals, and even for a single patient, activity levels fluctuate throughout the day. Traditional "one-size-fits-all" models often struggle to capture these nuances because they attempt a direct mapping between sensor data and behavior without considering context.

This paper introduces a data-driven method to identify Behavioral Predispositions (BPDs)—the tendency for individuals to exhibit certain behaviors at specific times. By automatically discovering these behavioral contexts directly from the data, the researchers aim to bridge the gap left by rigid time-based models.

Methodology

The core of the proposed approach is a three-stage pipeline designed to leverage behavioral context:

  1. Extracting BPDs via Clustering: The day is divided into time segments (e.g., 30 or 60 minutes). For each segment, a histogram representing the distribution of annotated behaviors is calculated. K-means clustering is then applied to these histograms. Each cluster represents a BPD—a specific "behavioral profile" (e.g., a cluster dominated by apathy versus one where pacing is frequent).
  2. Specialized Classification: Instead of a single global model, a separate classifier is trained for each BPD cluster. The study evaluated Naive Bayes (NB) and Support Vector Machines (SVM).
  3. Context-Aware Inference: During evaluation, the system assumes the BPD of the current time segment is known (acting as an "informed prior") and selects the corresponding specialized classifier to process raw accelerometer data.
The proposed pipeline for discovering Behavioral Predispositions (BPDs) and using them to select specialized classifiers.

This approach moves away from rigid clock-based segmentation, instead grouping time windows based on the similarity of the behavior distributions they contain.

Experimental Setup

The study utilized data from the insideDem project, involving 17 residents with moderate to severe dementia. Accelerometer data was collected from wrist and ankle bands at 50 Hz. Seven distinct behaviors (apathy, general restlessness, mannerisms, pacing, aggression, trying to get to a different place, and normal behavior) were recorded every five minutes over 29 days, resulting in approximately 17,000 annotated intervals.

Key Results and Insights

The researchers found that simple time-based partitioning performs poorly because of high daily variance. Clustering behavior distributions captures the essence of activity better than the clock. By selecting a classifier trained on a specific BPD, the model effectively "rules out" unlikely behaviors, simplifying the search space.

Recognition performance (Average F1) improves as the number of BPD clusters increases for both Naive Bayes (top) and SVM (bottom) classifiers.

Key findings include:

  • Significant Accuracy Boost: The $F_1$ score improved consistently as the number of BPD clusters increased. With 20 BPDs and 30-minute segments, the SVM reached an $F_1$ score of approximately 0.5, a substantial increase over the single-classifier baseline.
  • K-Means vs. Time-Based: The data-driven K-means approach consistently outperformed simple time-based segmentation, confirming that behavioral states are more predictive than the time of day.
  • Behavioral Overlap: Confusion matrices revealed that "normal behavior," "apathy," and "mannerisms" are frequently confused, likely due to their physical similarity (e.g., sitting still) when measured only by a wrist-worn accelerometer.

Limitations and Future Work

The primary limitation is the "Oracle" assumption—the study assumes the BPD is known during testing. In real-world applications, the system would first need to predict the BPD from sensor data. Additionally, while current BPDs are subject-specific, future work will investigate "global BPDs" applicable across different patients. Incorporating additional sensors (sound, light) or temporal models like Hidden Markov Models (HMMs) could also improve the capture of transitions between behavioral states.

Summary updated

Abstract

The automatic, sensor-based assessment of challenging behavior of persons with dementia is an important task to support the selection of interventions. However, predicting behaviors like apathy and agitation is challenging due to the large inter- and intra-patient variability. Goal of this paper is to improve the recognition performance by making use of the observation that patients tend to show specific behaviors at certain times of the day or week. We propose to identify such segments of similar behavior via clustering the distributions of annotations of the time segments. All time segments within a cluster then consist of similar behaviors and thus indicate a behavioral predisposition (BPD). We utilize BPDs by training a classifier for each BPD. Empirically, we demonstrate that when the BPD per time segment is known, activity recognition performance can be substantially improved.
BibTeX
@inproceedings{popko2022discovering,
  author = {Maximilian Popko and Sebastian Bader and Stefan Lüdtke and Thomas Kirste},
  title = {{Discovering Behavioral Predispositions in Data to Improve Human Activity Recognition}},
  booktitle = {iWOAR},
  year = {2022},
  url = {https://openreview.net/forum?id=hFcjsm3HOq},
}
Figure