Discovering Behavioural Predispositions in Data to Improve Human Activity Recognition

Maximilian Popko, Sebastian Bader, Stefan Lüdtke, Thomas Kirste

TL;DR

This paper introduces a data-driven method to identify "Behavioral Predispositions" (BPDs)—the tendency for individuals, specifically those with dementia, to exhibit certain behaviors at specific times. By clustering these behavioral patterns and training specialized classifiers for each, the researchers demonstrate that human activity recognition (HAR) performance can be significantly improved when the behavioral context is known.

What Problem Does This Paper Address?

Human Activity Recognition (HAR) is notoriously difficult in the context of dementia care. Patients exhibit high inter- and intra-subject variability; what looks like "agitation" in one person might look like "normal movement" in another, and a single patient’s activity levels can fluctuate wildly throughout the day.

Traditional HAR models often struggle because they try to find a "one-size-fits-all" mapping between sensor data and behavior. While some researchers have tried using external knowledge (like the time of day) to provide context, these methods often fail because behaviors like "sundowning" (increased restlessness in the evening) don't happen at the exact same time every day. The authors aim to bridge this gap by automatically discovering these behavioral "moods" or contexts—which they call Behavioral Predispositions (BPDs)—directly from the data.

Methods

The authors propose a three-stage pipeline to leverage behavioral context:

  1. Extracting BPDs via Clustering: The day is divided into time segments (e.g., 30 or 60 minutes). For each segment, they calculate a histogram representing the distribution of annotated behaviors. They then use K-means clustering on these histograms. Each cluster represents a BPD—a specific "behavioral profile" (e.g., a cluster where "apathy" is dominant vs. one where "pacing" is frequent).
  2. Specialized Classification: Instead of training one massive model, they train a separate classifier for each BPD cluster. They tested three types: Majority Vote (baseline), Naive Bayes, and Support Vector Machines (SVM).
  3. Context-Aware Inference: During evaluation, they assume the BPD of the current time segment is known (acting as an "informed prior"). The system selects the classifier specifically trained for that BPD to process the raw accelerometer data.

This approach is novel because it moves away from rigid, time-based segmentation and instead groups time windows based on the similarity of the behavior distributions they contain.

Datasets

The study utilizes data from the insideDem project:

  • Participants: 17 residents (11 women, 6 men) with moderate to severe dementia across two nursing homes.
  • Sensors: Accelerometer data from wristbands (dominant hand) and ankle bands, sampled at 50 Hz.
  • Annotations: Seven distinct behaviors (apathy, general restlessness, mannerisms, pacing, aggression, trying to get to a different place, and normal behavior) recorded every five minutes over 29 days.
  • Scale: Approximately 17,000 annotated five-minute intervals.

Key Insights and Ideas

  • The "Aha" Moment for Clustering: Simple time-based partitioning (e.g., "always use the 10 AM classifier") performs poorly because of high daily variance. Clustering behavior distributions (histograms) captures the essence of the activity better than the clock does.
  • Simplifying the Search Space: By selecting a classifier trained on a specific BPD, the model effectively "rules out" unlikely behaviors. If a BPD cluster is 90% apathy, the specialized classifier doesn't have to work as hard to distinguish apathy from rare events like aggression.
  • Behavioral Overlap: The confusion matrices revealed that "normal behavior," "apathy," and "mannerisms" are frequently confused. The authors suggest this is because, in a nursing home setting, these behaviors are physically similar (e.g., sitting still or small repetitive movements) and difficult to distinguish using only a wrist-worn accelerometer.

Key Results

  • Significant Accuracy Boost: The $F_1$ score improved as the number of BPD clusters increased. With 20 BPDs and 30-minute segments, the SVM reached an $F_1$ score of approximately 0.5, a substantial increase over the single-classifier baseline.
  • K-Means vs. Time-Based: The data-driven K-means approach consistently outperformed simple time-based segmentation. This confirms that behavioral states are more predictive than the time of day.
  • Classifier Performance: While the Majority Vote classifier improved with more clusters (as clusters became more "pure"), the SVM remained the superior choice for larger segment lengths, showing it could still extract meaningful features from the sensor data even within a biased BPD.

Limitations and Future Work

  • The "Oracle" Assumption: The biggest limitation is that the study assumes the BPD is known during testing. In a real-world deployment, the system would first need to predict the BPD from sensor data before choosing the classifier.
  • Single-Subject Models: Currently, BPDs are constructed for individuals. Future work needs to investigate "global BPDs" that could work across different patients.
  • Sensor Scope: The study only used acceleration data. Incorporating other sensors (like sound or light levels) or using more sophisticated models like Hidden Markov Models (HMMs) or Latent Dirichlet Allocation (LDA) could better capture the temporal transitions between behavioral states.

Summary updated

Abstract

The automatic, sensor-based assessment of challenging behavior of persons with dementia is an important task to support the selection of interventions. However, predicting behaviors like apathy and agitation is challenging due to the large inter- and intra-patient variability. Goal of this paper is to improve the recognition performance by making use of the observation that patients tend to show specific behaviors at certain times of the day or week. We propose to identify such segments of similar behavior via clustering the distributions of annotations of the time segments. All time segments within a cluster then consist of similar behaviors and thus indicate a behavioral predisposition (BPD). We utilize BPDs by training a classifier for each BPD. Empirically, we demonstrate that when the BPD per time segment is known, activity recognition performance can be substantially improved.
BibTeX
@inproceedings{popko2022discovering,
  author = {Maximilian Popko and Sebastian Bader and Stefan Lüdtke and Thomas Kirste},
  title = {{Discovering Behavioural Predispositions in Data to Improve Human Activity Recognition}},
  booktitle = {iWOAR},
  year = {2022},
  url = {https://openreview.net/forum?id=hFcjsm3HOq},
}

Loading related content...

Figure