A Decision Theoretic Model for Stress Recognition and User Assistance

Wenhui Liao, Weihong Zhang, Zhiwei Zhu, Qiang JiAAAI 2005

Abstract

This paper presents a unified probabilistic framework for real-time human stress recognition and automated user assistance. Utilizing Dynamic Influence Diagrams, the model integrates multi-modal sensory data—including physiological, physical, and behavioral indicators—to infer a user's internal stress state. By framing the intervention decision as a mathematical optimization problem, the system balances the benefits of stress reduction against the costs of interruption, creating a closed-loop assistant capable of timely and effective support.

Introduction

Monitoring and mitigating human stress is a complex challenge due to its subjective nature, the inherent noise in sensory data, and its dynamic evolution over time. Traditional systems often focus solely on recognition, leaving the critical question of when and how to intervene unanswered. An ill-timed intervention can be as detrimental as no intervention at all, potentially increasing user frustration or interrupting critical tasks. This research addresses the "intervention dilemma" by treating user assistance as a decision-theoretic problem, optimizing for the highest expected utility.

The Dynamic Influence Diagram Framework

The core of the proposed system is a Dynamic Influence Diagram (ID), which extends Bayesian Networks by incorporating decision and utility nodes. This unified model allows the system to simultaneously perform diagnostic inference (recognizing stress from observations) and predictive reasoning (deciding on the best course of action).

The Dynamic Influence Diagram structure, illustrating the predictive and diagnostic components of the stress model.

The model is structured into two primary segments:

  1. Predictive Component: This part models how external variables—such as task workload, environmental context, and individual traits—influence the latent stress state.
  2. Diagnostic Component: This part maps the internal stress state to observable multi-modal features. These include physical appearance (e.g., pupil dilation, eyelid movement), physiological signals (e.g., heart rate, GSR), behavioral patterns (e.g., mouse pressure), and task performance metrics (e.g., accuracy).

To optimize resource usage, the system employs Active Sensing, selecting the most informative subset of sensors at any given time to update the probability of stress while minimizing computational overhead.

Experimental Methodology

To validate the model, the researchers conducted experiments with five subjects across multiple sessions. Participants performed cognitively demanding tasks, including mental arithmetic and memory tests. The system utilized a custom "Emotional Mouse" equipped with pressure, temperature, and GSR sensors, alongside visual sensors for eye and head tracking.

Because subjective self-reporting is often unreliable, the study used workload—defined by the frequency and complexity of tasks—as a proxy for ground-truth stress. This approach is supported by psychological theories of demand and control, providing a consistent baseline for evaluating the model's inference capabilities.

Results and Discussion

The system demonstrated a high degree of sensitivity, with 16 different measures proving statistically significant as stress indicators. A key finding was the model's ability to track the temporal dynamics of stress, including "ramp-up" and "cool-down" periods that persist even after a stressful task concludes.

Comparison between experimental workload and the system's inferred stress levels over time.

In closed-loop testing, the system successfully triggered interventions—such as adjusting task speed or providing calming stimuli—when inferred stress exceeded a threshold (approximately 0.75). These automated actions effectively reduced inferred stress levels back to a manageable range (0.3–0.4), demonstrating the practical utility of the decision-theoretic approach.

Conclusion

This research provides a robust foundation for building intelligent, affect-aware assistants. By integrating recognition and assistance into a single probabilistic framework, the model moves beyond simple detection toward meaningful human-computer collaboration. Future work will focus on expanding the model to recognize a wider array of emotional states and testing the system in more diverse, real-world environments.

Summary updated

BibTeX
@inproceedings{liao2005decision,
  author = {Wenhui Liao and Weihong Zhang and Zhiwei Zhu and Qiang Ji},
  title = {{A Decision Theoretic Model for Stress Recognition and User Assistance}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2005},
}
Figure