CogReact: A Reinforced Framework to Model Human Cognitive Reaction Modulated by Dynamic Intervention

Songlin Xu, Xinyu Zhang

TL;DR

CogReact is a novel hybrid framework that combines classical cognitive modeling (Drift-Diffusion Models) with Deep Reinforcement Learning to simulate how dynamic environmental stressors, like fluctuating time pressure, impact human decision-making. By integrating these two worlds, the model achieves higher accuracy in predicting human response times and trains 10x faster than standard deep learning approaches while remaining highly interpretable.

What Problem Does This Paper Address?

Most current AI models designed to simulate human behavior operate under "ideal" conditions. They assume a static environment where the relationship between a stimulus and a response is straightforward. However, real life is messy and dynamic.

Consider a student taking a math test with a ticking clock. The "time pressure" isn't just a binary state (on or off); it’s a continuous, visual stimulus that changes every second, influencing the student's internal cognitive state. Traditional models either ignore these environmental disturbances or treat them as constants. The authors of CogReact set out to bridge this gap: How can we precisely simulate the impact of dynamic, fine-grained environmental stimuli on human cognitive behavior?

Methods

The authors propose a four-step hybrid pipeline called CogReact. It moves away from "black-box" neural networks toward a more "gray-box" approach that incorporates known psychological principles.

  1. The Logical Reasoning Agent: First, they train an LSTM-based "math agent" to solve the task perfectly. This agent learns the underlying logic of the problems.
  2. Knowledge Transfer: Features from this AI agent are mapped to real human data using Support Vector Machines (SVMs). This establishes a "baseline" of how a human would perform under ideal, stress-free conditions.
  3. The Cognitive Engine (DDM): They use a Drift-Diffusion Model (DDM), a classic tool in cognitive science. DDM posits that humans make decisions by accumulating "evidence" until it hits a certain threshold.
  4. The Dynamic Intervener (DRL): This is the "secret sauce." A Deep Reinforcement Learning (DRL) agent is introduced to simulate the perturbation. As the environment changes (e.g., a progress bar fills up), the DRL agent takes "actions" that nudge the evidence accumulation process in the DDM, either speeding it up or slowing it down.

By using DRL to modulate a DDM, the framework can simulate how a specific frame of a video or a specific moment of stress changes the trajectory of a human's thought process.

Datasets

The researchers evaluated CogReact across three distinct cognitive domains:

  • Math Reasoning (New): A newly collected dataset of 21,157 responses from 44 participants solving modular arithmetic problems under various time-pressure conditions (none, static, random, and adaptive).
  • CPC18 (Decision Making): A public benchmark for human choice in gambling tasks.
  • PeerEdu (Learning): A public dataset tracking student curiosity and cognitive states while watching online lectures with peer feedback.

Key Insights and Ideas

  • The Power of Priors: Pure Deep Reinforcement Learning (Pure DRL) struggled to model the nuances of human behavior from scratch. By "seeding" the model with a Drift-Diffusion Model, the system had a psychological "prior" that made learning much more efficient.
  • Stress Adaptation: The model revealed that humans adapt quickly to static stress. However, randomly applied time pressure kept participants "regulated" longer, leading to more significant reductions in response time. CogReact was able to capture these group-level behavioral trends qualitatively better than any baseline.
  • Interpretability via Trajectories: Unlike standard neural networks, CogReact allows researchers to visualize the "action trajectory." You can literally see the moment-by-moment "nudges" the environment gives to the human's internal decision-making process.

Key Results

  • Higher Accuracy: CogReact achieved the lowest Mean Average Percentage Error (MAPE) across all tasks. In the math task, it reached a MAPE of 0.2703, significantly outperforming LSTMs, Transformers, and even "Pure" DRL agents.
  • 10x Training Speedup: Because the hybrid framework doesn't have to "re-invent the wheel" of decision-making, it converged in just 4.42 minutes, compared to 38.30 minutes for a pure DRL approach on the same hardware.
  • Generalization: The model successfully adapted to different input types (video, text, and numeric) and different cognitive measures (response time and curiosity levels), proving it isn't just a "math-only" solution.

Limitations and Future Work

While CogReact is a major step forward, the authors acknowledge several limitations:

  • Focus on Timing: The current study focuses heavily on when a human responds (response time) rather than what they choose (accuracy), primarily because the participants were instructed to prioritize accuracy.
  • Scope of Measures: Future iterations could incorporate more diverse cognitive measures, such as eye-tracking data or EEG signals, to further refine the "internal" simulation.
  • Task Complexity: While math and gambling are great benchmarks, moving toward even more complex, multi-step real-world tasks remains a challenge for future research.

Summary updated

Abstract

Using deep neural networks as computational models to simulate cognitive process can provide key insights into human behavioral dynamics. Challenges arise when environments are highly dynamic, obscuring stimulus-behavior relationships. However, the majority of current research focuses on simulating human cognitive behaviors under ideal conditions, neglecting the influence of environmental disturbances. We propose CogReact, integrating drift-diffusion with deep reinforcement learning to simulate granular effects of dynamic environmental stimuli on human cognitive process. Quantitatively, it improves cognition modelling by considering temporal effect of environmental stimuli on cognitive process and captures both subject-specific and stimuli-specific behavioural differences. Qualitatively, it captures general trends in human cognitive process under stimuli, better than baselines. Our approach is examined in diverse environmental influences on various cognitive tasks. Overall, it demonstrates a powerful, data-driven methodology to simulate, align with, and understand the vagaries of human cognitive response in dynamic contexts.
BibTeX
@inproceedings{xu2025cogreact,
  author = {Songlin Xu and Xinyu Zhang},
  title = {{CogReact: A Reinforced Framework to Model Human Cognitive Reaction Modulated by Dynamic Intervention}},
  booktitle = {International Conference on Machine Learning},
  year = {2025},
  url = {https://openreview.net/forum?id=NXroN0sD8V},
}
Figure