Modeling Human Cognition with a Hybrid Deep Reinforcement Learning Agent

Songlin Xu, Xinyu Zhang

Abstract

Predicting human cognitive reactions to dynamic environmental stressors remains a significant challenge for artificial intelligence. While classical cognitive models offer interpretability and deep learning models provide predictive power, neither fully captures the fine-grained temporal dynamics of human decision-making under pressure. This paper introduces CogReact, a novel hybrid framework that integrates the Drift-Diffusion Model (DDM) with Deep Reinforcement Learning (DRL). By simulating how fluctuating stimuli "perturb" internal evidence accumulation, CogReact achieves superior accuracy in predicting human response times and demonstrates a 10x improvement in training efficiency over pure DRL approaches.

Introduction

Human cognition is inherently dynamic, constantly influenced by external factors such as time pressure, stress, and feedback. Traditional AI models often struggle to account for these fluctuations, typically excelling only under static or ideal conditions. Existing methodologies generally fall into two categories: classical cognitive models like the Drift-Diffusion Model (DDM), which are transparent but limited in predictive capability for complex environments, and deep learning models, which act as "black boxes" with high predictive accuracy but little mechanistic insight.

The central question addressed by this research is: How can we simulate the impact of dynamic environmental stimuli on human cognitive behaviors with precision at a fine-grained level? To answer this, the authors propose CogReact, a framework designed to bridge the gap between explainable cognitive science and powerful reinforcement learning.

The CogReact Framework

CogReact employs a four-step hybrid architecture to model the decision-making process:

  1. Task Encoding: An LSTM-based agent is trained to solve specific cognitive tasks (e.g., modular arithmetic), learning to extract features that represent task difficulty.
  2. Agent-to-Human Transfer: Using Support Vector Machines (SVM), the internal features of the "math agent" are mapped to real-world human baseline performance—specifically choice and response time—under ideal, stress-free conditions.
  3. Cognitive Modeling (DDM): The framework utilizes the DDM to represent decision-making as a trajectory of evidence accumulation toward a threshold.
  4. DRL Modulation Loop: A DRL agent, utilizing Proximal Policy Optimization (PPO), observes environmental changes frame-by-frame. As stimuli like a visual time-pressure bar evolve, the DRL agent takes "actions" that perturb the DDM’s evidence accumulation process, effectively simulating how external stressors nudge the human decision process.
Overview of the CogReact framework: (a) Agent logical reasoning, (b) Transfer from agent to human, (c) Decoding reasoning via DDM, and (d) Simulating stimuli perturbation via DRL.

Experimental Evaluation

The CogReact framework was validated across three distinct cognitive domains:

  • Mathematical Reasoning: A primary dataset of 21,157 responses from 44 participants performing modular arithmetic under varying time-pressure conditions (None, Static, Random, and Rule-based).
  • Decision Making (CPC18): A benchmark involving gambling tasks where participants choose between different reward and loss probabilities.
  • Learning (PeerEdu): A dataset capturing student curiosity levels during online lectures influenced by external peer feedback.

Key Results and Insights

The hybrid approach of CogReact yields several significant advantages over traditional and pure deep learning models.

Predictive Accuracy and Generalization

CogReact achieved a Mean Average Percentage Error (MAPE) of 0.2703 in predicting response times, significantly outperforming baselines such as hGRUs, LSTMs, and Transformers, which typically ranged from 0.30 to 0.36. The framework's success across math, gambling, and educational datasets demonstrates its robust generalizability across different task types and feedback modalities.

Computational Efficiency

One of the most striking findings is the efficiency gain provided by the hybrid structure. By incorporating the DDM as a "prior" that constrains the DRL agent's search space, CogReact converged 10x faster than a "Pure DRL" approach. In testing, the hybrid model reached convergence in approximately 4.42 minutes, compared to 38.30 minutes for the pure DRL agent on identical hardware.

Performance metrics: (a-d) Response time predictions across conditions, (e) Correlation between model and human response times, (f-g) Training efficiency comparison between Pure DRL and Hybrid DRL.

Interpretability

Unlike standard neural networks, CogReact offers a "window" into the human mind. By analyzing the "action trajectories" of the DRL agent as it perturbs the DDM, researchers can pinpoint exactly when a participant's decision process was most affected by environmental stimuli. This granularity allows for a deeper understanding of the continuous influence of stressors rather than treating them as simple binary states.

Conclusion and Future Directions

CogReact represents a significant step forward in modeling human-environment interactions. By combining the strengths of cognitive modeling and reinforcement learning, it provides a tool that is both highly accurate and deeply interpretable.

Future work aims to expand the model's scope beyond response time and curiosity to include deeper measures of accuracy, attention, and anxiety. Additionally, the authors envision using CogReact to develop adaptive AI interventions that can proactively adjust user interfaces when they predict a human is becoming overwhelmed by environmental stress.

Summary updated

BibTeX
@inproceedings{xu2025modeling,
  author = {Songlin Xu and Xinyu Zhang},
  title = {{Modeling Human Cognition with a Hybrid Deep Reinforcement Learning Agent}},
  booktitle = {International Conference on Machine Learning},
  year = {2025},
  url = {https://openreview.net/forum?id=NXroN0sD8V},
}
Figure