GPAgentBench-2K: Benchmarking Large Language Model Agents in Complex Clinical Action Space

Boqi Chen, Xudong Liu, Yunke Ao, Heejin Do, Jianing QiuarXiv 2026 · arXiv

TL;DR

The authors introduce GPAgentBench-2K, the first Constrained Markov Decision Process (CMDP) benchmark designed to evaluate LLM agents within a realistic, six-action primary-care clinical workflow. Their evaluation of 16 state-of-the-art LLMs uncovers a severe "quality-safety gap": even frontier models with high diagnostic accuracy violate critical safety constraints in over half of high-risk cases.

What Problem Does This Paper Address?

Most existing medical AI benchmarks oversimplify clinical decision-making. They either treat diagnosis as a static, single-step prediction task (assuming all patient data is perfectly structured and available upfront) or model it as an unconstrained process with a tiny action set (e.g., only allowing the agent to "ask" or "diagnose").

In the real world, general practitioners (GPs) navigate a complex, sequential workflow. They must recursively gather evidence, order cost-effective tests, and make high-stakes branching decisions—specifically deciding whether a patient can be safely managed locally or must be urgently referred to a specialist. Crucially, real medicine operates under strict safety and resource constraints where trading patient safety for diagnostic accuracy is unacceptable.

Methods

To mirror real-world clinical workflows, the authors frame the environment as a Constrained Markov Decision Process (CMDP).

Comparison of the proposed CMDP workflow against prior fixed/fragmented workflows

Key components of their approach include:

  • Full-Spectrum Action Space: Agents can execute six distinct actions: ask, body_exam, test, diagnose, treat, and refer.
  • Topological Action Masking: To prevent invalid clinical trajectories (like prescribing treatment before a diagnosis), the environment uses state-dependent action masks designed by clinical experts.
  • Safety-Informed Abstention: Referral is treated as a first-class terminal outcome, allowing agents to safely abstain from local management when a case is too high-risk.
  • Constrained RL Baseline (C-GRPO): The authors adapt Group Relative Policy Optimization (GRPO) using a Lagrangian relaxation. Instead of treating safety violations as soft additive penalties (which unconstrained RL models often simply absorb in favor of high accuracy rewards), C-GRPO explicitly penalizes safety constraint violations during fine-tuning.
Figure 2: Overview of GPAGENTBENCH-2K. (1) Each case pairs an expert-validated GP record with a sampled patient persona; the ground-truth diagnosis and disposition is hidden from the agent. (2) Agent produces trajectories scored by diagnosis and treatment rewards (RDx, RT x) under a safety cost Csafe and a diagnostic test cost Cdiag.

Datasets

The benchmark introduces GPAgentBench-2K, which consists of 2,428 high-quality cases derived from real-world GP records across 9 primary and 62 secondary medical departments.

To ensure clinical realism, the dataset is split into two main pathways:

  1. Diagnose-and-manage: Cases safely manageable within primary care.
  2. Abstain-and-refer: Complex or high-risk cases requiring specialist escalation.

Furthermore, the patient simulator is persona-driven, augmenting the clinical data with diverse communication hurdles across four dimensions: personality (e.g., overanxious, verbose), language proficiency, memory fidelity, and cognitive disorientation.

Key Insights and Ideas

  • The Action Space Penalty: LLM performance degrades monotonically as the action space scales. Moving from a trivial 2-action setup to a realistic 6-action setup causes diagnostic accuracy to plummet by up to 21 percentage points, largely due to the massive combinatorial complexity introduced by active questioning and test ordering.
Figure 3: Impact of action space complexity on per- formance. Clinical quality degrades as parsed action set grows from a minimal 2-action set (diagnose, treat) to the full 6 actions (+ ask, test, body_exam, refer).
  • The Clinical Quality-Safety Gap: High diagnostic accuracy does not equal safe triage. Frontier models frequently exhibit a dangerous overconfidence bias—they accurately name the disease but fail to realize the patient needs immediate specialist referral, choosing instead to treat the high-risk condition locally.
  • Uncalibrated Extremes: Off-the-shelf models generally fall into two unsafe categories: over-confident models (like Claude 4.7 Opus) that aggressively treat locally and miss critical referrals, or over-cautious models (like Kimi K2.6) that refer almost everyone due to uncertainty, failing at basic local management.

Key Results

  • Frontier Model Failures: Claude Opus 4.7 achieved the highest diagnostic accuracy (70.0%) but missed 67.2% of required referrals. Similarly, GPT-5.4 missed 51.2% of required referrals.
  • Parameter Scaling Limits: Scaling up model parameters (e.g., Qwen2.5 from 7B to 72B) improved raw medical knowledge but caused unpredictable fluctuations in triage safety, sometimes doubling the rate of missed critical conditions. It also induced a "shotgun medicine" effect, where larger models over-ordered expensive diagnostic tests without improving safety profiles.
  • C-GRPO Improvements: Fine-tuning a 7B Qwen2.5 model with C-GRPO boosted diagnostic accuracy by 11.8% and improved management accuracy. However, even with explicit constraint modeling, the agent still missed 82.1% of required referrals, underscoring how difficult it is to achieve clinically acceptable safety.

Limitations and Future Work

  • English-Only Focus: The benchmark is built entirely on English-language clinical records, omitting the linguistic and cultural nuances of multilingual primary care.
  • Simulation Constraints: The environment relies on a simulated patient agent and assumes healthcare-institution-specific referral norms that might not generalize globally.
  • Academic Action Masking: The use of rigid action masks prevents parser failures but simplifies real-world clinical environments where workflow boundaries are often fuzzy.

The authors note that closing the residual safety gap in complex, partially observable clinical action spaces remains an open and vital challenge for the AI research community.

Summary updated

Abstract

Large Language Models (LLMs) show great potential as clinical agents, yet existing benchmarks reduce clinical workflows to static predictions or unconstrained Markov Decision Processes (MDPs) with coarse action sets. To address this, we introduce GPAgentBench-2K, the first Constrained MDP (CMDP) LLM-agent benchmark for primary-care clinical decision-making, constructed from expert-validated records of real-world GP encounters. Our environment models a full spectrum of six foundational clinical actions, imposes a topological workflow prior over the action space, and operationalizes safety-informed abstention as a first-class outcome. Evaluating 16 state-of-the-art LLMs reveals a significant performance degradation as the action space scales. Crucially, we uncover a clinical quality-safety gap: even frontier models with the highest diagnosis accuracy violate safety constraints in over half of high-risk cases. Finally, we establish a reference point using Constrained Group Relative Policy Optimization (C-GRPO), and show that while explicitly modeling constraints improves performance over unconstrained RL methods, it remains far from clinically acceptable safety.
BibTeX
@misc{chen2026gpagentbench,
  author = {Boqi Chen and Xudong Liu and Yunke Ao and Heejin Do and Jianing Qiu},
  title = {{GPAgentBench-2K: Benchmarking Large Language Model Agents in Complex Clinical Action Space}},
  year = {2026},
  eprint = {2608.30188},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.30188},
}
Figure