Scalably Enhancing the Clinical Validity of a Task Benchmark with Physician Oversight
Abstract: Modern clinical benchmarks often rely on Large Language Models (LLMs) to generate their own ground truth labels, creating a "biased teacher" problem where systemic errors are baked into evaluation sets. This paper introduces a phased stewardship protocol that combines agentic LLM workflows with targeted physician oversight to audit and refine benchmark data. Applying this process to the popular MedCalc-Bench, we identified that approximately 27% of labels were clinically inaccurate. Correcting these labels led to a significant 8.7% improvement in model performance during Reinforcement Learning (RL) fine-tuning, demonstrating that high-quality, human-verified labels are essential for aligning AI with clinical reality.
Introduction
In the domain of medical AI, benchmarks serve as the primary yardstick for measuring progress. MedCalc-Bench, for instance, evaluates a model's ability to calculate clinical scores such as the Glasgow Coma Scale. To scale benchmark creation, researchers often employ LLMs like GPT-4 to extract data and generate "correct" answers. This practice creates a "circularity" flaw: new models are trained and evaluated against labels that may inherit the specific failure modes—such as hallucinations or formula misapplications—of the generating model. If researchers use these flawed labels for training, they risk aligning AI with previous errors rather than clinical truth.
Methods
To address this, we developed a phased stewardship protocol designed to optimize the use of expert physician time through automated triage.
- Phase 1 (Automated Audit): An LLM agent (Gemini 2.5 Pro) with web-search capabilities reviewed original labels and their derivation metadata. The agent was instructed to flag cases only when a clear clinical inconsistency could be articulated.
- Phase 2 (Independent Recomputation): A separate, blinded LLM pipeline recomputed scores from raw patient notes. This pipeline was designed to avoid anchoring bias by remaining unaware of the original labels.
- Phase 3 (Physician Adjudication): Cases exhibiting high disagreement between original and recomputed labels were triaged for review by licensed physicians to establish a "gold standard."
To evaluate the impact of label quality, we conducted a controlled RL experiment using Group Relative Policy Optimization (GRPO) to fine-tune two versions of Qwen3-8B: one on the original noisy labels and one on the cleaned dataset.
Key Results
Our audit revealed a high error rate in the original benchmark, with 26.6% of test cases flagged as clinically suspect and 27.3% likely mislabeled based on independent recomputation. In a head-to-head validation on 50 contentious cases, our recomputed labels achieved 74% agreement with physicians, whereas the original MedCalc-Bench labels agreed only 20% of the time.
The impact on training was substantial. The model fine-tuned on cleaned labels achieved a test accuracy of 71.4%, an 8.7% absolute improvement over the model trained on original labels (62.6%). This suggests that label noise in clinical benchmarks is not merely stochastic but introduces systematic distortions in model alignment.
Discussion and Conclusion
A key insight from this work is the importance of the "N/A" safety guardrail. Many clinical notes lack sufficient information for specific calculations; forcing a numeric answer in these cases is clinically dangerous. We advocate for benchmarks to include abstention labels and to be treated as "living documents" that evolve through continuous auditing.
While physician time is expensive, our phased approach demonstrates that agentic workflows can effectively scale expert oversight. This ensures that medical AI is evaluated against clinical reality rather than model-generated artifacts, preventing the propagation of errors in the next generation of clinical models.
Limitations
The physician-verified "gold standard" was limited to 50 cases due to the high cost of expert time. Additionally, while we used different LLMs and blinding techniques, there remains a theoretical risk of shared "blind spots" between different models used in the audit process. Future work should focus on expanding human-in-the-loop verification and developing robust rewards for model abstention.
Summary updated