TL;DR
CasePlay is a self-play reinforcement learning (RL) framework that transforms unlabeled clinical case reports into high-quality training signals for medical LLMs. By conditioning question generation on specific "knowledge points" and using a multi-dimensional rubric for feedback, it overcomes the common pitfalls of repetitive questioning and shallow reasoning, leading to significant gains across medical benchmarks.
What Problem Does This Paper Address?
Improving the medical reasoning of Large Language Models (LLMs) typically requires massive amounts of expert-labeled data or expensive synthetic pipelines. While the medical literature is full of high-quality, unlabeled clinical case reports, turning these documents into effective training data is surprisingly difficult.
The authors identify two "pitfalls" in standard document-grounded self-play:
- Knowledge-Point Fixation: A model tasked with generating questions from a report tends to "anchor" on the most obvious clues (like the final diagnosis) and ignores the subtle, information-dense journey of the clinical course.
- Difficulty-Quality Mismatch: Standard RL often rewards "intermediate difficulty" questions (where the model gets it right ~50% of the time). However, in medicine, a question can be "hard" simply because it is poorly phrased or relies on trivia, rather than requiring deep clinical integration.
Methods
CasePlay introduces a co-evolutionary loop between a Proposer (which generates questions) and a Reasoner (which solves them).
The framework features two novel components:
- Knowledge-Conditioned Proposer: Before training, the system extracts multiple "knowledge points" (KPs) from each case report (e.g., specific symptom progressions or treatment responses). During self-play, the proposer is forced to generate questions based on a specific KP using a round-robin strategy. This ensures the model explores the entire document rather than just the highlights.
- Rubric-Judge Reasoner: Instead of providing a simple binary reward (correct/incorrect), the reasoner evaluates the proposer’s questions against a rubric:
- Clue Integration: Does the question require synthesizing multiple clinical clues?
- Reasoning Depth: Does it require multi-hop clinical inference?
- Option Quality: Are the distractors plausible and discriminative?
The models are optimized using Group Relative Policy Optimization (GRPO) in an alternating fashion: the proposer is updated to create better questions, and the reasoner is updated to solve them more accurately.
Datasets
- Training: A corpus of 4,281 clinical case reports filtered from PubMed Central (PMC) published between 2020 and 2025.
- Evaluation: The authors test on seven benchmarks across three categories:
- Expert Medical Reasoning: MedXpert-Text, MedBullets (OP4/OP5), and MedQA-USMLE.
- Clinical Case Reasoning: LiveClin, which focuses on patient narratives.
- Scientific QA: MMLU-Pro (Health/Biology) and GPQA (Genetics/Molecular Biology).
Key Insights and Ideas
- Forced Exploration is Necessary: Without explicit conditioning, document coverage saturates quickly. As shown in Figure 2(a), standard methods (like SPICE) stop finding new information in reports, while CasePlay continues to increase its "document coverage ratio" by cycling through different knowledge points.
- Difficulty $\neq$ Quality: The authors found that 31.3% of "intermediate difficulty" questions generated by baselines were actually low quality (rubric score < 0.6). By rewarding the proposer for quality (clue integration and depth) alongside learnability (difficulty), CasePlay generates much more effective training examples.
- Diversity in Question Types: Conditioning on KPs naturally leads to a more balanced distribution of question types (diagnosis, treatment, mechanism, etc.), whereas unconditioned models tend to obsess over diagnosis-related questions.
Key Results
CasePlay consistently outperformed both medical reasoning models and existing self-play baselines:
- Average Gain: Improved the warm-start (SFT) model by 3.1 points on average across all benchmarks.
- LiveClin Performance: Achieved a 6.3-point gain on the LiveClin benchmark, which is particularly significant as LiveClin mimics real-world clinical narratives.
- Robustness: The method proved effective across different model scales (4B to 8B parameters) and different model families (Qwen3, OctoThinker).
- Ablation: Removing the "knowledge-point" conditioning or the "rubric-judge" feedback led to significant performance drops, confirming that both components are vital.
Limitations and Future Work
- Static Knowledge Points: Currently, KPs are extracted once before RL begins. The authors suggest that "online discovery" or refining KPs as the model improves could be a promising next step.
- Text-Only Focus: The current framework ignores images (X-rays, pathology slides) often found in case reports. Extending CasePlay to a multimodal setting is a clear direction for future research.
- Case Report Bias: Case reports often highlight "unusual" or "publication-worthy" cases rather than routine presentations, which might introduce specific biases into the trained model.
Summary updated