TL;DR
Fleming-R1 introduces a verifiable medical reasoning framework that pairs knowledge-graph-guided synthetic data with a two-stage reinforcement learning regime (RLVR). By training models to think iteratively rather than just predict answers, the 7B variant outperforms traditional 72B baselines, while the 32B model reaches near-parity with GPT-4o on the challenging MedXpertQA benchmark.
What Problem Does This Paper Address?
Large language models (LLMs) often achieve strong scores on multiple-choice medical benchmarks, but their clinical utility is severely limited by "answer without justification." In clinical decision-making, an unfaithful or opaque chain of thought is dangerous—a correct diagnostic guess supported by flawed pathophysiological reasoning can lead to fatal treatment decisions.
Existing models struggle with clinical reasoning due to three main issues:
- Static, shallow training data: Most medical datasets feature standard questions with sparse rationale supervision, largely ignoring rare diseases, niche medications, and multi-step trade-offs.
- Outcome-only optimization: Standard reinforcement learning optimizes for the correct final answer, failing to penalize intermediate diagnostic leaps or dosage errors.
- Fragile initialization: Without structured guidance at cold start, reasoning schemas easily collapse on out-of-distribution or compositionally complex cases.
Methods
Fleming-R1 addresses these challenges through a three-stage training pipeline:
- Reasoning-Oriented Data Strategy (RODS):
- Combines curated public medical question-answering datasets with synthetic data generated from an autonomous Wikipedia-derived medical knowledge graph (>100,000 entities).
- Samples multi-hop paths (e.g., symptom $\rightarrow$ pathophysiology $\rightarrow$ diagnostic test $\rightarrow$ treatment) focusing deliberately on underrepresented conditions. Plausible-but-wrong distractors are procedurally synthesized via relation-preserving perturbations.
- Chain-of-Thought (CoT) Cold Start:
- Rather than simple supervised fine-tuning, the authors distill reasoning trajectories from a high-capacity teacher model (
GPT-OSS-120B). - Uses an iterative refinement loop featuring backtracking, path exploration, and self-correction to build reasoning priors that check assumptions and correct intermediate errors.
- Two-Stage RLVR with GRPO:
- Employs Group Relative Policy Optimization (GRPO) using verifiable rewards strictly based on final answer accuracy and output formatting (avoiding length bias).
- Stage 1: Consolidates core reasoning on a balanced mix of easy and moderate cases.
- Stage 2: Employs adaptive hard-sample mining to dynamically target persistent model failure modes on difficult questions requiring multi-step inference.
Datasets
- Training: Public medical QA corpora (MedQA, MedMCQA, CMExam, PubMedQA) combined with topologically sampled subgraphs from a curated Wikipedia knowledge graph.
- Evaluation: Evaluated across nine medical benchmarks:
- MedXpertQA (Text): Complex clinical scenarios spanning 17 specialties and 11 body systems.
- Standard Medical Exams & QA: MedQA (USMLE), MedMCQA, Medbullets, PubMedQA, and CareQA (Spanish healthcare exam questions translated/rephrased).
- Clinical Consultation & Domain Understanding: JMED (21-option doctor-patient hospital dialogue benchmark) and MMLU-Pro subsets (Biology and Health).
Key Insights and Ideas
- Reasoning priors matter more than brute-force scaling: The 7B model initialized with self-correcting CoT and reinforced with RLVR outperforms several general-purpose and medical 32B/72B baselines.
- Curriculum-driven hard negative mining prevents reward sparsity: Transitioning from easy/moderate problems to an adaptive curriculum focused on repeated failures prevents reinforcement learning from plateauing when dealing with difficult multi-hop diagnostic cases.
- Knowledge graph masking creates rigorous clinical reasoning: By extracting subgraphs and masking key intermediate nodes (like lab results or treatments), models are forced to articulate disambiguating clinical evidence rather than rely on shallow surface cues.
Key Results
- Fleming-R1-7B:
- Achieved a 63.37% benchmark average, outperforming
HuatuoGPT-O1-7B(56.12%) andQwen2.5-7B(55.43%). - Surpassed the larger
Qwen2.5-32B(63.40% vs. 63.37% essentially tied overall, but beating it on MedQA: 75.10% vs. 71.56%, and Medbullets: 57.05% vs. 48.99%). - Scored 19.14% on MedXpertQA, substantially outperforming
Qwen2.5-7B-Instruct(11.96%) and beating the 72BQwen2.5-72B(18.90%). - Fleming-R1-32B:
- Achieved a 75.42% benchmark average, leading all tested open-source models in the 10B–100B tier (ahead of
Qwen3-32Bat 72.89% andHuatuoGPT-O1-72Bat 72.45%). - Reached 30.33% on MedXpertQA, achieving near parity with closed-source GPT-4o (30.37%).
Limitations and Future Work
- Evaluation is primarily multiple-choice: While benchmarks like MedXpertQA and JMED (21 options) stress reasoning, evaluation is constrained to structured MCQs rather than open-ended, multi-turn doctor-patient dialogues.
- Slight regressions on dialogue tasks: On the 21-option JMED benchmark, the 32B model exhibited a small performance dip (-0.6 pp) during Stage 2 RL, indicating that hard-sample mining on exam-style questions may occasionally cause minor domain fluctuations.
- Future Directions: Deploying verifiable reasoning into interactive clinical decision support systems and supporting real-time compliance auditing.
Summary updated