Teaching Small Medical Models When Not to Trust the Evidence

Language models asked to summarize medical studies fail in two opposite ways. Sometimes they ignore the studies in front of them and answer from memory. Sometimes they believe whatever the studies say, even when the "evidence" recommends a poison. A large evaluation of 25 models on questions drawn from Cochrane systematic reviews found that accuracy is about 92% when every source study agrees with the review's conclusion and falls to about 41% when the studies conflict. The same evaluation found that models fine-tuned on medical text do worse than general models, because they lean on memorized knowledge instead of reading the evidence (Can Large Language Models Match the Conclusions of Systematic Reviews?). A second study showed the reverse failure. When real randomized-trial evidence is altered so that a treatment is replaced by a toxic substance such as mustard gas, frontier models still follow that evidence more than 60% of the time. Telling them to be skeptical rarely changes this (Faithfulness vs. Safety: Evaluating LLM Behavior Under Counterfactual Medical Evidence).

We propose to train a small (7–8B) medical model to decide how much to trust the evidence it is given, based on explicit judgments of study quality and safety. The training signal comes from contrastive evidence pairs: two versions of the same question and the same set of studies, identical except for one change that should alter the correct answer. One version might describe a study as a 12-patient non-randomized pilot instead of a multi-center randomized trial. Another might replace a drug with a toxic compound. A model that always follows the context fails one half of each pair, and so does a model that always answers from memory. Only a model that actually reads study methods and safety cues can get both halves right. Our claim is that this pairing, not domain data or safety data alone, is what closes the gap on conflicting evidence without making the model less safe.

Core Claim

The skepticism gap and the faithfulness–safety failure are the same missing skill: deciding conditionally how much to trust each source. Prior work treats them separately. Domain fine-tuning adds knowledge, and safety fine-tuning adds refusals (MedSafetyBench; A Human-Centric Pipeline for Aligning Large Language Models with Chinese Medical Ethics). Neither teaches the model to change its answer when source quality changes.

We predict three things for a small model trained with contrastive pairs. It will beat both plain domain fine-tuning and prompted frontier models on the conflicting-evidence part of MedEvidence. It will reduce how often it follows toxic evidence on MedCounterFact without refusing plausible substitutions. It will keep the ethical-refusal behavior measured by MedSafetyBench. The distinctive, testable prediction is quality sensitivity: on held-out pairs where only study methods differ, the model's verdict should change in the right direction far more often than for any baseline.

Research Program

Source material. We start from MS^2, which pairs roughly 20,000 systematic reviews with about 470,000 cited studies (MS^2: Multi-Document Summarization of Medical Studies). Its authors report that only 53.8% of individual study conclusions match the final review conclusion. Disagreement is therefore the normal case in this data, not a rare one. For each review we extract the question in Population, Intervention, Comparator, Outcome form (PICO), the abstracts of the included studies, and the review's conclusion, collapsed to MedEvidence-style labels. Those labels cover a higher effect, a lower effect, no difference, and insufficient evidence. Any review that appears in MedEvidence, or that shares included studies with one that does, is removed before training. Otherwise the test would be contaminated.

Building the contrastive pairs. We make three kinds of pairs.

Quality pairs. A teacher model rewrites the methods of one or more studies in two directions, upgrading them (larger, randomized, low dropout) in one copy and downgrading them in the other, with results left unchanged. Both copies are rewritten by the same generator, so writing style cannot reveal which version is which. The gold label for each copy follows a transparent rule: the conclusion is whatever the studies judged adequate-quality still support. For example, if a positive effect rests only on downgraded studies, the label becomes insufficient evidence. Clinicians annotate a stratified sample of several hundred pairs to measure how often this rule is right. Pair types where agreement is low are dropped.

Safety pairs. We follow the MedCounterFact substitution scheme, which replaces the real treatment with nonce, medical, non-medical, or toxic entities. The gold behavior differs by category. For plausible or unfamiliar entities the model should follow the evidence and note the unfamiliarity. For toxic ones it should state that no evidence set justifies recommending that substance.

Conflict pairs. We change the balance of agreeing and disagreeing studies while keeping the number of studies fixed. This targets the documented habit of siding with whichever answer has more documents (Retrieval-Augmented Generation with Conflicting Evidence).

Training. Training has two stages.

Stage one: supervised fine-tuning. The model learns a structured output: a short appraisal of each study, then a map of where the studies agree and conflict, then a verdict with a certainty level. The traces are produced by a large teacher model. A checker model filters them, and only traces that reach the gold verdict for the stated reasons are kept. This reuses the guardian-filtered distillation method of the Chinese medical ethics pipeline, applied to evidence judgments rather than ethics rules. In that pipeline, a 32B checker selected traces from a 671B teacher and cut a 7B model's risk rate from 16.24% to 3.20%.

Stage two: preference optimization on the pairs. For each pair, the preferred response to one copy is its gold answer. The rejected response is the gold answer of the other copy, meaning the answer that ignores the one thing that changed. This directly penalizes quality-blindness and toxicity-blindness.

MedSafety-Improve demonstrations are mixed into both stages, so the model keeps refusing overtly unethical requests.

Base models. We train two base models. One is a general instruction-tuned 8B model. The other is Meditron-7b, a model that MedSafetyBench and MedEvidence both show to be damaged by specialization. Including it tests whether the method can repair an existing specialized model, not only avoid damaging a fresh one.

Baselines

The naive reference point is the untrained base model with all study abstracts concatenated into the prompt. This is the setting MedEvidence reports as collapsing to about 41% when studies conflict. The same model is also run with the "Skeptical" and "Expert" prompts from the faithfulness–safety study, so that prompting gets a fair chance with an identical input format.

The strongest existing reference on MedEvidence is a prompted frontier model. DeepSeek V3 and GPT-4.1 reach "roughly 60-62%" accuracy, against a time-constrained human expert baseline above 75% (Can Large Language Models Match the Conclusions of Systematic Reviews?). On MedCounterFact, the reference is frontier models' toxic-evidence adherence, reported as often exceeding 60% (Faithfulness vs. Safety). We rerun an accessible frontier model under the same prompts and the same study sets rather than relying only on the published numbers. Beating this baseline means two things for a 7–8B model. It must exceed 62% overall on MedEvidence, with the gain concentrated in the conflicting subset. It must also cut toxic adherence well below 60% while keeping adherence on nonce and medical substitutions within a few points of the base model.

The strongest structured competitor is multi-agent debate. We run MADAM-RAG, which gives each document its own agent and combines their arguments over several rounds with an aggregator (Retrieval-Augmented Generation with Conflicting Evidence). It uses the same 8B base model and the same number of forward passes per question as our model's appraisal step. On RAMDocs it reached 34.40% exact match against 32.60% for plain Llama3.3-70B-Instruct. That narrow margin suggests debate alone does not supply a sense of study quality. We also run a DrugAgent-style pipeline, a frozen model that extracts agreement and conflict followed by fixed decision rules (DrugAgent). To make its rules quality-aware, we add study design and sample size as extracted fields.

Two ablations isolate the key ingredient, and they matter most.

Stage one only. This model has the same examples and the same structured traces but no preference stage.

Plain domain fine-tuning. This model is trained on the unperturbed MS^2 review conclusions plus MedSafety-Improve, with matched token count and tuning budget.

All fine-tuned variants use the same base model, the same data volume, the same number of LoRA or full fine-tuning steps, and the same hyperparameter search budget. If the plain domain fine-tuning ablation matches the full method on the conflicting subset and on quality sensitivity, the pairing idea is wrong.

Why not just…?

Why not just use a bigger model with a skeptical prompt? MedEvidence reports diminishing returns beyond 70B parameters. It also finds that reasoning models such as o1 and DeepSeek-R1 do not consistently beat standard models of the same size on synthesis. The faithfulness–safety study found that 405B models were as easily swayed by counterfactual evidence as small ones, and that skeptical prompts rarely produced expressions of uncertainty. That study also found chain-of-thought traces that noticed a toxic entity and then glossed over it in the final answer. The frontier baseline above tests this directly on our pairs. If prompted GPT-class models show high quality sensitivity, the case for training a small model reduces to cost.

Why not just filter outputs with a domain guardrail such as ExpGuard? ExpGuard is a fine-tuned 7B moderator that reaches 93.3% prompt F1 on jargon-camouflaged attacks (ExpGuard: LLM Content Moderation in Specialized Domains). It classifies single prompt-response pairs as safe or unsafe, however. It has no way to tell that "no difference" is the wrong conclusion because the only positive trial was underpowered. It might catch some toxic-substance recommendations, so we run it as an output filter on the base model for the MedCounterFact comparison. Our expectation is that it helps on the toxic category and does nothing for accuracy on conflicting evidence. That would be a division of labor, not a replacement. If the filtered base model matches our model on toxic adherence, we will report that honestly and narrow our safety claim.

Why not just extract the numbers and run a standard meta-analysis? This is the most serious alternative. TrialMind already automates search, screening, extraction, and forest plots, with extraction accuracy 29.6–61.5% above GPT-4 (Accelerating Clinical Evidence Synthesis with Large Language Models). When abstracts report usable effect sizes, fixed statistical pooling weighted by sample size may beat any language model. We therefore run a TrialMind-style extraction step followed by random-effects pooling as a baseline. We report results separately for questions where it can extract enough numbers and for questions where it cannot. Our claim is limited to the second group, plus the safety judgments that no statistical formula makes, unless the results show otherwise.

Why not just combine domain fine-tuning with safety fine-tuning? MedSafetyBench showed that LoRA safety tuning lowers harmfulness from about 3.0 to about 1.0 while keeping MedQA near 52%. Safety can therefore be restored cheaply. That work, however, never tests whether evidence judgment improves. The matched plain domain fine-tuning ablation is exactly this recipe, and it decides whether contrastive pairs add anything.

Proposed Experiments

Headline measurements. The first headline number is MedEvidence accuracy, reported overall and broken down by the fraction of source studies that agree with the conclusion. This shows whether gains fall where the documented drop from 92% to 41% occurs. The second is MedCounterFact evidence adherence for each substitution category, which measures toxic compliance and over-refusal on the same test. We add MS^2 evidence-direction agreement, for which BART and LED agreed with the gold direction only about 50% of the time. For preserved safety and knowledge we report MedSafetyBench harmfulness and MedQA accuracy. As a transfer probe of reading comprehension under conflicting evidence, we evaluate on NLI4CT two-trial instances. There, the best reported baseline F1 is 0.644, and a baseline that sees only the statement reaches 0.626, so we report the gap over the statement-only baseline rather than raw F1.

Decisive check. The decisive check is quality sensitivity on held-out contrastive pairs built from reviews that none of the models saw. It measures the fraction of pairs on which the verdict moves in the gold direction when only methods text changes. We run this check against clinician-written pairs rather than generated ones, to rule out the possibility that the model learned artifacts of the teacher's rewriting.

Stress test. A secondary stress test reruns the CliMedBench False Information Test on the Chinese-capable variant. On that test, specialized models fell from about 47% to 8% accuracy when given false references (CliMedBench).

What would falsify the idea. The idea fails under any of three outcomes:

  • Quality sensitivity does not beat the plain domain fine-tuning ablation.
  • Gains on conflicting evidence come with higher refusal of plausible nonce substitutions.
  • The extraction-plus-meta-analysis baseline matches the model on the questions where it cannot extract numbers.