Do Clinical Language Models Read the Patient or the Population?

When a large language model writes a summary of a patient's record, two sources can produce each sentence: the record in front of it, and what it learned in training about patients "like this." In ordinary cases the two agree, so a summary can look perfectly faithful while the record contributed almost nothing. The two diverge only for patients who differ from the textbook population: an unusual stage, an atypical lab profile, or a chronically abnormal baseline. In those cases a model that leans on population knowledge will confidently write down the typical value instead of the true one. Neuroscience has already learned a version of this lesson from the opposite direction. Findings from epilepsy patients do not transfer cleanly to healthy brains, and interventions tuned on healthy volunteers degrade in patients with remodeled circuits. Our position is that clinical language models carry the same population mismatch inside their parameters, and that current faithfulness metrics cannot see it.

We propose to measure this failure directly. We take real patient records and make controlled, clinically coherent edits that move one fact away from the population norm. We then ask whether the model's output follows the edit or reverts to the prior. Because the patient and the task stay fixed and only the evidence-prior agreement changes, this test separates "the model read the record" from "the model knew the answer anyway." The test yields a curve of how often the model echoes the prior as a function of how atypical the patient is. That curve is the quantity a deployment decision actually needs, because atypical patients are the ones harmed.

Core Claim

Grounded-support rates and related metrics overstate how faithful clinical language models are. The overstatement grows systematically with how far a patient sits from the population the model absorbed in training. A counterfactual evidence-conflict test exposes the gap, and so does a cheap detector built on it. The detector flags a claim when the model's output matches what it says with no evidence at all, despite conflicting evidence in the record. That detector catches more of these errors at a fixed coverage than probability thresholds or general claim verifiers.

The closest existing work is The Hallucination Dependence Index. It compares unsupported-claim rates with and without a patient evidence bundle on 119 TCGA-LUAD lung adenocarcinoma cases. It found a ranking inversion: Gemini-2.5-flash had 93.2% grounded support yet suppressed fewer confabulations than Gemini-3-flash-preview at 81.9%. It also separated overt confabulation from calibrated refusal, with GPT-5.4-mini at an index of 0.336. That paper motivates this proposal but cannot answer our question. Its grounded condition uses real records, in which evidence and prior mostly agree, so a claim the model would have made anyway counts as supported. Our edits create disagreement on purpose. The index also gives a single number per model, whereas we predict a dependence on patient atypicality, which is the population-generalizability question HDI leaves open.

Research Program

The first component is an edit protocol for three record types that span static and continuous data. Static oncology bundles are rebuilt from TCGA-LUAD, following the HDI construction. These contain staging, pathology text and RNA expression values. Narrative case reports come from PMC-Patients as used in MHB, which contributes the MHB-Report split. Continuous home-monitoring time series come from TIHM-1.5, the dementia cohort of 56 people and 2,803 patient-days benchmarked in Benchmarking LLM Summaries of Multimodal Clinical Time Series.

For each record, we choose a target fact and edit it to a value at a controlled percentile of the cohort distribution. Every dependent field is changed with it, so the record stays internally consistent; for example, a stage change also moves the matching nodal and metastasis descriptors. A physician panel then rates each edited record as plausible or implausible. This step matters. If an edit makes the record self-contradictory, reverting to the prior or flagging the conflict may be the correct response. Beyond Idealized Patients showed that hardened models already "correct" valid patient statements too often, and we do not want to reward models for ignoring true evidence. We therefore keep two edit families. Consistent-atypical edits should be followed. Deliberately inconsistent edits should be flagged.

The time-series arm adds a distinction the static arms cannot make. We inject or remove events, such as a sustained heart-rate rise or a night of disrupted sleep, measured against two reference points: the patient's own baseline and the cohort's baseline. Some TIHM participants have chronically atypical vitals. For them, a model reasoning from population norms will both invent abnormal events on their ordinary days and miss real deviations from their personal baseline. This mirrors the confound described in The development of aperiodic neural activity in the human brain, where background spectra shift with age and pathology and distort group comparisons. It also suggests one explanation for the near-zero abnormality recall that zero-shot models show on TIHM: the "fluency illusion," in which summaries read well but miss events. That benchmark did not test whether population priors cause it, and our design does.

The second component is the measurement. For each edited fact $f$, we label the model's statement about it as following the edit, echoing the original prior-typical value, or other (omitted or invented). We define the prior echo rate at atypicality level $a$ as

$$E(a) = \frac{\#\{\text{edited facts at level } a \text{ stated with the unedited, prior-typical value}\}}{\#\{\text{edited facts at level } a \text{ that the output mentions}\}}.$$

Here $a$ is the cohort percentile distance of the edited value from the cohort median. For the time series, $a$ is computed separately against the personal baseline and the cohort baseline. Labels come from a judge model, checked against physician adjudication on a stratified sample. HDI did the same with 300 claims, and we copy that design.

The third component is a detector called prior-echo flagging. For each claim, we compare its content with the model's output in the HDI-style ungrounded condition, which gives the model only a case identifier and cohort name. A claim is flagged if it matches the ungrounded output and the record contains a field that conflicts with it. The detector is training-free and needs only one extra generation per case. For open-weight models, we add a second signal: the attention-based attribution of Cite-While-You-Generate. That method traces generated sentences to source spans through decoder cross-attention and reaches 76.33 Macro-F1, compared with 40.06 for self-attribution. We use it to ask whether echoed claims attend to the conflicting field at all. We then feed both signals into a rejector trained with the surrogate loss of Learning to Reject with a Fixed Predictor, which leaves the generator frozen.

The models are those HDI studied (GPT-4o-mini, GPT-5.4-mini, Gemini-2.5-flash, Gemini-3-flash-preview), plus open models (Llama-3-8B and Qwen2.5-VL) whose internals we can inspect. For the time-series arm, we also include the vision-rendered pipeline from the TIHM benchmark.

Baselines

The naive reference point is the unedited evaluation: grounded-support rate on the original records, as HDI reports it. Under our hypothesis this number looks good and stays flat, while echo rate rises with atypicality. A model ranking based on it should therefore disagree with a ranking based on $E(a)$ at high atypicality. Reproducing HDI's reported 93.2% versus 81.9% grounded support for the two Gemini models on our rebuilt bundles serves as a sanity check that our pipeline matches theirs.

The strongest existing diagnostic is the HDI itself, reported at 0.336 to 0.984 across four models. It is run on exactly the same cases, models and judge as our test. Our test beats it if it predicts physician-adjudicated errors on atypical patients that HDI does not, measured as partial correlation with adjudicated error after controlling for HDI.

For the detector, the comparisons run on identical claims, with tuning on the same development split and the same compute. The first comparison is MaxProb, which abstains when token probability is low. In the decontextualization study, it kept about 64.4% coverage at half the error rate, against 90.6% for the trained rejector. The second is that trained rejector using only text and probability features. The third is a claim verifier that checks each claim against the bundle; for this role we use MediHallDetector from Detecting and Evaluating Medical Hallucinations in Large Vision Language Models where its modality applies, and an LLM judge otherwise. For the time-series arm, the strongest reported pipeline is the vision-rendered Gemini-2.5-Pro summarizer, at 45.7% abnormality recall; statistical conditioning with Llama-3-8B reaches 33.3%. No reported result was found on Lacuna for any model under evidence-prior conflict edits, so every conflict number here will be new.

The detector beats these baselines if it reaches higher recall of echoed errors at matched coverage on consistent-atypical edits, and does not produce more false flags on unedited records. The key ablation removes the ungrounded-comparison signal and keeps only attention and probability features. If performance does not drop, the "prior echo" idea is not doing the work.

Why not just…?

Why not just use the Hallucination Dependence Index as it stands? HDI measures how much evidence reduces confabulation in general. It cannot tell whether a supported claim was read or recalled, because in real records the two coincide. Our consistent-atypical edits are the experiment that decides this. If HDI and $E(a)$ rank models identically and predict the same adjudicated errors, our test adds nothing, and we will report that.

Why not just stratify ordinary accuracy by rare versus common cases? Rare cases differ from common ones in many ways: harder reasoning, sparser records, different specialties. A drop in accuracy on them mixes difficulty with prior intrusion. Editing a single fact within a fixed record holds everything else constant. The stratified analysis is still run as a comparison, and the counterfactual design should explain part of the rare-case gap, not all of it.

Why not just check every claim against the record with a verifier? This is the real threat. A verifier with full access to the bundle should, in principle, catch any claim that contradicts it, whatever the cause. Our concern is that verifiers are themselves language models with the same population priors. They may accept a typical-sounding claim even when the record says otherwise, just as Beyond Idealized Patients found that self-review fails because the priors behind the error also blind the reviewer. We test this directly by measuring verifier miss rates as a function of $a$. If a verifier's miss rate stays flat across atypicality and matches our detector's recall, the right recommendation is "use a verifier," and our contribution narrows to the diagnostic curve.

Why not just render signals as plots, or retrieve more context? Vision rendering is the best reported fix on TIHM, and MedCite shows that statement-level retrieval improves citation quality over plain retrieval. Both change what the model sees. Neither changes whether the model trusts what it sees over what it expects. We run the vision pipeline under the time-series edits. If its echo rate against personal baselines stays near zero, rendering alone solves the continuous-data case.

Proposed Experiments

The first decisive check is whether $E(a)$ rises with atypicality on consistent edits that physicians rated plausible, in all three record types. A flat curve would falsify the core claim.

The second is whether models' rankings by $E(a)$ at high atypicality diverge from their rankings by grounded support and by HDI. Divergence would show that existing metrics miss this failure.

The third, in TIHM, is whether invented and missed events depend on distance from the cohort baseline rather than from the patient's own baseline. That result would show that models import population norms into individuals.

The fourth is whether prior-echo flagging beats MaxProb, the text-only rejector and the verifier at matched coverage. A further requirement is that it not over-flag on unedited records or on correct handling of inconsistent edits; those are the "unnecessary corrections" failure that safety prompting is known to cause.

Physician adjudication on a stratified sample anchors all four checks. If the judge's labels and the physicians' labels disagree on echoed claims, that disagreement is itself evidence for the verifier concern above.