Minimal-Pair Chest X-rays as Preference Data for Grounded Report Generation

Medical vision-language models write fluent radiology reports that often describe findings the image does not show. They are leaning on textbook text patterns instead of the pixels. The usual fix is preference tuning, which shows the model a preferred and a dispreferred answer and pushes it toward the former. The dispreferred answers are typically manufactured. A large language model plants a plausible error in the true report, or a detector locates the lesion so it can be blurred out. We propose to take preference pairs from the archive instead of manufacturing them. A minimal pair is two real chest X-rays whose reports, once parsed into a structured graph of clinical findings, differ by exactly one finding, for example "small left effusion" versus "no effusion". Each image's report is then the preferred answer for that image and the dispreferred answer for the other.

The two reports in each pair are equally plausible as text. They are written by radiologists in the same style, and only the image can say which one is correct. This addresses three problems at once. A model cannot win these comparisons using language priors, because the text alone does not separate the answers. No detector, segmentation mask, or proprietary teacher is needed. And because each comparison holds the image fixed, the pairs satisfy the symmetric preference objective that SymMPO showed is required for the training objective to be theoretically sound. In this design, preference data generation is the mechanism, the gap between visual and textual representations is the failure being targeted, hallucination is what gets measured, and radiology is the setting whose domain knowledge, encoded in a clinical schema, defines what counts as a minimal difference.

Core Claim

Synthetic dispreferred answers built by editing text teach a model to spot implausible text, not to look at the image. Our central diagnostic follows from this. Score a pair of answers with the image replaced by a blank, and see whether the model still prefers the chosen one. If it does, the pair carries a text-only shortcut. We predict two things. First, perturbation-based negatives from text edits or image noise will show a large blank-image preference margin, while radiology-graph minimal pairs will show almost none. Second, training on minimal pairs will reduce hallucinated findings on MIMIC-CXR and on grounded medical VQA more than an equal number of perturbation pairs, and most of the gain will appear on counterfactual tests where the image changes but the plausible text does not.

Research Program

The pair miner uses RadGraph. RadGraph is a schema of anatomy and observation entities linked by relations such as "located at" and "suggestive of". Its extractor reaches micro F1 of 0.94 for entities and 0.82 for relations on MIMIC-CXR, and its schema covers 87–92% of report sentences. We run the extractor over all MIMIC-CXR reports. Each study is collapsed into a set of normalized findings, where a finding is an observation, its anatomical location, and its presence status (present, absent, or uncertain).

Two studies form a candidate pair when four conditions hold:

  • they share the same view;
  • their finding sets differ by exactly one edit (adding, removing, relocating, or flipping the presence of one finding);
  • their image embeddings are close, so the pair is not separated mainly by gross differences such as devices or patient habitus;
  • they come from different patients.

Real reports differ in phrasing well beyond the one finding. We therefore build the dispreferred text by editing rather than by using the partner's whole report. We take image $m$'s own report $y$ and replace only the sentence carrying the differing finding with the corresponding sentence from partner $m'$'s report. This yields $\tilde y$, which differs from $y$ in one sentence of real radiologist prose, and $\tilde y'$ is built the same way for $m'$.

Training uses the symmetric form of direct preference optimization (DPO, which fits a policy directly to preference pairs without a separate reward model). Here $\pi_\theta$ is the model being trained, $\pi_{\text{ref}}$ is the frozen starting model, $\beta$ is a temperature, and $\sigma$ is the logistic function:

$$ \mathcal{L} = -\log\sigma\!\Big(\beta\big[\log\tfrac{\pi_\theta(y\mid m)}{\pi_{\text{ref}}(y\mid m)} - \log\tfrac{\pi_\theta(\tilde y\mid m)}{\pi_{\text{ref}}(\tilde y\mid m)}\big]\Big) - \log\sigma\!\Big(\beta\big[\log\tfrac{\pi_\theta(y'\mid m')}{\pi_{\text{ref}}(y'\mid m')} - \log\tfrac{\pi_\theta(\tilde y'\mid m')}{\pi_{\text{ref}}(\tilde y'\mid m')}\big]\Big). $$

Each term conditions both answers on the same image, so the normalizing constant cancels as SymMPO requires. Image-corruption negatives, such as the lesion noising in MMedPO, compare across different images and do not cancel.

The analytic part of the program characterizes the mined data. We will measure:

  • how many exact one-edit pairs exist per finding type;
  • how coverage falls off for rare findings;
  • how much RadGraph extraction error and radiologist inconsistency contaminate labels, estimated by having a radiologist audit a sample of a few hundred pairs.

Where exact pairs are scarce, we test a relaxed variant that accepts two-edit pairs and replaces both sentences. This tells us whether the method depends on strict minimality.

The approach is positioned against several lines of work.

Three medical systems supply negatives by perturbation:

  • MMedPO uses GPT-4o-injected errors, plus noise placed in lesions found by the MedKLIP detector, with debate-derived severity weights.
  • EAGLE uses pathology-encoder masking and entity swaps found by medical named-entity recognition.
  • MINT uses predictions from upstream expert models.

All three create the dispreferred answer artificially. Ours is a real report about a real, different patient. EAGLE's term swap is the closest idea, and we treat it as a central ablation rather than a strawman.

General-domain methods offer two contrasts:

  • HDPO uses attention and truncation heuristics. It motivates our work, because its negatives target known failure modes but are still generated from the same image.
  • HA-DPO identified style bias between chosen and rejected answers and fixed it by rewriting both with GPT-4 using Visual Genome boxes. We get style matching for free from sentence-level splicing and need no boxes.

SymMPO supplies the objective. It finds similar images by generic visual similarity, with captions synthesized by open models. Our contribution is to define "similar but differently correct" through a domain knowledge schema, so that each pair's difference is exactly one clinical fact. Localizing Before Answering and Thinking Like a Radiologist motivate the problem: hallucination follows from failing to look at the relevant region. Both, however, need mask supervision or slow interleaved image-token generation, about 2 minutes per case. We test whether pair construction alone can induce looking.

Baselines

The naive reference point is the base medical vision-language model after supervised fine-tuning on MIMIC-CXR reports. We also include a stronger form of "do nothing more": continued supervised fine-tuning on the chosen halves of our pairs, with the same number of tokens. This separates the effect of contrast from the effect of extra in-domain text.

The strongest existing method is MMedPO, which combines GPT-4o-injected plausible errors, MedKLIP lesion noising, and severity weighting. The file reports it only as relative gains: a 26.0% report-generation improvement on MIMIC-CXR, a 61.9% gain on IU-Xray, and a 14.2% average factual-accuracy gain on SLAKE and VQA-RAD. No absolute MIMIC-CXR clinical F1 for MMedPO was found on Lacuna. The most concrete absolute anchor on MIMIC-CXR is the multi-objective preference method of Radiology Report Generation via Multi-objective Preference Optimization, at BLEU-4 0.139 and clinical F1 0.353. We will rerun MMedPO's released pipeline, or a faithful reimplementation, on our base model rather than compare against its reported numbers.

The other comparators are:

  • an EAGLE-style entity-swap negative built from the same reports;
  • SymMPO with nearest-neighbor image pairs chosen by embedding similarity alone, with no graph constraint;
  • an HDPO-style heuristic negative set.

The SymMPO-without-graph comparator is the ablation that isolates our key ingredient. If generic visually similar pairs work as well as graph-defined minimal pairs, the domain-knowledge component adds nothing.

Fairness is enforced in four ways:

  • every method gets the same base checkpoint;
  • every method gets the same number of preference pairs, drawn from the same MIMIC-CXR training studies;
  • every method gets the same adapter type and the same budget of eight runs sweeping $\beta$ and learning rate, selected on a shared validation split;
  • every method gets the same number of training steps.

Beating MMedPO means a statistically significant gain in RadGraph F1 and in 14-observation clinical F1 on the MIMIC-CXR test split, using bootstrap confidence intervals over studies. It also means no loss on general-language fluency metrics, and a clearly larger gain on the counterfactual tests described below.

Why not just…?

Why not just do supervised fine-tuning on more clean reports? Likelihood training rewards copying common report templates, and those templates are exactly the language prior we are trying to break. HA-DPO reports that on MiniGPT-4, preference tuning raised POPE accuracy from 51.13% to 86.13%, a margin it attributes to learning the boundary between correct and incorrect descriptions rather than only seeing correct ones. Our chosen-halves fine-tuning baseline tests this directly in radiology with matched data.

Why not just swap entities in the text, as EAGLE does? This is the real threat, because it is cheap and works on any report. Our hypothesis is that swapped reports are often separable by text alone. A swapped finding may clash with the rest of the report, or contradict the impression section. The blank-image margin diagnostic measures this directly. If entity-swap pairs show a blank-image margin as small as ours and match our hallucination reductions, we will report that the gain comes from the symmetric objective and not from mining real minimal pairs, and recommend the cheaper method.

Why not just corrupt the lesion region, as MMedPO does? Doing so requires a detector for each finding type and each modality. It also compares the same text across a real image and a corrupted one, which SymMPO argues leaves the normalizing constant uncanceled. A noised image is also an unnatural input that the model may learn to recognize as corrupted rather than learn to read. The head-to-head with MMedPO will settle this. We will also report its blank-image margin.

Why not just retrieve the nearest training report? Retrieval would reproduce the nearest neighbor's findings. Minimal pairs are, by construction, near neighbors with different findings. On the counterfactual test below, a retrieval baseline should therefore err roughly half the time, which makes it an informative lower bound.

Proposed Experiments

The first decisive check is the blank-image margin. For every negative-construction method, we compute the reference model's preference margin with the true image and with a blank image on held-out pairs. We predict that minimal pairs show a near-zero blank-image margin and that perturbation negatives show a substantially positive one. If this prediction fails, the central premise is wrong.

The second check is a counterfactual flip test. On held-out minimal pairs from the test split, we generate a report for each image and check, using RadGraph, whether the differing finding is stated correctly for each image. Language-prior-driven models will tend to write the same finding for both images. This test is exactly what the method claims to improve.

The third set covers standard benchmarks:

  • RadGraph F1, 14-observation clinical efficacy F1, and BLEU-4 on MIMIC-CXR and IU-Xray, against the anchor of 0.139 BLEU-4 and 0.353 F1;
  • factual accuracy on SLAKE and VQA-RAD, testing transfer from report-level pairs to question answering;
  • the visual and textual perturbation tests of HEAL-MedVQA, which measure whether answers depend on the right region without our method ever seeing a mask.

The fourth set measures robustness and cost:

  • performance as a function of the number of mined pairs per finding, which shows whether rare findings benefit;
  • the effect of relaxing to two-edit pairs;
  • the radiologist-audited label noise rate;
  • wall-clock cost of generating pairs, compared against MMedPO's GPT-4o and debate pipeline.