RadFusion: Towards Threshold-Controllable Radiology Report Generation

Ying Jin, Noel C. F. Codella, John Corring, Mu Wei, Dinei Florencio, Eric HorvitzarXiv 2026 · arXiv

TL;DR

RadFusion introduces the first framework for threshold-controllable radiology report generation, allowing clinicians to dynamically tune the sensitivity-specificity trade-off of AI-generated text to match different clinical contexts (such as emergency triage vs. confirmatory interpretation). By fusing a calibrated image classifier with a visual question-answering report generator via an LLM rewriter, the system produces reports that mathematically conform to standard ROC curves, paving a clear path for regulatory approval.

What Problem Does This Paper Address?

Automated radiology report generation has advanced rapidly using large vision-language models, typically treating the problem as an image captioning task. However, these conventional models produce a single, fixed report with a hidden, unadjustable operating point.

In medicine, one size does not fit all. Emergency triage requires high sensitivity to ensure no critical findings are missed, while confirmatory interpretation demands high specificity to avoid unnecessary, costly, or invasive interventions. Furthermore, regulatory bodies like the FDA expect Receiver Operating Characteristic (ROC) analysis for medical device clearance—a validation method that traditional text-generation models cannot support because they lack numerical confidence scores.

Methods

RadFusion solves this by decoupling diagnostic decision-making from text generation and fusing them back together using a three-stage pipeline:

  1. Perception Model: A medical image classifier (e.g., fine-tuned MedImageInsight) computes calibrated confidence scores for predefined disease classes. Clinicians can adjust a threshold ($\tau$) to binarize these scores into positive or negative findings.
  2. Report Generation Model: A specialized model (QRad) generates an initial free-text report. Crucially, QRad uses an "Auto-VQA" loop that can answer follow-up queries. If a low threshold forces a previously omitted finding to be marked positive, RadFusion queries QRad to retrieve the missing visual details (location, severity) rather than letting the system hallucinate.
  3. LLM Rewriter: An off-the-shelf LLM (such as GPT-5) takes the thresholded binary decisions and the evidence pool from the generator, rewriting the final report so its diagnoses perfectly align with the classifier's operating point while preserving rich clinical descriptions.
Figure 1: Overview of RadFusion, our threshold-controllable report generation framework. The classification model produces confidence scores for predefined disease categories, which are thresh- olded to binary decisions. The report generation model produces an initial report. The LLM rewriter combines both outputs so that the report’s stated diagnoses conform to the binary classifications while preserving clinical detail. We use QRad, a report generation model supporting follow-up questions to obtain evidence omitted from the initial report.

The authors also demonstrate an extension to three-dimensional threshold control, allowing the system to handle borderline findings with explicit hedging (e.g., "cannot exclude early consolidation") and prioritize findings based on clinical urgency.

Datasets

The framework was trained and evaluated on MIMIC-CXR, one of the largest publicly available chest X-ray datasets, containing 227,835 studies with 377,110 images. The authors targeted the Findings section for generation and evaluated performance across 13 disease classes defined by the CheXpert schema using the official test split of 2,347 studies.

Key Insights and Ideas

  • Inheriting ROC Characteristics: The most profound insight is that text generation can be made to strictly conform to a classifier's ROC curve. By mapping the rewritten text back to binary labels, the authors proved that the text behavior perfectly mirrors the mathematical trade-offs of the underlying perception model.
  • Fusing for Accuracy: Combining a classifier and a generator doesn't just add control; it actually improves raw performance. The structured supervision of class labels grounds the model, while the free-text reports provide fine-grained visual features that prevent the representation space from collapsing.
  • Auto-VQA as a Safeguard: Instead of relying on an LLM to blindly add a disease to a report when a threshold is lowered, using a VQA model to query the image ensures that any newly added diagnosis is grounded in actual visual evidence.

Key Results

  • ROC Conformance: Across all evaluated conditions, the generated reports' diagnostic accuracy closely tracked the classifier's ROC curve.
  • Performance Boost: Compared to standard, uncontrolled report generation, RadFusion achieved a 6.9% increase in sensitivity at a matched specificity, and a massive 20.7% increase in specificity at a matched sensitivity.
  • Prompting Sensitivity: Including concrete per-class text examples in the LLM's prompt was critical; removing them caused the macro-average AUC-ROC to drop from 0.90 to 0.88 for GPT-5.
Figure 2: Threshold-controllable report generation on four representative classes. Cardiomegaly: common, both models perform strongly. Edema: threshold tuning is clinically valuable since mild cases often need no treatment. Atelectasis: challenging, requires fine visual details. Pleural Other: rare (1.3% prevalence). Across all four, the report ROC (red) closely tracks the classifier (blue), and the non-controlled report (green cross) falls below the curves.

Limitations and Future Work

  • Error Propagation: The system is highly dependent on its components; errors in the classifier or omissions in the initial report generator can propagate directly into the final output.
  • Domain and Modality Constraints: The rewriting prompts are currently template-based and heavily optimized for chest X-rays. Adapting RadFusion to other modalities (like CT or MRI) or clinical domains would require re-tuning the prompts.
  • Predefined Classes: The threshold control only applies to a fixed set of disease categories (the 13 CheXpert classes). Findings outside this set cannot be dynamically adjusted.
  • Future Directions: The authors suggest that developing clinician-tunable operating-point user interfaces is a promising direction for real-world clinical deployment.

Summary updated

Abstract

Automated radiology report generation is advancing rapidly in response to the shortage of radiologists, yet unlike a perception model, existing generation models offer no control over the sensitivity-specificity trade-off of their diagnostic content. Such control is essential because clinical scenarios diverge: emergency triage prioritizes sensitivity to reduce missed findings, whereas confirmatory interpretation emphasizes specificity to limit unnecessary interventions. A single fixed report can neither adapt to these scenarios nor support the ROC-based validation widely expected for regulatory clearance. We introduce RadFusion, a framework that equips report generation with threshold controllability. Our method fuses a multi-label classifier, which provides per-disease confidence scores, with a VQA-based report generator, which describes medical findings in detail; an LLM then rewrites the report so that its stated diagnoses follow the classifier's decisions at the selected threshold while staying grounded in the generator's descriptions. On MIMIC-CXR, the performance of RadFusion conforms to the classifier's ROC curve: sweeping the threshold and mapping the reports back to class labels reproduces the classifier's validated ROC performance. This conformance makes generated reports quantitatively evaluable through ROC analysis, strengthening the case for regulatory clearance, and enables operating-point selection that matches report behavior to clinical context. Moreover, combining the two model types improves diagnostic accuracy over uncontrolled generation: sensitivity increases by 6.9% at matched specificity, and specificity by 20.7% at matched sensitivity. These results show that RadFusion makes report generation clinically adaptable, quantitatively verifiable, and diagnostically more reliable.
BibTeX
@misc{jin2026radfusion,
  author = {Ying Jin and Noel C. F. Codella and John Corring and Mu Wei and Dinei Florencio and Eric Horvitz},
  title = {{RadFusion: Towards Threshold-Controllable Radiology Report Generation}},
  year = {2026},
  eprint = {2608.10505},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.10505},
}
Figure