Abstract
Automated Radiography Report Generation (RRG) aims to assist radiologists by drafting descriptive reports from medical images. However, standard training objectives often fail to capture clinical nuances, leading to generic or inaccurate outputs. We present Random Direct Preference Optimization (RDPO), a simple, model-agnostic framework that aligns Vision-Language Models (VLMs) with clinical accuracy. By utilizing randomly sampled reports as negative examples, RDPO improves clinical performance metrics by up to 5% without requiring expensive human annotations or additional data. Our results demonstrate that RDPO effectively reduces hallucinations and improves the specificity of generated reports.
Introduction
The primary goal of RRG is to generate accurate textual descriptions of findings in medical images, such as chest X-rays. Traditional models are typically trained using next-token prediction (cross-entropy loss). While effective for learning medical terminology and grammar, this objective often encourages models to produce generic, "safe" statements (e.g., "no acute cardiopulmonary process") that may ignore specific visual evidence. Furthermore, the high variability in reporting styles among radiologists makes it difficult for simple predictive models to distinguish between stylistic differences and clinical errors.
Methodology
To address these limitations, we adapt Direct Preference Optimization (DPO), a technique originally designed for aligning Large Language Models with human preferences. Standard DPO requires a curated dataset of "chosen" (correct) and "rejected" (incorrect) response pairs.
The RDPO Framework
The core innovation of this work is the Random sampling strategy for negative examples. In the context of radiography, a report is highly specific to a single patient's image. Therefore, a report belonging to Patient A is almost certainly an incorrect description for Patient B.
- Chosen Response: The ground-truth report associated with the specific input X-ray.
- Rejected Response: A report randomly sampled from a different image within the same training dataset.
- Optimization: The model is fine-tuned to maximize the relative log-likelihood of the chosen report over the rejected one. This contrastive approach forces the VLM to identify specific visual features that justify the ground-truth text while penalizing the generation of generic or irrelevant medical clichés.
This method is entirely model-agnostic and eliminates the need for a separate reward model or costly human labeling efforts.
Experimental Setup
The framework was evaluated using four major chest X-ray (CXR) datasets:
- MIMIC-CXR: The primary training and evaluation corpus (377,110 images).
- CheXpert Plus: A large-scale dataset of 223,228 image-report pairs.
- IU X-Ray: Used to test the model's ability to generalize to out-of-distribution data.
- Interpret-CXR: A massive composite dataset used for the final alignment phase of high-capacity models.
Results and Analysis
RDPO was tested across three state-of-the-art architectures, including the MAIRA-2 model. Quantitative evaluations showed consistent improvements across several clinical metrics.
Key Findings
- Clinical Metric Boost: The authors observed a consistent improvement of up to 5% in metrics such as RadCliQ-v1 (a composite metric correlating with expert preference) and Semb-score (measuring semantic alignment).
- State-of-the-Art Performance: Applying RDPO to the MAIRA-2 model established a new benchmark for radiography report generation.
- Expert Validation: In a blinded side-by-side evaluation of 100 cases, a board-certified radiologist preferred the RDPO-aligned model 33% of the time, compared to only 7% for the baseline (with 60% ties). Qualitative analysis indicated that RDPO models provide more specific details regarding pathologies rather than relying on generic summaries.
Discussion and Limitations
While RDPO provides a "free lunch" by leveraging existing data for alignment, some challenges remain. A performance dip was noted on the IU X-Ray dataset, suggesting that domain shifts between different hospital systems remain a hurdle for generalization. Future work may explore more sophisticated sampling strategies, such as selecting "hard negatives" (reports that are semantically similar but clinically incorrect) to further refine the model's discriminative capabilities. Additionally, expanding the framework to other modalities like CT or MRI scans is a promising avenue for future research.
Conclusion
Random Direct Preference Optimization offers a highly efficient path toward more clinically accurate AI in radiology. By reframing existing data as a preference task, RDPO helps models break the habit of outputting generic medical clichés and encourages a deeper alignment between visual findings and textual reports.
Summary updated