EditGRPO: Reinforcement Learning with Post -Rollout Edits for Clinically Accurate Chest X-Ray Report Generation

Kai Zhang, Christopher Malon, Lichao Sun, Martin Renqiang Min

Introduction

Generating radiology reports requires a level of clinical precision that goes far beyond general image captioning. While Multimodal Large Language Models (MLLMs) have shown impressive capabilities in describing natural scenes, they often struggle with the specific diagnostic accuracy required in medical settings. A common issue in training these models is "shortcut bias." Because the majority of chest X-rays are normal, models trained with standard Reinforcement Learning (RL) often learn that reporting "no acute abnormality" yields high rewards, leading to "policy collapse" where the model fails to identify rare but critical findings.

To address this, researchers from NEC Laboratories and Lehigh University have developed EditGRPO, a reinforcement learning framework that incorporates sentence-level clinical corrections during the training process to ensure factual reliability.

The EditGRPO Framework

EditGRPO builds upon Group Relative Policy Optimization (GRPO), an efficient RL variant. The primary innovation is the introduction of Post-Rollout Edits, which provide "off-policy" guidance to the model during training.

The process follows four main steps:

  1. Rollout: The policy model generates multiple candidate reports for a given chest X-ray.
  2. NER Analysis: A specialized Named Entity Recognition (NER) tool (RaTE-NER) extracts clinical entities from both the generated drafts and the ground-truth reference reports.
  3. The Edit Operation: The system identifies discrepancies between the draft and the reference. It then performs three types of edits: deleting false positives (hallucinations), replacing incorrect findings, and adding missing critical information (false negatives).
  4. Learning: The model is optimized to minimize the gap between its original output and the "edited" version, effectively learning from its mistakes in a structured, clinically-grounded way.
The EditGRPO framework uses NER-based entity comparison to perform post-rollout edits (delete, replace, add) on model-generated reports.

By focusing on sentence-level edits rather than paragraph-level rewrites, the framework allows the model to absorb fine-grained clinical nuances without destabilizing the training process.

Experimental Setup and Results

The researchers evaluated EditGRPO using the Qwen2.5-VL-3B model across four major chest X-ray datasets: MIMIC-CXR and RexGradient for primary training, and IU-XRay and PadChest-GR for testing out-of-domain generalization.

The results demonstrate that EditGRPO consistently outperforms both Supervised Fine-Tuning (SFT) and vanilla GRPO baselines.

Performance comparison of EditGRPO against SFT and Dr. GRPO baselines across various clinical metrics on PadChest-GR and IU-XRay datasets.

Key findings include:

  • Clinical Accuracy Improvement: EditGRPO achieved a 3.4% average improvement in clinical metrics such as CheXbert and RadGraph scores.
  • Robust Generalization: The model showed a 5.9% performance gain on unseen datasets (IU-XRay and PadChest-GR), indicating that the edit-based training helps the model learn more generalizable clinical features rather than just dataset-specific phrasing.
  • Reduction in Hallucinations: By explicitly targeting and deleting false positives during the edit phase, the model becomes significantly less prone to inventing findings.

Discussion and Conclusion

The success of EditGRPO highlights the importance of clinical grounding in medical AI. While standard RL often falls into the "normal trap," EditGRPO acts as a safety net, forcing the model to encounter and learn from abnormal findings.

However, the framework does have limitations. It is dependent on the accuracy of the NER tool used for editing; any errors in entity extraction can lead to flawed training signals. Additionally, scaling this approach to much larger models remains a computational challenge.

In conclusion, EditGRPO provides a robust blueprint for training MLLMs in high-stakes domains. By treating the training process as a cycle of drafting and expert-informed editing, it moves AI radiology closer to the level of factual reliability required for clinical practice.

Summary updated

BibTeX
@misc{zhang2025editgrpo,
  author = {Kai Zhang and Christopher Malon and Lichao Sun and Martin Renqiang Min},
  title = {{EditGRPO: Reinforcement Learning with Post -Rollout Edits for Clinically Accurate Chest X-Ray Report Generation}},
  year = {2025},
  eprint = {2509.22812},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.22812},
}
Figure