Introduction
Generating radiology reports from medical imaging is a complex task that extends beyond simple image captioning. Unlike identifying objects in a scene, a radiologist must synthesize multiple views of an X-ray into a structured narrative. This narrative must maintain clinical precision, logical flow, and provide actionable recommendations.
Current "black-box" end-to-end neural models often struggle with this complexity. Common failure modes include hallucinations (reporting non-existent findings), redundancy (repetitive observations), and incoherence (lack of logical transitions between sentences). This research introduces a modular architecture designed to address these issues by providing explicit control over the generation process.
Proposed Architecture
The authors propose a sentence-by-sentence generation framework that incorporates several novel mechanisms to ensure topical continuity and conciseness.
The architecture, illustrated in Figure 1, centers on three key innovations:
- Gated Content Selection: The model employs two distinct gates. The Informative Content (IC) gate filters image features relevant to the current sentence context, while the Topical Continuity (TC) gate ensures that the generated sentence follows logically from the preceding one.
- Orthogonal Sentence States (O): To mitigate repetition, the model enforces a diversity constraint. Each new sentence state is mathematically updated to be orthogonal to the average of previous states, forcing the decoder to focus on new information.
- Entity-Guided Pre-training (PT): The model is pre-trained to predict medical entities (e.g., "pleural effusion") from images. This provides an inductive bias that grounds the model's internal representations in clinical concepts before it learns full sentence generation.
The system uses a ResNet-152 backbone for image encoding and a Transformer-based encoder for the "indication text," which provides the clinical reason for the imaging.
Experimental Setup
The model was evaluated using the MIMIC-CXR database, a large-scale collection of chest X-rays and associated reports. The researchers utilized a subset of 19,800 entries, extracting 1,060 medical entity clusters for the pre-training task. The data was split into 18,000 records for training, and 900 each for development and testing.
Results and Discussion
The proposed model (Img + IC + TC + O + PT) was compared against standard baselines, including basic RNN and attention-based models.
As shown in Figure 2, the full model consistently outperformed baseline configurations. Key findings include:
- Objective Performance: The model achieved an 18% improvement in informativeness (BLEU/ROUGE) and a 29% improvement in content ordering compared to standard baselines.
- Conciseness and Flow: The "Orthogonal Sentence States" mechanism effectively reduced redundancy. Qualitative analysis showed the model successfully transitions between different anatomical observations (e.g., lungs to heart to spine) in a logical sequence, whereas baselines often repeated the same lung findings.
- Human Validation: Blind tests by human evaluators rated the factual accuracy and flow of the proposed method 16% higher than the next-best baseline.
Conclusion
By moving away from purely "black-box" logic and introducing interpretable gates and mathematical constraints for diversity, this architecture significantly improves the quality of automated radiology reports. While the study noted limitations regarding data sparsity for certain medical entities, the results demonstrate that context-specific representations and orthogonality are powerful tools for generating coherent, clinical-grade text. Future work will focus on refining entity clustering and exploring larger datasets to further strengthen the model's medical grounding.
Summary updated