Hybrid Reinforced Medical Report Generation with M-Linear Attention and Repetition Penalty

Wenting Xu, Zhenghua Xu, Junyang Chen, Chang Qi, Thomas Lukasiewicz

TL;DR

This paper introduces HReMRG-MR, a novel framework that significantly improves automatic medical report generation from chest X-rays. By combining high-order "M-linear" attention, a multi-metric hybrid reinforcement learning reward, and an adaptive repetition penalty, the model overcomes the standard text-generation traps of poor coherence and repetitive phrasing to produce highly accurate and readable clinical reports.

What Problem Does This Paper Address?

Writing radiology reports is a time-consuming, high-volume task. While automatic medical report generation using deep learning (typically CNN-RNN encoder-decoder architectures) could drastically reduce radiologist workloads, current state-of-the-art models suffer from three major bottlenecks:

  1. Incomprehensive Optimization Goals: Most models rely on cross-entropy loss (which only looks at word-level errors) or use Reinforcement Learning (RL) optimized for a single metric like CIDEr. This partial optimization fails to capture the overall sentence coherence required for long, paragraph-based medical text.
  2. Low-Order Attention Mechanisms: Standard spatial or semantic attention mechanisms only capture simple, first-order relationships. They miss the complex, high-order interactions between visual regions and text tokens across different feature channels, making it hard to pinpoint subtle abnormalities.
  3. Repeated Generation: Because a large portion of medical reports describe "normal findings" using highly repetitive phrasing, standard models frequently get stuck in loops, generating redundant, meaningless words that ruin readability.

Methods

The authors propose the HReMRG-MR framework to tackle these challenges through three core innovations:

1. M-Linear Attention Module

To capture fine-grained visual and semantic details, the framework introduces a stack of bilinear pooling-based attention blocks. Bilinear pooling calculates the outer product between two feature vectors, allowing the model to capture second-order feature interactions across all elements. By stacking these blocks, the encoder can capture up to 8th-order intra-modal feature interactions, while the decoder uses them to fuse multi-modal (visual and textual) features.

Fig. 1. Architecture of our proposed HReMRG-MR, where Embed denotes a non-linear projection operation, and GLU denotes gated linear units.

2. Hybrid Reward with a Linear Search Algorithm

Instead of optimizing for a single metric, the authors use Self-Critical Sequence Training (SCST) with a hybrid reward combining seven distinct natural language generation (NLG) metrics. Because finding the optimal weights for seven metrics using standard grid search is computationally impossible ($O(n^7)$ complexity), they developed a greedy-search-based algorithm that reduces the search complexity to a linear level ($O(n)$), making multi-metric optimization practical.

3. Repetition Penalty Module

To stop the model from repeating itself, an exponential penalty is integrated directly into the text generation phase. It tracks generated trigrams and subtracts a value proportional to their frequency from the output word log-probabilities, forcing the model to favor diverse and coherent phrasing.

Fig. 2. M-linear attention block.

Datasets

The framework was trained and evaluated on two prominent public chest X-ray datasets:

  • IU X-Ray: A dataset from Indiana University containing 7,470 chest X-ray images and 3,955 reports. After filtering out incomplete data, the authors utilized 6,222 images (paired frontal and lateral views) and 3,111 reports.
  • MIMIC-CXR: The largest publicly available dataset of its kind. The authors filtered it down to 76,724 pairs of multi-view images and free-text reports, focusing specifically on the "Impression" and "Findings" sections.

Key Insights and Ideas

  • Bilinear Pooling is Powerful for Multi-Modal Fusion: Originally popular in fine-grained image classification, bilinear pooling proves highly effective at aligning subtle radiological visual cues with specific medical terminology.
  • The Danger of Single-Metric Optimization: The paper demonstrates that optimizing purely for a metric like CIDEr creates a skewed model that scores well on that specific metric but produces less readable text overall. A balanced, multi-metric hybrid reward is essential for paragraph-length generation.
  • Linearizing the Reward Space: The greedy search strategy proves that you don't need an exhaustive grid search to find effective hyperparameter weights; iteratively optimizing the most influential metric yields identical results at a fraction of the computational cost.

Key Results

HReMRG-MR consistently outperformed state-of-the-art baselines (including Top-down, MRMA, RTMIC, and X-LAN) across all evaluated metrics on both datasets.

  • IU X-Ray Results: The model achieved a BLEU-4 score of 0.1490 and a CIDEr score of 0.5239, substantially higher than the strong X-LAN baseline (0.1314 BLEU-4, 0.4508 CIDEr).
  • MIMIC-CXR Results: It achieved a BLEU-4 score of 0.1921 and a CIDEr score of 0.3715.
  • Ablation Studies: Removing the repetition penalty or replacing the M-linear attention with standard X-linear attention caused noticeable drops in performance, validating that every component directly contributes to the model's success.

Limitations and Future Work

Despite the strong results, the authors point out a few areas for improvement:

  • Clinical Accuracy vs. NLG Metrics: Standard text-generation metrics (like BLEU and CIDEr) evaluate linguistic similarity but do not guarantee clinical accuracy (e.g., correctly identifying a rare but critical disease). Developing clinical-specific evaluation metrics is a key future direction.
  • Search Overhead: While the greedy reward search is linear, it still requires multiple training iterations. The authors suggest that Inverse Reinforcement Learning (IRL) could be used in the future to automatically learn an optimal reward function directly from human-written reports.

Summary updated

Abstract

To reduce doctors' workload, deep-learning-based automatic medical report generation has recently attracted more and more research efforts, where deep convolutional neural networks (CNNs) are employed to encode the input images, and recurrent neural networks (RNNs) are used to decode the visual features into medical reports automatically. However, these state-of-the-art methods mainly suffer from three shortcomings: (i) incomprehensive optimization, (ii) low-order and unidimensional attention mechanisms, and (iii) repeated generation. In this article, we propose a hybrid reinforced medical report generation method with m-linear attention and repetition penalty mechanism (HReMRG-MR) to overcome these problems. Specifically, a hybrid reward with different weights is employed to remedy the limitations of single-metric-based rewards. We also propose a search algorithm with linear complexity to approximate the best weight combination. Furthermore, we use m-linear attention modules to explore high-order feature interactions and to achieve multi-modal reasoning, while a repetition penalty applies penalties to repeated terms during the model's training process. Extensive experimental studies on two public datasets show that HReMRG-MR greatly outperforms the state-of-the-art baselines in terms of all metrics. We also conducted a series of ablation experiments to prove the effectiveness of all our proposed components. We also performed a reward search toy experiment to give evidence that our proposed search approach can significantly reduce the search time while approximating the best performance.
BibTeX
@article{xu2022hybrid,
  author = {Wenting Xu and Zhenghua Xu and Junyang Chen and Chang Qi and Thomas Lukasiewicz},
  title = {{Hybrid Reinforced Medical Report Generation with M-Linear Attention and Repetition Penalty}},
  journal = {IEEE Trans. Neural Networks Learn. Syst.},
  year = {2022},
  url = {https://openreview.net/forum?id=NnbaiwmYr4},
}
Figure