Improving Radiology Report Generation with D-Net: When Diffusion Meets Discriminator
Lists
Only you see this
Abstract
Radiology report generation (RRG) aims to automatically provide observations and insight into a patient’s condition based on radiology images, which is able to greatly reduce the workload of physicians on the premise of ensuring the quality of medical treatment. Existing works leverage the Transformer decoder to generate reports word-by-wordly. However, unlike image captioning, radiology reports are long text containing many semantic words. The autoregressive method, such as the Transformer-base method, will accumulate errors in the generation process and generate unsatisfied reports. Benefiting from the recent success of Diffusion, we propose a novel Diffusion-based paradigm for RRG, which leverages visual information as a condition, making the generation process focus on pathological features within the radiology image. Meanwhile, we integrate a discriminator into each layer of the Diffusion to actively judge whether the generated words are meaningful, which, on the one hand, controls the length of predicted reports and, on the other hand, calibrates confidence scores and token generation results, improving the quality of the generated reports. Extensive experiment results demonstrate the superiority of our proposed method. Source code is available at: https://github.com/Yuda-Jin/D-2-Net.
BibTeX
@inproceedings{jin2024improving,
author = {Yuda Jin and Weidong Chen and Yuanhe Tian and Yan Song and Chenggang Yan and Zhendong Mao},
title = {{Improving Radiology Report Generation with D-Net: When Diffusion Meets Discriminator}},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
year = {2024},
url = {https://openreview.net/forum?id=Ef3dPnc1Tq},
}Loading related content...