A Diagnostic Report Generator from CT Volumes on Liver Tumor with Semi-supervised Attention Mechanism

Jiang Tian, Cong Li, Zhongchao Shi, Feiyu XuMICCAI 2018 · OpenReview

Abstract

Automatically generating diagnostic reports with interpretability for computed tomography (CT) volumes is a new challenge for the computer-aided diagnosis (CAD). In this paper, we propose a novel multimodal data and knowledge linking framework between CT volumes and textual reports with a semi-supervised attention mechanism. This multimodal framework includes a CT slices segmentation model and a language model. Semi-supervised attention mechanism paves the way for visually interpreting the underlying reasons that support the diagnosis results. This multi-task deep neural network is trained end-to-end. We not only quantitatively evaluate our system performance (76.6% in terms of BLEU@4), but also qualitatively visualize the attention heat map for this framework on a liver tumor dataset.
BibTeX
@inproceedings{tian2018diagnostic,
  author = {Jiang Tian and Cong Li and Zhongchao Shi and Feiyu Xu},
  title = {{A Diagnostic Report Generator from CT Volumes on Liver Tumor with Semi-supervised Attention Mechanism}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2018},
  url = {https://openreview.net/forum?id=7DphL63rzh},
}

Loading related content...

Figure