Multimodal Image-to-Image Translation for Generation of Gastritis Images

Ren Togo, Takahiro Ogawa, Miki Haseyama

Abstract

We present a new multimodal image-to-image translation model for the generation of gastritis images using X-ray and blood inspection results. In clinical situations, clinicians estimate the prognosis of the target disease by considering multiple inspection results. Similarly, we take a multimodal approach in the task of gastric cancer risk prediction. Visual characteristics of the gastric X-ray image and blood index values are highly related in the evaluation of gastric cancer risk. If we can generate a prediction image from blood index values, it contributes to the clinicians' sophisticated and integrated diagnosis. Hence, we learn a model that can map non-gastritis images to gastritis images based on the blood index values. Although this is a challenging multimodal task in medical image analysis, experimental results showed the effectiveness of our model.
BibTeX
@inproceedings{togo2020multimodal,
  author = {Ren Togo and Takahiro Ogawa and Miki Haseyama},
  title = {{Multimodal Image-to-Image Translation for Generation of Gastritis Images}},
  booktitle = {ICIP},
  year = {2020},
  doi = {10.1109/icip40778.2020.9190863},
}

Loading related content...

Figure