Multi-Modal Fusion Learning For Cervical Dysplasia Diagnosis

Tingting Chen, Xinjun Ma, Xingde Ying, Wenzhe Wang, Chunnv Yuan, Weiguo Lu, Danny Z. Chen, Jian Wu

Abstract

Fusion of multi-modal information from a patient's screening tests can help improve the diagnostic accuracy of cervical dysplasia. In this paper, we present a novel multi-modal deep learning fusion network, called MultiFuseNet, for cervical dysplasia diagnosis, utilizing multi-modal data from cervical screening results. To exploit the relations among different image modalities, we propose an Attention Mutual-Enhance (AME) module to fuse features of each modality at the feature extraction stage. Specifically, we first develop the Fused Faster R-CNN with AME modules for automatic cervix region detection and fused image feature learning, and then incorporate non-image information into the learning model to jointly learn non-linear correlations among all the modalities. To effectively train the Fused Faster R-CNN, we employ an alternating training scheme. Experimental results show the effectiveness of our method, which achieves an average accuracy of 87.4% (88.6% sensitivity and 86.1% specificity) on a large dataset, outperforming the methods using any single modality alone and the known multi-modal methods.
BibTeX
@inproceedings{chen2019multi,
  author = {Tingting Chen and Xinjun Ma and Xingde Ying and Wenzhe Wang and Chunnv Yuan and Weiguo Lu and Danny Z. Chen and Jian Wu},
  title = {{Multi-Modal Fusion Learning For Cervical Dysplasia Diagnosis}},
  booktitle = {ISBI},
  year = {2019},
  doi = {10.1109/isbi.2019.8759303},
}

Loading related content...

Figure