Mammo-Net: Integrating Gaze Supervision and Interactive Information in Multi-view Mammogram Classification

Changkai Ji, Changde Du, Qing Zhang, Sheng Wang, Chong Ma, Jiaming Xie, Yan Zhou, Huiguang He, Dinggang Shen

Abstract

Breast cancer diagnosis is a challenging task. Recently, the application of deep learning techniques to breast cancer diagnosis has become a popular trend. However, the effectiveness of deep neural networks is often limited by the lack of interpretability and the need for significant amount of manual annotations. To address these issues, we present a novel approach by leveraging both gaze data and multi-view data for mammogram classification. The gaze data of the radiologist serves as a low-cost and simple form of coarse annotation, which can provide rough localizations of lesions. We also develop a pyramid loss better fitting to the gaze-supervised process. Moreover, considering many studies overlooking interactive information relevant to diagnosis, we accordingly utilize transformer-based attention in our network to mutualize multi-view pathological information, and further employ a bidirectional fusion learning (BFL) to more effectively fuse multi-view information. Experimental results demonstrate that our proposed model significantly improves both mammogram classification performance and interpretability through incorporation of gaze data and cross-view interactive information.
BibTeX
@inproceedings{ji2023mammo,
  author = {Changkai Ji and Changde Du and Qing Zhang and Sheng Wang and Chong Ma and Jiaming Xie and Yan Zhou and Huiguang He and Dinggang Shen},
  title = {{Mammo-Net: Integrating Gaze Supervision and Interactive Information in Multi-view Mammogram Classification}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2023},
  doi = {10.1007/978-3-031-43990-2\_7},
}

Loading related content...

Figure