Task-Driven Self-Supervised BI-Channel Networks Learning for Diagnosis of Breast Cancers with Mammography

Ronglin Gong, Shihui Ying, Jun ShiICIP 2022 · OpenReview

TL;DR

This paper introduces a novel "Task-driven Self-supervised Bi-channel Networks" (TSBN) framework to improve breast cancer diagnosis from mammograms, especially when training data is scarce. TSBN uses a new "Gray-scale Image Mapping" (GSIM) pretext task that embeds class labels into image restoration, allowing two distinct networks (one for GSIM, one for diagnosis) to be jointly trained and share knowledge, leading to better diagnostic accuracy.

What Problem Does This Paper Address?

Diagnosing breast cancer using mammography is crucial, and deep learning (DL) methods, particularly Convolutional Neural Networks (CNNs), have shown promise in Computer-Aided Diagnosis (CAD). However, a major hurdle is the "small sample size problem" – acquiring and annotating large datasets of mammograms is expensive and time-consuming, leading to insufficient data to train accurate and stable DL models. While Self-Supervised Learning (SSL) offers a way to leverage unlabeled data, conventional SSL frameworks can struggle to pre-train models effectively due to limitations in pretext tasks and fine-tuning mechanisms. This paper aims to bridge this gap by proposing a more effective SSL approach tailored for mammography-based CAD with limited data.

Methods

The core innovation is the Task-driven Self-supervised Bi-channel Networks (TSBN) framework. Unlike traditional SSL that often uses a single network backbone for both pretext and downstream tasks (fine-tuning), TSBN employs two distinct, specialized networks:

  1. Pretext Task Channel Module: This module uses a novel pretext task called Gray-scale Image Mapping (GSIM). GSIM is an image restoration task where class label information (benign vs. malignant) is embedded into the generated "pseudo-labels" (transformed mammograms). This forces the pretext network to learn discriminative features that are directly relevant to the diagnostic task. The paper experimented with U-Net and Residual Dense Network (RDN) as architectures for this module.
  2. Downstream Task Channel Module: This module is responsible for the actual breast cancer diagnosis. The authors chose ResNet50, a well-established architecture for CAD, for this task.
  3. Collaborative Transfer Module: This module facilitates knowledge transfer between the two distinct networks. Instead of fine-tuning, TSBN uses a collaborative transfer loss to jointly train both networks. This allows the networks to learn from each other, with the pretext task network guiding the downstream network to learn more robust and discriminative features. The overall loss for the downstream task combines this collaborative loss with the standard binary cross-entropy loss for classification.

Datasets

The study was evaluated on the INbreast dataset, a public dataset for mammography-based breast cancer diagnosis.

  • It contains 115 patients and 410 images.
  • Samples were categorized into benign (BI-RADS 1 & 2) and malignant (BI-RADS 4, 5, & 6). BI-RADS 3 ("probably benign") was excluded.
  • This resulted in 287 benign samples and 100 malignant samples.
  • The paper does not introduce a new benchmark dataset.

Key Insights and Ideas

  • Embedding Class Labels into Pretext Tasks: The most significant insight is that by embedding class label information directly into the pretext task (GSIM), the self-supervised learning process becomes much more relevant to the downstream diagnostic task. This contrasts with traditional SSL where pretext tasks might learn general image features without explicit class correlation.
  • Distinct Networks for Pretext and Downstream Tasks: The paper argues that using specialized network architectures for the pretext (e.g., U-Net, RDN for restoration) and downstream (e.g., ResNet for classification) tasks, rather than a single shared backbone, leads to better feature learning. This allows each network to excel at its specific job.
  • Collaborative Transfer for Knowledge Sharing: Moving beyond simple fine-tuning, the proposed collaborative transfer mechanism allows for a more nuanced and effective sharing of learned representations between the two networks, boosting performance on the limited data.
  • GSIM's Advantage over Generic Restoration: The GSIM task, by incorporating label information, provides more discriminative features than generic image restoration pretext tasks, leading to improved performance for the downstream classification.

Key Results

The TSBN framework, tested on the INbreast dataset, demonstrated superior performance compared to several baseline methods:

  • TSBN-U (using U-Net for pretext): Achieved a mean classification accuracy of 85.53%±1.87%, outperforming ResNet (82.94%) by 2.59% and SimCLR (84.46%) by 1.07%. It also showed improvements in sensitivity, specificity, Youden Index (YI), and F1-score.
  • TSBN-R (using RDN for pretext): Achieved a mean classification accuracy of 85.78%±2.65%, also outperforming baselines.
  • AUC Values: TSBN-U and TSBN-R achieved AUC values of 0.893 and 0.899, respectively, which were higher than those of fine-tuning based algorithms like contrastive and generative SSL.
  • The paper highlights that TSBN improves performance by at least 0.77% on accuracy and 1.38% on F1-score compared to other methods.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that while U-Net and RDN are effective for image restoration, their specific architectures were chosen for evaluation purposes. The optimal architecture for the pretext task might vary.
  • The study focused on binary classification. Extending to multi-class scenarios might require adjustments.
  • Future Work:
    • The authors plan to explore different types of pretext tasks that can be designed based on label information and the specific characteristics of medical data.
  • They also aim to enhance the transfer performance between the two networks to further improve the classification accuracy of the downstream CAD model.

Summary updated

Abstract

Deep learning can promote mammography-based computer-aided diagnosis (CAD) for breast cancers, but it generally suffers from the small size sample problem. In this work, a task-driven self-supervised bi-channel networks learning (TSBNL) framework is proposed to improve the network performance with limited mammograms. In particular, a new gray-scale image mapping (GSIM) task for image restoration is designed as the pretext task to improve discriminative feature representation with label information of mammograms. TSBNL then innovatively integrates this image restoration network and the downstream classification network into a unified SSL framework, and transfers the knowledge from the pretext network to the classification network with improved diagnostic accuracy. The proposed algorithm is evaluated on a public INbreast mammogram dataset. The experimental results indicate that it outperforms the conventional SSL algorithms for the diagnosis of breast cancers with limited samples.
BibTeX
@inproceedings{gong2022task,
  author = {Ronglin Gong and Shihui Ying and Jun Shi},
  title = {{Task-Driven Self-Supervised BI-Channel Networks Learning for Diagnosis of Breast Cancers with Mammography}},
  booktitle = {ICIP},
  year = {2022},
  url = {https://openreview.net/forum?id=ONhxA9-ZLPX},
}
Figure