Integration of Self-supervised BYOL in Semi-supervised Medical Image Recognition

Hao Feng, Yuanzhe Jia, Ruijia Xu, Mukesh Prasad, Ali Anaissi, Ali BrayteeLecture Notes in Computer Science 2024 · DOI · OpenAlex

Abstract

Image recognition techniques heavily rely on abundant labeled data, particularly in medical contexts. Addressing the challenges associated with obtaining labeled data has led to the prominence of self-supervised learning and semi-supervised learning, especially in scenarios with limited annotated data. In this paper, we proposed an innovative approach by integrating self-supervised learning into semi-supervised models to enhance medical image recognition. Our methodology commences with pre-training on unlabeled data utilizing the BYOL method. Subsequently, we merge pseudo-labeled and labeled datasets to construct a neural network classifier, refining it through iterative fine-tuning. Experimental results on three different datasets demonstrate that our approach optimally leverages unlabeled data, outperforming existing methods in terms of accuracy for medical image recognition.

Introduction

Deep learning models in medical imaging face a significant bottleneck: the scarcity of high-quality labeled data. Annotating medical images like X-rays, OCT scans, or endoscopic images requires specialized expertise, making large-scale labeling expensive and time-consuming. While Semi-Supervised Learning (SSL) attempts to utilize unlabeled data, its performance is often limited by the quality of the initial model trained on a small labeled set.

To overcome these limitations, this paper introduces a hybrid framework that combines Bootstrap Your Own Latent (BYOL) self-supervised pre-training with iterative semi-supervised fine-tuning. By first learning robust structural representations from raw data without the need for explicit negative pairs, the model establishes a strong foundation for subsequent clinical diagnostic tasks.

Methodology

The proposed approach follows a two-stage pipeline designed to maximize the utility of unlabeled medical data.

1. Self-Supervised Pre-training with BYOL

The first stage utilizes BYOL to learn feature representations. Unlike traditional contrastive methods that require "negative" samples—which can be ambiguous in medical imaging due to structural similarities between different pathologies—BYOL uses an online and a target network to learn from different augmented views of the same image. This allows the encoder to capture essential anatomical features and textures without being misled by false distinctions.

2. Semi-Supervised Iterative Fine-tuning

Following pre-training, the model undergoes a semi-supervised phase:

  • Classifier Initialization: The pre-trained encoder is coupled with a classification head.
  • Pseudo-Label Generation: The model generates labels for the unlabeled dataset.
  • Iterative Refinement: The labeled data and high-confidence pseudo-labeled data are combined to retrain the classifier. This process repeats, progressively improving the model's accuracy as the pseudo-labels become more reliable.
Flowchart of the proposed hybrid self-supervised and semi-supervised pipeline.

Experimental Results

The framework was evaluated on three medical imaging benchmarks: OCT2017 (retinal disease), COVID-19 X-ray, and Kvasir (endoscopy).

The results indicate that the "warm start" provided by BYOL significantly enhances the quality of pseudo-labels compared to standard SSL methods. As shown in the table below, the proposed method consistently achieves higher accuracy across all three datasets, outperforming established models like MixMatch, FixMatch, and MeanTeacher.

Performance comparison of the proposed method against state-of-the-art semi-supervised learning techniques.

Specifically, the model achieved 96.6% accuracy on OCT2017, 98.7% on COVID-19 X-ray, and 97.6% on Kvasir. Analysis of hyperparameter sensitivity also revealed that increasing training epochs improves performance, though it introduces a computational trade-off (e.g., 94 minutes for 250 epochs vs. 15 minutes for 50 epochs).

Conclusion

Integrating self-supervised BYOL with iterative semi-supervised learning provides a powerful solution for medical image recognition with limited labels. By leveraging unlabeled data for both general feature extraction and task-specific refinement, the proposed method sets a new performance standard. Future work will focus on optimizing computational efficiency and implementing advanced noise-filtering techniques for pseudo-labels to further enhance diagnostic precision.

Summary updated

BibTeX
@inproceedings{feng2024integration,
  author = {Hao Feng and Yuanzhe Jia and Ruijia Xu and Mukesh Prasad and Ali Anaissi and Ali Braytee},
  title = {{Integration of Self-supervised BYOL in Semi-supervised Medical Image Recognition}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2024},
  doi = {10.1007/978-3-031-63772-8\_16},
}

Loading related content...

Figure