Highly Efficient Representation and Active Learning Framework for Imbalanced Data and its Application to COVID-19 X-Ray Classification

Heng Hao, Sima Didari, Jae Oh Woo, Hankyu Moon, Patrick BangertarXiv 2021 · arXiv

Introduction

Deep learning has demonstrated transformative potential in medical imaging. However, two significant challenges hinder its widespread clinical adoption: the labeling bottleneck and extreme class imbalance. Expert medical annotation is a scarce and expensive resource, often requiring consensus from multiple specialists. Furthermore, real-world clinical data is inherently imbalanced; "normal" cases typically dwarf "pathological" ones. Standard Convolutional Neural Networks (CNNs) often struggle in these scenarios, overfitting the majority class and failing to identify rare but critical conditions, such as emerging viral strains.

To address these issues, we propose a data-efficient framework designed to achieve high classification accuracy with minimal human intervention while remaining robust to lopsided data distributions.

Methodology

The proposed framework utilizes a two-step decoupled approach to maximize information gain from unlabeled data.

1. Unsupervised Representation Learning

Instead of traditional supervised training, we employ SimCLR, a contrastive learning approach, with a ResNet-50 backbone. By training the model to recognize that different augmentations of the same image should yield similar feature representations, the system learns the fundamental structural characteristics of medical images without requiring labels. This pre-training phase ensures the model develops robust features that are not biased toward the majority class.

2. Gaussian Process (GP) Classifier and Active Learning

The 2048-dimensional features extracted by the ResNet-50 are fed into a Gaussian Process classifier. Unlike standard neural network "softmax" layers, GPs are non-parametric Bayesian models that provide mathematically grounded uncertainty estimates.

The proposed hybrid framework combining unsupervised representation learning (SimCLR) with a Gaussian Process (GP) classifier in an active learning loop.

The system operates in an active learning loop:

  • Initialization: Start with a minimal set of labeled data.
  • Prediction: Use the GP to predict labels for the unlabeled "pool" of images.
  • Selection: Identify samples where the GP exhibits the highest uncertainty.
  • Oracle Labeling: These "hard" samples are sent to a human expert for labeling.
  • Update: The newly labeled samples are added to the training set, and the process repeats.

Experimental Evaluation

The framework was evaluated on two distinct medical imaging tasks:

  • COVIDx Dataset: A highly imbalanced collection of 15,521 chest X-rays where only ~4% of training samples are COVID-19 positive.
  • Nerthus Dataset: A collection of 5,176 colonoscopy frames categorized by bowel preparation quality, serving to test the framework's generalizability on more balanced data.

Results

The hybrid framework demonstrated exceptional label efficiency and stability across both datasets.

Label Efficiency

On the COVIDx dataset, the framework achieved the same accuracy as a model trained on 100% of the data using only approximately 10% of the labels. For the Nerthus dataset, this efficiency was even more pronounced, requiring only 8% of labels to reach peak performance.

Comparison of COVID-19 classification accuracy across different learning strategies as the percentage of labeled training data increases.

Intelligent Rebalancing

A key finding was the framework's ability to perform "automatic rebalancing." Because the model was naturally most uncertain about rare COVID-19 cases, the active learning loop prioritized these samples for labeling. This effectively over-sampled the minority class through intelligent selection rather than manual data manipulation.

Stability

The CNN-GP hybrid exhibited significantly tighter variance and higher mean accuracy compared to standard architectures, particularly when the available label budget was extremely small (less than 20% of the total dataset).

Conclusion and Future Work

This research demonstrates that decoupling representation learning from classification, combined with Bayesian uncertainty for active learning, creates a highly efficient pipeline for medical image analysis.

While successful, future work will address the computational complexity of Gaussian Processes for larger-scale datasets. We also intend to explore more sophisticated acquisition functions to better balance exploration and exploitation, and investigate advanced self-supervised techniques to further refine the quality of unsupervised medical image representations.

Summary updated

Abstract

We propose a highly data-efficient active learning framework for image classification. Our novel framework combines: (1) unsupervised representation learning of a Convolutional Neural Network and (2) the Gaussian Process (GP) method, in sequence to achieve highly data and label efficient classifications. Moreover, both elements are less sensitive to the prevalent and challenging class imbalance issue, thanks to the (1) feature learned without labels and (2) the Bayesian nature of GP. The GP-provided uncertainty estimates enable active learning by ranking samples based on the uncertainty and selectively labeling samples showing higher uncertainty. We apply this novel combination to the severely imbalanced case of COVID-19 chest X-ray classification and the Nerthus colonoscopy classification. We demonstrate that only . 10% of the labeled data is needed to reach the accuracy from training all available labels. We also applied our model architecture and proposed framework to a broader class of datasets with expected success.
BibTeX
@misc{hao2021highly,
  author = {Heng Hao and Sima Didari and Jae Oh Woo and Hankyu Moon and Patrick Bangert},
  title = {{Highly Efficient Representation and Active Learning Framework for Imbalanced Data and its Application to COVID-19 X-Ray Classification}},
  year = {2021},
  eprint = {2103.05109},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2103.05109},
}
Figure