Insite: Labelling Medical Images Using Submodular Functions and Semi-Supervised Data Programming

Akshat Gautam, Anurag Shandilya, Akshit Srivastava, Venkatapathy Subramanian, Ganesh Ramakrishnan, Kshitij S. Jadhav

TL;DR

The INSITE framework addresses the "expert bottleneck" in medical AI by combining submodular subset selection with data programming. By picking just 10–30 highly representative images for a doctor to label, the system can automatically generate high-quality labels for thousands of other images, outperforming state-of-the-art semi-supervised methods in ultra-low-budget scenarios.

What Problem Does This Paper Address?

Deep learning has revolutionized medical imaging, but it has a voracious appetite for labeled data. In the medical domain, "ground truth" labels must come from highly trained specialists, making data annotation prohibitively expensive and slow—especially in resource-constrained regions.

While semi-supervised learning (SSL) aims to use small amounts of labeled data alongside large amounts of unlabeled data, it often struggles when the initial labeled set is too small or contains noisy labels. The authors identify a specific gap: how do we maximize the utility of a tiny labeling budget (e.g., only 10 or 20 images) to label an entire dataset?

Methods

The authors propose INSITE (labelINg medical imageS usIng submodular funcTions and sEmi-supervised data programming), which operates in three main stages:

  1. Informed Subset Selection: Instead of picking images at random for an expert to label, the system uses submodular functions. Specifically, they use Facility Location (to find representative images that "cover" the dataset) and Log Determinant (to ensure a diverse set of examples).
  2. Exemplar-based Labeling Functions (LFs): Once an expert labels a few "exemplar" images, these images are used to create rules. For any new, unlabeled image, the system calculates its similarity to these exemplars in a feature space (using a pre-trained ResNet). If an unlabeled image is highly similar to a "diseased" exemplar, the labeling function suggests a "diseased" label.
  3. CAGE (Generative Model): Because different exemplars might give conflicting or noisy suggestions, the authors use the CAGE framework. CAGE is a generative model that aggregates these "weak" labeling functions, considering both the predicted labels and the similarity scores to reach a final, high-confidence consensus for every image in the dataset.

Datasets

The framework was evaluated on three significant medical imaging benchmarks:

  • APTOS 2019: Retinal fundoscopy images for Diabetic Retinopathy detection.
  • Chest X-Ray Images: A dataset of 5,856 images used to diagnose pediatric pneumonia.
  • Tuberculosis (TB) X-Ray Images: A repository of chest X-rays containing both TB-positive and normal cases.

In all cases, the authors converted multi-class problems into binary classification (e.g., "Normal" vs. "Diseased") to test the system as a screening tool.

Key Insights and Ideas

  • The Power of Selection: The paper demonstrates that which images you choose to label matters as much as how many. Using submodular functions to pick the most representative "anchors" significantly boosts the performance of downstream labeling.
  • Images as Heuristics: Traditional "Data Programming" (like the Snorkel framework) usually relies on text-based rules (e.g., "if the word 'cough' appears, label as sick"). INSITE treats individual images as visual "rules," bridging the gap between human-centric heuristics and computer vision.
  • Efficiency at the Edge: INSITE shines brightest when the budget is extremely tight (10–20 images). This makes it particularly valuable for rare diseases or new clinical environments where expert time is at an absolute premium.

Key Results

  • Superiority at Low Budgets: On the APTOS dataset, with a budget of only 10 labeled images, INSITE (using LogDeterminant selection) achieved 0.89 accuracy, vastly outperforming random selection combined with CAGE (0.64).
  • Outperforming Baselines: On the Chest X-Ray dataset, the FacilityLocation + CAGE approach outperformed several state-of-the-art semi-supervised methods (like Mean-Teacher variants) at budgets of 10 and 20 images.
  • Diminishing Returns: Interestingly, as the budget increased to 40 images, some traditional SSL methods began to catch up or slightly exceed INSITE. This suggests INSITE is a specialized tool for the "cold-start" problem in data labeling.

Limitations and Future Work

The authors acknowledge a few key areas for growth:

  • Binary Focus: The current study focuses primarily on binary classification (Normal vs. Diseased). Real-world clinical settings often require multi-class grading (e.g., stages of a disease).
  • Feature Dependency: The system relies on features extracted from a pre-trained ResNet. While effective, these features might not capture the nuanced pathological markers a specialist looks for in every specific medical domain.
  • Future Directions: The team aims to extend the framework to multi-class datasets and explore more complex labeling functions that could incorporate broader clinical metadata.

Summary updated

Abstract

The necessity of large amounts of labeled data to train deep models, especially in medical imaging creates an implementation bottleneck in resource-constrained settings. In Insite (labelINg medical imageS usIng submodular funcTions and sEmi-supervised data programming) we apply informed subset selection to identify a small number of most representative or diverse images from a huge pool of unlabelled data subsequently annotated by a domain expert. The newly annotated images are then used as exemplars to develop several data programming-driven labeling functions. These labelling functions output a predicted-label and a similarity score when given an unlabelled image as an input. A consensus is brought amongst the outputs of these labeling functions by using a label aggregator function to assign the final predicted label to each unlabelled data point. We demonstrate that informed subset selection followed by semi-supervised data programming methods using these images as exemplars perform better than other state-of-the-art semi-supervised methods. Further, for the first time we demonstrate that this can be achieved through a small set of images used as exemplars.
BibTeX
@inproceedings{gautam2024insite,
  author = {Akshat Gautam and Anurag Shandilya and Akshit Srivastava and Venkatapathy Subramanian and Ganesh Ramakrishnan and Kshitij S. Jadhav},
  title = {{Insite: Labelling Medical Images Using Submodular Functions and Semi-Supervised Data Programming}},
  booktitle = {ISBI},
  year = {2024},
  url = {https://openreview.net/forum?id=vQhxeCQQlS},
}

Loading related content...

Figure