Extracting 2D weak labels from volume labels using multiple instance learning in CT hemorrhage detection

Samuel Remedios, Zihao Wu, Camilo Bermudez, Cailey I. Kerley, Snehashis Roy, Mayur B. Patel, John A. Butman, Bennett A. Landman, Dzung L. PhamMedical Imaging: Image Processing 2019 · DOI · OpenReview

TL;DR

This paper introduces a novel application of Multiple Instance Learning (MIL) to extract detailed 2D hemorrhage information from 3D CT scans, using only whole-volume labels. This approach bypasses the need for laborious slice-by-slice annotations, demonstrating strong performance and providing insights into the data requirements for effective MIL in medical imaging.

What Problem Does This Paper Address?

Medical imaging, particularly CT scans, often presents challenges due to anisotropic voxel resolutions (meaning slices are much thinner than they are wide). This makes applying standard 3D deep learning models difficult. While 2D models can work on individual slices, they typically require precise, slice-level annotations, which are time-consuming and expensive to obtain. Clinicians, however, often provide "weak" labels at the volume level (e.g., "hemorrhage present in this scan"). This paper tackles the problem of bridging this gap: how can we train accurate 2D models for hemorrhage detection using only these readily available, but less precise, 3D volume labels, without needing detailed 2D slice annotations?

Methods

The core of this work is the application of Multiple Instance Learning (MIL) to CT hemorrhage detection. In MIL, data is organized into "bags" (here, entire 3D CT volumes) and "instances" (here, individual 2D slices within a volume). A bag is labeled positive if at least one of its instances is positive, and negative only if all instances are negative.

The authors trained a ResNet-34 deep convolutional neural network. For each CT volume (bag), the model processed each 2D slice (instance) to predict the probability of hemorrhage. The key MIL aspect is the "max pooling" strategy: the model identifies the slice with the highest predicted probability of being positive. Gradients are then calculated only for this most-positive instance. These gradients are accumulated over a batch of volumes before being applied to update the model. This allows the model to learn from the "weak" bag-level labels by focusing on the most indicative slices.

Datasets

The study utilized a dataset of 4,882 CT image volumes from 4,033 patients collected at Vanderbilt University Medical Center. These volumes were initially processed by an automated segmentation network and then manually reviewed by a trained rater. The ground truth labels were binary: a volume was labeled as positive if it contained a hemorrhage larger than 20 cm³, and negative otherwise. The dataset was split into training (80%), validation (10%), and testing (10%) sets, with careful undersampling of negative samples to balance classes in training and validation.

Key Insights and Ideas

  • MIL as a Bridge: MIL proves to be a powerful paradigm for leveraging weak, volume-level labels to train detailed 2D models, effectively overcoming the annotation bottleneck in medical imaging.
  • Anisotropy Advantage: The paper highlights how MIL is particularly well-suited for anisotropic CT data, as it allows 2D slice-based processing without requiring isotropic resampling or complex 3D architectures.
  • Data Requirements are Crucial: The study provides valuable empirical evidence on the sample size needed for MIL to converge and generalize effectively, suggesting that a significant number of annotated volumes are necessary.
  • Focusing on the "Most Positive": The "max pooling" approach in MIL, by concentrating on the most indicative instances, is shown to be an effective mechanism for learning from noisy or imprecise labels.

Key Results

The leading model (Model 672), trained on the full dataset, achieved impressive results:

  • Overall Accuracy: 99.33%
  • Negative Class Accuracy: 99.36% (with 25 false positives)
  • Positive Class Accuracy: 98.10% (with 2 false negatives)
  • Average Precision: 0.9698 (over five-fold cross-validation)

Crucially, the study investigated the impact of training data size. They found that:

  • Models trained with fewer than 100 samples were unable to generalize to the test set.
  • At least 400 patient image volumes were needed to achieve accurate per-slice hemorrhage detection and a generalizable model.
  • Performance gains from adding data beyond 500 volumes were significantly reduced.

Limitations and Future Work

The authors acknowledge that the ground truth labels were binary (hemorrhage present/absent in the volume) and not slice-specific. While their MIL approach successfully inferred slice-level information, the manual labeling errors in the dataset (as shown in Figure 5) represent a limitation.

Future work could explore incorporating information from clinical reports (which are often text-based and contain weak labels) to further enhance the training of 2D segmentation approaches. The authors also suggest that MIL could be valuable for pre-training models or for transfer learning in other weakly supervised medical imaging tasks. The code and models are made publicly available to facilitate further research.

Summary updated

Abstract

Multiple instance learning (MIL) is a supervised learning methodology that aims to allow models to learn instance class labels from bag class labels, where a bag is defined to contain multiple instances. MIL is gaining traction for learning from weak labels but has not been widely applied to 3D medical imaging. MIL is well-suited to clinical CT acquisitions since (1) the highly anisotropic voxels hinder application of traditional 3D networks and (2) patch-based networks have limited ability to learn whole volume labels. In this work, we apply MIL with a deep convolutional neural network to identify whether clinical CT head image volumes possess one or more large hemorrhages (&gt;; 20cm<sup>3</sup> ), resulting in a learned 2D model without the need for 2D slice annotations. Individual image volumes are considered separate bags, and the slices in each volume are instances. Such a framework sets the stage for incorporating information obtained in clinical reports to help train a 2D segmentation approach. Within this context, we evaluate the data requirements to enable generalization of MIL by varying the amount of training data. Our results show that a training size of at least 400 patient image volumes was needed to achieve accurate per-slice hemorrhage detection. Over a five-fold cross-validation, the leading model, which made use of the maximum number of training volumes, had an average true positive rate of 98.10%, an average true negative rate of 99.36%, and an average precision of 0.9698. The models have been made available along with source code<sup>1</sup> to enabled continued exploration and adaption of MIL in CT neuroimaging.
BibTeX
@inproceedings{remedios2019extracting,
  author = {Samuel Remedios and Zihao Wu and Camilo Bermudez and Cailey I. Kerley and Snehashis Roy and Mayur B. Patel and John A. Butman and Bennett A. Landman and Dzung L. Pham},
  title = {{Extracting 2D weak labels from volume labels using multiple instance learning in CT hemorrhage detection}},
  booktitle = {Medical Imaging: Image Processing},
  year = {2019},
  doi = {10.1117/12.2549356},
}
Figure