A Multi-armed Bandit to Smartly Select a Training Set from Big Medical Data

Benjamín Gutiérrez, Loïc Peter, Tassilo Klein, Christian Wachinger

Abstract: Large-scale medical imaging datasets present significant computational and heterogeneity challenges. This paper proposes a Multi-Armed Bandit (MAB) framework that utilizes metadata to intelligently select training samples. By employing Thompson Sampling, the method achieves higher prediction accuracy for brain age estimation using only a fraction of the available data, demonstrating superior efficiency and domain adaptation capabilities.

Introduction

In the current era of "Big Data," the prevailing philosophy in machine learning is often that more data leads to better performance. However, in the field of medical imaging, this approach faces significant practical hurdles. Processing thousands of high-resolution brain MRIs is computationally expensive, and the inherent heterogeneity of multi-site datasets can sometimes introduce noise that degrades model accuracy.

This paper, originally presented at MICCAI 2017, proposes a novel reinforcement learning strategy to navigate these challenges. Instead of a brute-force approach that processes every available scan, the authors introduce a Multi-Armed Bandit (MAB) framework to "smartly" select the most informative training samples from a large, diverse pool.

The Challenge of Big Medical Data

The proliferation of public medical data initiatives has made thousands of brain scans available to researchers. Despite this abundance, two major obstacles remain:

  1. Computational Cost: Pre-processing tools like FreeSurfer, which extract anatomical measurements from raw images, are notoriously slow. Processing 10,000 images simply to determine their utility is a massive expenditure of time and resources.
  2. Data Heterogeneity: Medical datasets are often messy, comprising scans from different hospitals, scanners, and patient populations. Blindly aggregating these datasets can introduce site-specific biases or noise that hinders a model's ability to generalize.

The central question addressed by this research is: Can we predict which samples are worth processing based only on their "cheap" metadata (e.g., age, sex, diagnosis) before performing expensive image analysis?

Methodology: Multi-Armed Bandits for Sample Selection

The authors frame the selection of training samples as a Multi-Armed Bandit (MAB) problem. In this paradigm, an agent must choose between several options ("arms") to maximize a reward, balancing the exploration of unknown options with the exploitation of known high-performing ones.

The MAB Framework

  • The "Arms": Each arm represents a cluster of data defined by metadata attributes, such as "Males from the ABIDE dataset" or "Subjects aged 20-30."
  • The Strategy: The researchers employed Thompson Sampling. For each cluster, the algorithm maintains a probability distribution (specifically a Beta distribution) that represents the likelihood of a sample from that cluster being "useful."
  • The Selection Process:
    1. The algorithm selects a cluster based on its current estimated success rate.
  • A single sample is "revealed" from that cluster (i.e., its image features are processed).
  • This sample is added to the training set, and a Ridge Regression model is retrained.
  • The Reward: If the model’s accuracy ($r^2$ score) on a validation set improves, the cluster receives a "success" (+1); otherwise, it receives a "failure."

This iterative process allows the system to focus on clusters that consistently improve the model while occasionally exploring ignored clusters to identify potentially valuable data types.

Experimental Setup

The study utilized an aggregate of 10 public datasets, totaling 7,250 subjects (including ABIDE, ADHD200, AIBL, COBRE, IXI, GSP, HCP, MCIC, PPMI, and OASIS). The primary task was brain age estimation—predicting a subject's chronological age from brain volume and thickness measurements. The dataset was highly heterogeneous, spanning a wide range of ages, sexes, and clinical diagnoses.

Results and Discussion

The experimental results demonstrate that the MAB selection (MABS) approach significantly outperforms random sampling in both efficiency and final accuracy.

Comparison of $r^2$ scores for different sampling strategies across various test sets.

Key Findings

  • Efficiency and Accuracy: As shown in the "Prediction on All Data" plot, MABS reaches higher $r^2$ scores much faster than random selection. In many cases, a guided selection of approximately 1,000 samples outperformed a random selection of 4,000 samples. This suggests that "bad" or redundant data points can act as distractors, and identifying the "saturation point" can save hundreds of hours of computation.
  • Domain Adaptation: The framework proved particularly effective for target-specific predictions. When evaluating on specific datasets like COBRE, CORR, or OASIS, the MAB successfully "tailored" the training set by prioritizing samples from other datasets that shared similar characteristics with the target. For instance, in the CORR dataset, MABS (Diagnostic) reached peak performance with significantly fewer samples than random selection.
  • Metadata as a Proxy: The results confirm that metadata is a powerful proxy for data utility. The MAB naturally identifies which demographic or site-specific clusters are most beneficial for the current state of the model. Furthermore, the approach is robust; even when metadata (like 'sex' for age prediction) is less relevant, Thompson Sampling learns to ignore those clusters in favor of more informative ones.

Conclusion and Future Directions

The Multi-Armed Bandit approach offers a scalable solution for managing the computational and qualitative challenges of big medical data. By intelligently selecting training samples, researchers can build more accurate models with significantly less data processing.

While the current method requires model retraining in each iteration—which could be a bottleneck for large neural networks—it opens doors for more efficient data utilization. Future work may involve using "Contextual Bandits" to handle continuous metadata more effectively or applying this framework to guide prospective patient recruitment in clinical settings.

Summary updated

BibTeX
@inproceedings{gutierrez2017multi,
  author = {Benjamín Gutiérrez and Loïc Peter and Tassilo Klein and Christian Wachinger},
  title = {{A Multi-armed Bandit to Smartly Select a Training Set from Big Medical Data}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2017},
  doi = {10.1007/978-3-319-66179-7\_5},
}
Figure