UKBOB: One Billion MRI Labeled Masks for Generalizable 3D Medical Image Segmentation

Emmanuelle Bourigault, Amir Jamaludin, Abdullah Hamdi

TL;DR

The authors introduce UKBOB, the largest labeled medical imaging dataset to date, featuring over 1.37 billion 2D segmentation masks across 72 organs from 51,761 3D MRI scans. By leveraging automated labeling, geometric filtering, and a novel test-time adaptation strategy, they developed Swin-BOB, a foundation model that achieves state-of-the-art results in 3D medical image segmentation across multiple benchmarks.

What Problem Does This Paper Address?

In the broader computer vision world, datasets like ImageNet and LAION have fueled the rise of powerful foundation models. Medical imaging, however, has lagged behind due to a "data desert" caused by stringent privacy regulations, high expert annotation costs, and complex logistics.

Existing medical datasets are often limited to specific regions (e.g., just the brain or abdomen) or lack the scale necessary to train models that generalize well across different scanners, protocols, and modalities. The authors aim to bridge this gap by creating a massive, anatomically diverse dataset that can serve as a cornerstone for generalizable 3D medical image segmentation.

Methods

Creating a billion-mask dataset manually is impossible. The authors employed a multi-stage pipeline to scale up safely:

  1. Automated Labeling: They used the TotalVibeSegmentator tool to generate initial pseudo-labels for the 51,761 MRI scans in the UK Biobank.
  2. Specialized Organ Label Filter (SOLF): To clean the resulting noisy labels, they introduced a filtering mechanism based on biological "blueprints." For each organ, they calculated three geometric features: normalized volume, eccentricity, and sphericity. If an automated mask deviated significantly from statistical norms in at least two of these categories, it was flagged as inaccurate and discarded.
  3. Entropy Test-time Adaptation (ETTA): To handle residual label noise and domain shifts (e.g., moving from MRI to CT), they proposed ETTA. This method fine-tunes the model's batch normalization parameters during inference by minimizing the entropy of the predictions, effectively allowing the model to "self-correct" on new, unseen data.
  4. Foundation Model (Swin-BOB): They trained a Swin-UNetr architecture on the filtered UKBOB dataset to create a robust starting point for downstream medical tasks.
Figure 3. UKBOB-Manual. We collect manual labels for 300 samples of UKBOB for 11 abdominal organs totaling 3,000 images. UKBOB-manual acts as manual validation for the large UKBOB. Examples of axial slices are shown here.

Datasets

  • UKBOB (Ours): The primary contribution, containing 51,761 3D MRI samples and 1.37 billion 2D masks of 72 organs.
  • UKBOB-manual: A subset of 300 MRIs (3,000 images) manually annotated for 11 abdominal organs to validate the automated pipeline.
  • Evaluation Benchmarks: The model was tested on BRATS (brain tumor MRI), BTCV (abdominal CT), and AMOS (abdominal MRI) to prove its generalization capabilities.
Figure 1. UKBOB Size and Diversity. Our proposed UK Biobank Organs and Bones (UKBOB) is the largest labeled medical imaging dataset for segmentation, comprising body organs of 51,761 MRI 3D samples (17.9 M 2D images) and a total of more than 1.37 billion 2D masks of 72 organs. Left: we show label examples from UKBOB from axial, coronal, and sagittal views. Right: We show a plot of the size (number of 2D images) and diversity (number of classes) of our UKBOB compared to other medical images datasets. The size of the bubbles indicates 2D image resolution. This new scale in dataset size and diversity should unlock a new wave of applications and methods in the computer vision and medical imaging communities.

Key Insights and Ideas

  • Scaling Laws in Medical AI: The authors demonstrate a clear "scaling law": doubling the pre-training data size consistently leads to higher Dice scores on downstream tasks. This suggests that medical imaging is finally entering the era where "more data" is a primary driver of performance.
  • Geometric Priors as Quality Control: Instead of relying on flat label statistics, using 3D geometric properties (shape and volume) proved highly effective at distinguishing between segmentation failures and actual patient pathologies.
  • Cross-Modality Generalization: Remarkably, pre-training on MRI data (UKBOB) significantly improved performance on CT scan benchmarks (BTCV). This indicates the model is learning fundamental anatomical representations that transcend the specific imaging modality.

Key Results

  • SOTA Performance: Swin-BOB achieved state-of-the-art results on the BRATS 2023 brain tumor challenge (0.894 Dice score) and the BTCV abdominal benchmark (0.892 Dice score).
  • Zero-Shot Capability: The model showed strong zero-shot generalization on the AMOS dataset, achieving a mean Dice score of 0.840 even without fine-tuning on that specific data.
  • Validation: The automated labels achieved an 81.1% Dice score when compared against manual spine annotations, confirming the high quality of the generated dataset.
Figure 2. Accuracy of UKBOB Labels.An example of segmen- tation labels in UKBOB is shown in the sagittal view. The labels include “spine” (in purple) which we can compare to previously collected hand labels of the spine [5] (in red). We note that the newly collected labels match the manual labels in the spine with a total Dice score of 81.1% on a set of 250 manually annotated test samples, indicating accurate labels.

Limitations and Future Work

  • Anatomical Scope: The current dataset is limited to "neck-to-knee" MRI scans. Future work aims to incorporate head and limb scans, as well as more diverse anatomical regions.
  • Modality Diversity: While the model generalizes to CT, the training data is exclusively MRI. The authors plan to extend the dataset to include native CT scans to further improve cross-modality robustness.
  • Residual Noise: Despite the SOLF filtering, some label noise persists. The authors suggest that integrating clinical metadata (like age or weight) could further refine the filtering process.

Summary updated

Abstract

In medical imaging, the primary challenge is collecting large-scale labeled data due to privacy concerns, logistics, and high labeling costs. In this work, we present the UK Biobank Organs and Bones (UKBOB), the largest labeled dataset of body organs, comprising 51,761 MRI 3D samples (equivalent to 17.9 million 2D images) and more than 1.37 billion 2D segmentation masks of 72 organs, all based on the UK Biobank MRI dataset. We utilize automatic labeling, introduce an automated label cleaning pipeline with organ-specific filters, and manually annotate a subset of 300 MRIs with 11 abdominal classes to validate the quality (referred to as UKBOB-manual). This approach allows for scaling up the dataset collection while maintaining confidence in the labels. We further confirm the validity of the labels by demonstrating zero-shot generalization of trained models on the filtered UKBOB to other small labeled datasets from similar domains (e.g., abdominal MRI). To further mitigate the effect of noisy labels, we propose a novel method called Entropy Test-time Adaptation (ETTA) to refine the segmentation output. We use UKBOB to train a foundation model, Swin-BOB, for 3D medical image segmentation based on the Swin-UNetr architecture, achieving state-of-the-art results in several benchmarks in 3D medical imaging, including the BRATS brain MRI tumor challenge (with a 0.4% improvement) and the BTCV abdominal CT scan benchmark (with a 1.3% improvement). The pre-trained models and the code are available at https://emmanuelleb985.github.io/ukbob , and the filtered labels will be made available with the UK Biobank.
BibTeX
@inproceedings{bourigault2025ukbob,
  author = {Emmanuelle Bourigault and Amir Jamaludin and Abdullah Hamdi},
  title = {{UKBOB: One Billion MRI Labeled Masks for Generalizable 3D Medical Image Segmentation}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2025},
}
Figure