Introduction
In the fields of radiology and oncology, segmenting anatomical structures such as organs or tumors is a critical but labor-intensive task. While the "Segment Anything Model" (SAM) revolutionized interactive segmentation for natural images, its application to 3D medical data (CT and MRI) has been inconsistent. The RadioActive benchmark introduces a rigorous framework to bring order to this "Wild West" of 3D medical segmentation, establishing a standard for comparing 2D and 3D models fairly.
The Pitfalls of Current Research
The authors identify three major pitfalls in existing medical segmentation studies:
- Unrealistic Interaction (P1): Many benchmarks evaluate 2D models by assuming a clinician will provide prompts for every single slice of a 3D volume—a prohibitive time investment in clinical practice.
- Neglecting Refinement (P2): Most studies focus only on the "first click," ignoring the iterative process where humans correct AI errors.
- Inconsistent Evaluation (P3): A lack of standard datasets and metrics makes it impossible to determine which models truly lead the field.
Realistic Prompting Strategies
To address these pitfalls, RadioActive introduces Realistic Prompting Strategies that bridge the gap between 2D models and 3D volumes. These strategies simulate how a clinician would actually interact with a 3D dataset, allowing researchers to measure "Human Effort" (quantified by click count) on a level playing field.
- Interpolation: Instead of prompting every slice, a user provides points or boxes for a few key slices (e.g., top, middle, and bottom). The system mathematically interpolates these prompts for all intermediate slices.
- Propagation: The user prompts a single central slice, and the model uses its own prediction to "guess" the prompt location on adjacent slices, moving outward through the volume.
- Scribble Refinement: To simulate real-world correction, an algorithm generates "scribbles" on areas where the model failed, testing how well a model learns from iterative feedback.
Benchmark Methodology and Datasets
The benchmark utilizes 10 diverse, publicly available datasets covering CT and MRI modalities. To ensure a fair "zero-shot" evaluation, the authors selected datasets that were not part of the models' original training data. Target structures include healthy organs and various pathologies, such as MS lesions, lung tumors, and liver metastases.
Key Insights and Results
The benchmark yielded several surprising findings that challenge current assumptions in medical AI:
- SAM 2 Dominance: General-purpose models are proving highly effective. SAM 2 emerged as the new state-of-the-art for medical imaging, achieving a mean Dice Similarity Coefficient (DSC) of 81.88 using box interpolation. It outperformed specialized models trained on millions of medical images.
- The Power of Bounding Boxes: Bounding boxes consistently provide better spatial context than points. A few well-placed boxes are often sufficient for near-perfect segmentation.
- Efficiency of 2D Models: RadioActive demonstrates that with smart interpolation, 2D models can match the efficiency of native 3D models without sacrificing accuracy.
- Refinement is Essential: Models that lack support for iterative mask refinement (such as the original MedSAM) are significantly less useful in clinical settings where precision is paramount.
Among native 3D models, SegVol was the most competitive, particularly with point-based prompts, though it still struggled to match the box-based performance of SAM 2. However, the benchmark revealed that nearly all current models still struggle with small, low-contrast pathological lesions.
Conclusion and Future Work
RadioActive provides a necessary foundation for evaluating interactive 3D segmentation. While the current "Human Effort Proxy" (counting clicks) is a simplification, it offers a scalable way to compare model architectures. Future work will involve user studies with clinicians to measure real-world annotation time and cognitive load, as well as integrating the RadioActive codebase into standard medical imaging toolkits.
Summary updated