Introduction
The Segment Anything Model (SAM) has set a new benchmark in computer vision, demonstrating remarkable zero-shot generalization across diverse image domains. However, its application in medical imaging remains a significant challenge. Clinical workflows face two primary hurdles: the expertise bottleneck and the domain gap.
SAM typically requires manual prompts—points or bounding boxes—from experts to function effectively in "Prompt Mode." In a high-pressure clinical environment, radiologists and doctors rarely have the time to manually prompt every slice of a 3D scan. Conversely, SAM’s "Everything Mode" generates masks without semantic labels, leaving clinicians with unclassified regions that lack diagnostic utility. Furthermore, because SAM was trained predominantly on natural images, it often struggles with the unique textures, low contrast, and specific anatomical constraints inherent in medical modalities like CT, MRI, and ultrasound.
To address these issues, researchers have introduced EviPrompt, a novel, training-free method that automates medical image segmentation by treating it as an "in-context" learning task. By requiring only a single reference image-annotation pair, EviPrompt generates high-quality prompts that allow SAM to bridge the gap between natural and medical domains without expensive fine-tuning.
The EviPrompt Framework
EviPrompt operates on the principle that medical anatomy is consistent. It leverages a single annotated reference image to "teach" SAM how to identify specific organs or lesions in new, unseen target images. The pipeline consists of several innovative components designed to maximize SAM's internal attention mechanisms.
1. The 2x2 Comparator Board
Rather than processing images in isolation, EviPrompt utilizes a "2x2 Comparator Board." This grid stitches together one annotated reference image and three versions of the target image: the original, a geometrically transformed version, and a photometrically transformed version. By presenting these images simultaneously, the method forces SAM’s transformer-based backbone to perform cross-attention. This allows the model to "see" the relationship between the known label in the reference and the features in the target image in real-time.
2. Evidential Prompt Generation via Subjective Logic
Standard Softmax outputs in neural networks can often be overconfident, even when incorrect. EviPrompt addresses this by employing Subjective Logic to model uncertainty. The system extracts features from both reference and target images, calculates similarities, and treats these as "evidence." By applying a Dirichlet distribution, EviPrompt calculates a "belief mass" for each pixel. This allows the system to quantify its own certainty; points with the highest belief mass are selected as prompts, ensuring that SAM is guided by the most reliable data points.
3. Multi-Expert Committee
By using a "perturbation set" (the transformed versions of the target image), the system functions like a committee of experts. It aggregates belief masses from different views—such as rotated or color-shifted versions of the same image—to ensure that the selected prompt points are robust and not merely artifacts of a specific orientation or lighting condition.
4. Iterative Refinement
The process concludes with a refinement loop. The system generates an initial mask from the selected point prompts, converts that mask into a bounding box, and then re-prompts SAM with both the points and the box. This iterative approach produces a final, fine-grained segmentation that is more accurate than a single-pass attempt.
Experimental Evaluation
The researchers evaluated EviPrompt across six diverse medical imaging datasets, covering a wide range of modalities and anatomical structures:
- FLARE22: CT scans for Liver segmentation.
- Decathlon: MR scans for Prostate segmentation.
- JSRT: Chest X-rays (CXR) for Lung segmentation.
- UW Skin Cancer: Dermoscopy for Skin cancer.
- RIGA: Fundus images for Retinal disc segmentation.
- Kvasir SEG: Endoscopy for Polyp segmentation.
Key Results
EviPrompt demonstrated superior performance compared to existing benchmarks. Notably, in point-prompt mode, it significantly outperformed MedSAM—a version of SAM specifically fine-tuned on massive medical datasets—despite EviPrompt requiring no training whatsoever.
In the RIGA (Retinal disc) dataset, EviPrompt achieved higher Dice Similarity Coefficients (DSC) than an "Oracle" baseline (where prompts are derived perfectly from ground truth). This suggests that the method's evidential refinement process can actually correct for human-like ambiguity in manual labeling. Furthermore, the method proved highly efficient; on the JSRT dataset, it achieved a DSC of 88.27% using a ViT-H backbone with only five points.
Limitations and Future Work
While EviPrompt represents a significant leap forward for training-free medical segmentation, some challenges remain. The system still struggles with low-contrast environments, such as in certain polyp or prostate images where the target tissue closely resembles the background. Additionally, small anatomical structures remain difficult for point-based prompting, as a single misplaced point can significantly degrade the resulting mask.
Future research may focus on more sophisticated methods for injecting human prior knowledge into the prompt generation process, further narrowing the gap between general-purpose foundation models and specialized clinical tools. By removing the need for massive labeled datasets and expert manual intervention, EviPrompt paves the way for more accessible and scalable AI-assisted diagnostics.
Summary updated