Self-Prompt SAM: Automatic Prompt SAM Adaptation for Medical Image Segmentation
Its authors withdrew this paper, so there is no full text to read or summarize.
Lists
Only you see this
Abstract
The Segment Anything Model (SAM), a prompt-driven foundation model for natural image segmentation, has demonstrated impressive zero-shot performance and brought a range of unexplored capabilities to natural image segmentation tasks. However, as a very important branch of image segmentation, the performance of SAM remains uncertain when it is applied to medical image segmentation due to the significant differences between natural images and medical images. Meanwhile, it is harsh to meet the requirements of extra prompts provided, such as points or boxes to specify medical regions, since medical knowledge is not expected from users. In this paper, we aim to adapt pre-trained SAM models worked on from 2D natural images to 3D medical images without any prompts provided. Through the analysis of SAM models, we propose a novel self-prompt SAM adaptation framework for medical image segmentation, named Self-Prompt SAM. We designed a multi-scale prompt generator combined with the image encoder in SAM to generate auxiliary masks. Then, we use the auxiliary masks to generate bounding boxes as box prompts and utilize Distance Transform to select the points farthest from the boundary as point prompts. Meanwhile, we designed a 3D depth-fused adapter (DfusedAdapter) and injected the DFusedAdapter into each transformer block in the image encoder and mask decoder to enable pre-trained 2D SAM models to extract 3D information and adapt to 3D medical images. Extensive experiments demonstrate that our method outperforms existing state-of-the-art approaches on two challenging public ACDC and Synapse datasets.
BibTeX
@misc{xie2024self,
author = {Bin Xie and Hao Tang and Dawen Cai and Yan Yan},
title = {{Self-Prompt SAM: Automatic Prompt SAM Adaptation for Medical Image Segmentation}},
year = {2024},
url = {https://openreview.net/forum?id=uK4TYkVBJG},
howpublished = {OpenReview},
note = {Submitted to ICLR 2024},
}Loading related content...