SAMS : One-Shot Learning for the Segment Anything Model Using Similar Images

Fan Yin, Jun Li, Yifei Wei, Wei Zhang, Chaoyi XuIJCNN 2024 · OpenReview

Abstract

The field of computer vision is currently transitioning from closed-set to open-set tasks. Vision foundation models have already demonstrated success in open-set scenarios. Building on these models, the utilization of a feature supervision framework can further enhance results. Our paper introduces a new method called SAMS (Segment Anything Model using Similar Images), which is a type of feature supervision framework. It is designed to segment specific masks from visual supervision features. Our framework comprises a pre-trained segmentation model and an efficient, novel prompt generation model capable of generating new prompts based on pre-extracted image features. This innovation eliminates the need for manually crafted prompts in the mask generation phase by integrating the principles of one-shot or few-shot learning with visual instructions from similar images. The effectiveness of the SAMS method is evident in its performance across various tasks, particularly in open-set tasks where traditional models tend to struggle. The pretrained model not only achieves impressive mean Intersection over Union (mIOU) scores without incurring additional time loss, but also demonstrates potential for further improvement through targeted module training.
BibTeX
@inproceedings{yin2024sams,
  author = {Fan Yin and Jun Li and Yifei Wei and Wei Zhang and Chaoyi Xu},
  title = {{SAMS : One-Shot Learning for the Segment Anything Model Using Similar Images}},
  booktitle = {IJCNN},
  year = {2024},
  url = {https://openreview.net/forum?id=UXR3zM9Vff},
}

Loading related content...

Figure