ASLseg: Adapting SAM in the Loop for Semi-supervised Liver Tumor Segmentation

Shiyun Chen, Lin Li, Pujin Cheng, Xiaoying Tang

Abstract

Liver tumor segmentation is essential for computer-aided diagnosis, surgical planning, and prognosis evaluation. However, obtaining and maintaining a large-scale dataset with dense annotations is challenging. Semi-Supervised Learning (SSL) is a common technique to address these challenges. Recently, Segment Anything Model (SAM) has shown promising performance in some medical image segmentation tasks, but it performs poorly for liver tumor segmentation. In this paper, we propose a novel semi-supervised framework, named ASLseg, which can effectively adapt the SAM to the SSL setting and combine both domain-specific and general knowledge of liver tumors. Specifically, the segmentation model trained with a specific SSL paradigm provides the generated pseudo-labels as prompts to the fine-tuned SAM. An adaptation network is then used to refine the SAM-predictions and generate higher-quality pseudo-labels. Finally, the reliable pseudo-labels are selected to expand the labeled set for iterative training. Extensive experiments on the LiTS dataset demonstrate overwhelming performance of our ASLseg.

Introduction

Accurate liver tumor segmentation is a cornerstone of modern oncology, yet it remains a significant challenge for deep learning due to a severe data bottleneck. Training robust models typically requires massive amounts of pixel-perfect annotations, which are expensive, time-consuming, and require expert radiological knowledge.

Semi-supervised learning (SSL) attempts to mitigate this by training on a small set of labeled data alongside a larger set of unlabeled data. However, SSL heavily relies on "pseudo-labels"—the model’s predictions for unlabeled images. If these initial guesses are noisy or inaccurate, the model risks learning from its own errors, a phenomenon that limits performance on complex, indistinct liver tumors. While the Segment Anything Model (SAM) has revolutionized general image segmentation, it lacks the specific "medical intuition" required to identify liver tumors reliably on its own.

Proposed Method: ASLseg

The proposed ASLseg framework is a three-stage pipeline designed to bridge the gap between SAM's general edge-awareness and domain-specific medical expertise.

The three-stage ASLseg framework: initial SSL training, SAM-in-the-loop refinement, and final Adaptation Network processing.

Stage I: Initial SSL Training

A standard segmentation model (e.g., U-Net) is first trained using a semi-supervised paradigm (specifically R-Drop). This model generates initial, though imperfect, pseudo-labels for the unlabeled dataset.

Stage II: SAM in the Loop

These initial pseudo-labels are converted into prompts (such as coordinate points) and fed into a fine-tuned Medical SAM. This allows SAM to leverage its general knowledge of object boundaries to "clean up" the tumor shapes based on the domain-specific model's suggestions.

Stage III: The Adaptation Network (AN)

The most novel component is the Adaptation Network (AN), a separate U-Net trained to act as a refiner. To train the AN, ground-truth masks are intentionally distorted (simulating common errors like over-segmentation or edge shrinkage). The AN learns to reconstruct the correct mask from these "corrupted" inputs and the original image. In the final pipeline, the AN refines the SAM-generated labels further. Reliable labels are then selected via a consistency check (high Dice similarity between the SSL model and AN) to expand the training set iteratively.

Experimental Results

The framework was evaluated on the LiTS (Liver Tumor Segmentation) dataset. To simulate a data-scarce environment, only 10% of the data was used as labeled, with the remaining 90% treated as unlabeled.

Quantitative Performance

ASLseg demonstrated significant improvements over nine state-of-the-art semi-supervised methods. It achieved a Dice Similarity Coefficient (DSC) of 74.28%, outperforming the best baseline (RD at 72.16%). Notably, it significantly closed the gap with the fully supervised baseline (76.16%), which utilized ten times more labeled data.

Qualitative Analysis

Visual comparisons highlight ASLseg's superior ability to maintain precise boundaries and identify small tumors where other models often fail.

Qualitative comparison of ASLseg against other state-of-the-art semi-supervised methods across three different cases.

As shown in the figure above, ASLseg produces masks that are much more consistent with the ground truth (GT) across various challenging cases, avoiding the "blob-like" inaccuracies seen in methods like MT or CCT.

Conclusion

ASLseg successfully integrates the general-purpose capabilities of foundation models like SAM with domain-specific medical segmentation networks. By utilizing an Adaptation Network to learn from simulated failures, the framework creates a robust iterative loop that generates high-quality pseudo-labels. Future work will focus on expanding the framework to 3D volumes and exploring more diverse prompt types for SAM to further enhance segmentation accuracy.

Summary updated

Abstract

Liver tumor segmentation is essential for computer-aided diagnosis, surgical planning, and prognosis evaluation. However, obtaining and maintaining a large-scale dataset with dense annotations is challenging. Semi-Supervised Learning (SSL) is a common technique to address these challenges. Recently, Segment Anything Model (SAM) has shown promising performance in some medical image segmentation tasks, but it performs poorly for liver tumor segmentation. In this paper, we propose a novel semi-supervised framework, named ASLseg, which can effectively adapt the SAM to the SSL setting and combine both domain-specific and general knowledge of liver tumors. Specifically, the segmentation model trained with a specific SSL paradigm provides the generated pseudo-labels as prompts to the fine-tuned SAM. An adaptation network is then used to refine the SAM-predictions and generate higher-quality pseudo-labels. Finally, the reliable pseudo-labels are selected to expand the labeled set for iterative training. Extensive experiments on the LiTS dataset demonstrate overwhelming performance of our ASLseg.
BibTeX
@misc{chen2023aslseg,
  author = {Shiyun Chen and Lin Li and Pujin Cheng and Xiaoying Tang},
  title = {{ASLseg: Adapting SAM in the Loop for Semi-supervised Liver Tumor Segmentation}},
  year = {2023},
  eprint = {2312.07969},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.07969},
}
Figure