Introduction
Deep learning has revolutionized medical image analysis, particularly in the segmentation of complex lesions in CT scans. However, the development of robust models is often hindered by "data hunger." High-quality medical annotations require significant time and expertise from radiologists, making large-scale labeled datasets expensive and difficult to acquire. Furthermore, when researchers attempt to adapt massive pre-trained models to small, specialized datasets, they often encounter the problem of overfitting—where the model memorizes specific training examples rather than learning generalizable features.
To address these challenges, this paper introduces SLPT (Selective Labeling Meets Prompt Tuning). SLPT is a synergistic framework that combines parameter-efficient prompt tuning with a novel active learning strategy called TESLA (TandEm Selective LAbeling). By tuning only a fraction of the model's parameters and strategically selecting the most informative data for human labeling, SLPT achieves high-performance segmentation with minimal human intervention.
Methodology
The SLPT framework is built upon two primary innovations designed to maximize efficiency in both the model architecture and the data selection process.
1. Feature-aware Prompt Updater (FPU)
Traditional fine-tuning updates all parameters of a pre-trained model, which is computationally expensive and prone to overfitting on small datasets. SLPT instead utilizes "prompt tuning," a technique where the backbone of the model remains frozen. The authors introduce the Feature-aware Prompt Updater (FPU), which embeds prompt modules throughout the deep layers of a pre-trained Convolutional Neural Network (CNN). Unlike simple input-layer prompts, FPUs allow the prompts to interact dynamically with the image features at multiple scales, providing the model with the expressiveness needed for complex segmentation tasks while only training approximately 6% of the total parameters.
2. TESLA: TandEm Selective LAbeling
To solve the problem of which images should be labeled, the authors developed TESLA, a two-stage active learning strategy:
- Stage 0: Unsupervised Diversity: Before any human labels are provided, the system uses the frozen pre-trained backbone to analyze the entire unlabeled pool. It identifies a diverse set of representative images to serve as the initial training batch, ensuring the model starts with a broad understanding of the data distribution.
- Stage 1: Supervised Uncertainty: Once initial labels are obtained, the model employs "Diversified Visual Prompt Tuning." By generating multiple predictions for the same image using different prompts, the system can measure model confidence. If different prompts produce conflicting segmentations (high inter-prompt divergence) or if the prompts require significant gradient updates to process an image, that image is flagged as "uncertain" and prioritized for expert annotation.
Experimental Evaluation
The model was pre-trained on public liver segmentation datasets (such as LiTS) to establish a baseline for anatomical features. For downstream evaluation, the authors utilized a challenging in-house dataset of 941 CT scans from a clinical setting. This dataset is particularly rigorous, containing eight different types of hepatic lesions, including rare cases like hepatoblastomas and cysts, which are often difficult for standard models to identify.
Key Results
The SLPT framework demonstrated remarkable efficiency and performance across several metrics:
- Parameter Efficiency: SLPT requires only 2.71M trainable parameters, a 94% reduction compared to the 44.81M parameters required for standard full fine-tuning.
- Performance in Low-Data Regimes: When restricted to only 40 labeled patients, SLPT outperformed full fine-tuning by 5.4% in Mean Dice score. This suggests that prompt tuning acts as a powerful regularizer, preventing the model from overfitting to small sample sizes.
- The 5% Milestone: Using the TESLA selection strategy, the model reached 94% of the performance of a model trained on the entire 752-patient training set, while only requiring labels for 5% (40 patients) of the data.
Conclusion
SLPT represents a significant step forward in making medical AI more sustainable and accessible. By treating prompting not just as a tuning mechanism but as an "uncertainty sensor," the framework bridges the gap between model-centric and data-centric AI. The results demonstrate that we do not always need more data; rather, we need a more intelligent way to select data and a more efficient way to adapt our models. Future work will focus on extending SLPT to other modalities, such as MRI and X-ray, and automating the initialization of prompts for a wider variety of clinical tasks.
Summary updated