Abstract
Active learning generally involves querying the most representative samples for human labeling, which has been widely studied in many fields such as image classification and object detection. However, its potential has not been explored in the more complex instance segmentation task that usually has relatively higher annotation cost. In this paper, we propose a novel and principled semi-supervised active learning framework for instance segmentation. Specifically, we present an uncertainty sampling strategy named Triplet Scoring Predictions (TSP) to explicitly incorporate samples ranking clues from classes, bounding boxes and masks. Moreover, we devise a progressive pseudo labeling regime using the above TSP in semi-supervised manner, it can leverage both the labeled and unlabeled data to minimize labeling effort while maximize performance of instance segmentation. Results on medical images datasets demonstrate that the proposed method results in the embodiment of knowledge from available data in a meaningful way. The extensive quantitatively and qualitatively experiments show that, our method can yield the best-performing model with notable less annotation costs, compared with state-of-the-arts.
Introduction
Instance segmentation is a fundamental yet challenging task in computer vision, requiring simultaneous object classification, localization, and pixel-level masking. The complexity of this task translates to exceptionally high annotation costs, particularly in specialized domains like medical imaging. While Active Learning (AL) aims to reduce this burden by selecting the most informative samples for manual labeling, it often overlooks the potential of the remaining unlabeled data.
This paper introduces a novel framework that bridges the gap between Active Learning and Semi-supervised Learning (SSL). By combining a multi-dimensional uncertainty sampling strategy with a progressive pseudo-labeling regime, the proposed method achieves state-of-the-art performance while significantly reducing the human effort required for dataset curation.
Methodology
The proposed framework is built upon the Mask R-CNN architecture and introduces two key innovations to handle the complexities of instance segmentation.
Triplet Scoring Predictions (TSP)
Traditional active learning often relies solely on classification confidence. However, in instance segmentation, a model may be certain about a class but uncertain about the object's boundaries. To address this, the authors introduce two parallel branches to the RCNN and Mask heads to predict Bounding Box IoU and Mask IoU scores. The final uncertainty metric, Triplet Scoring Predictions (TSP), integrates scores from all three dimensions: class, box, and mask. This ensures that the model queries samples that are difficult in any aspect of the segmentation task.
Progressive Pseudo-labeling and Noise-Robust Loss
The framework utilizes a dual-path data flow. "Hard" samples identified by low TSP scores are sent for human labeling. Conversely, "easy" samples with high TSP scores are automatically assigned pseudo-labels. To mitigate the risk of "confirmation bias"—where the model learns from its own incorrect predictions—the authors implemented a noise-robust semi-supervised loss function ($L_{semi}$). This loss focuses on high-confidence positive regions while ignoring potential background noise.
Experimental Results
The framework was evaluated on two large-scale medical datasets: Head CT scans for Intracranial Hemorrhage (ICH) and Retinal OCT for Edema Lesions.
Performance and Efficiency
The results demonstrate a substantial reduction in manual labor. On the ICH dataset, the proposed semi-supervised TSP method achieved a mean Average Precision (mAP) of 0.4869 using only 1,500 labeled images. A standard baseline required double the amount of data (3,000 images) to reach a comparable performance level (0.4892 mAP).
Quantitatively, this efficiency translates to saving approximately 4,500 minutes of expert radiologist labor and over $10,000 in annotation costs for a single dataset. The synergy between AL (focusing on boundary cases) and SSL (stabilizing with certain cases) allows for a more complete representation of the data space with minimal supervision.
Conclusion
This work presents a principled approach to reducing the annotation bottleneck in instance segmentation. By holistically scoring predictions across classification and localization dimensions, the TSP strategy effectively identifies informative samples. When combined with semi-supervised pseudo-labeling, the framework maximizes the utility of the entire data pool. Future research will focus on extending this methodology to general-purpose natural image datasets and exploring its sensitivity to pseudo-labeling hyperparameters.
Summary updated