Introduction
Instance segmentation is a cornerstone of computer vision, yet its reliance on dense, pixel-level annotations makes it prohibitively expensive for many large-scale applications. Weakly-Supervised Instance Segmentation (WSIS) offers a cost-effective alternative by utilizing image-level labels. However, real-world applications often require models to learn new classes incrementally over time. This paper presents the first framework for Class-incremental Continual Learning for Instance Segmentation with Image-level Weak Supervision (CL4WSIS). This approach addresses the dual challenges of catastrophic forgetting and background shift in an incremental setting, all while maintaining the low annotation cost of image-level supervision.
Methodology
The CL4WSIS framework employs a two-phase training strategy to bridge the gap between semantic and instance-level understanding.
Two-Phase Training and Instance Cue Generation
The model first performs Weakly-Supervised Semantic Segmentation (WSSS) to establish semantic boundaries. These boundaries are then used to generate synthetic instance cues, including center points and offset maps. A novel Peak Generator (PG) module identifies object centers specifically within the semantic foreground, ensuring that instance cues are not erroneously placed in the background.
Enhancing Robustness and Preventing Forgetting
To improve mask quality, the framework utilizes Feature-Level Augmentation Consistency (FLAC), applying augmentations directly to internal feature maps to enforce consistency. To combat catastrophic forgetting, Semantic-aware Selective Distillation is used. Unlike standard distillation, this method selectively transfers knowledge from the previous model iteration only in regions where the old model is highly confident. This prevents the "background shift" problem, where new classes might be incorrectly suppressed because the old model perceives them as background.
Experimental Results
The proposed method was rigorously evaluated on Pascal SBD 2012, Pascal VOC 2012, and COCO datasets across various incremental splits.
As shown in the results table below, the framework significantly outperforms existing WSIS techniques adapted for incremental learning. On the Pascal SBD 15-5 overlap task, the model achieved an [email protected] of 43.9%, representing a substantial 10.8% improvement over previous benchmarks. The model also demonstrated strong performance in cross-dataset transfer tasks (COCO-to-VOC), proving its robustness across different data distributions.
Conclusion
CL4WSIS represents a significant step forward in making instance segmentation more scalable and adaptable. By successfully combining image-level weak supervision with class-incremental learning, the framework enables models to evolve without the need for expensive re-annotation. Future research will focus on further reducing supervision requirements and refining the efficiency of the self-distillation process.
Summary updated