Abstract
We present a weakly supervised instance segmentation algorithm based on deep community learning with multiple tasks. This task is formulated as a combination of weakly supervised object detection and semantic segmentation, where individual objects of the same class are identified and segmented separately. We address this problem by designing a unified deep neural network architecture, which has a positive feedback loop of object detection with bounding box regression, instance mask generation, instance segmentation, and feature extraction. Each component of the network makes active interactions with others to improve accuracy, and the end-to-end trainability of our model makes our results more robust and reproducible. The proposed algorithm achieves state-of-the-art performance in the weakly supervised setting without any additional training such as Fast R-CNN and Mask R-CNN on the standard benchmark dataset. The implementation of our algorithm is available on the project webpage: https://cv.snu.ac.kr/research/WSIS_CL.
Introduction
Instance segmentation, the task of identifying individual objects and their precise pixel-level boundaries, typically requires extensive datasets with manually annotated masks. Weakly Supervised Instance Segmentation (WSIS) aims to overcome this by using only image-level labels. However, WSIS models often suffer from "hot spot" detections, where the network focuses only on the most discriminative parts of an object (e.g., a cat's ears) rather than the entire body. Furthermore, combining detection and segmentation often leads to noise propagation between tasks.
Deep Community Learning Framework
The proposed "Deep Community Learning" framework addresses these issues by treating object detection, mask generation, and segmentation as a collaborative community.
The architecture consists of four primary components:
- Feature Extractor: A shared ResNet50 backbone provides features to all subsequent modules.
- Object Detection Module: This module identifies object locations and generates pseudo-labels. It utilizes a class-agnostic bounding box regressor, which helps the model learn general object boundaries and reduces bias toward specific discriminative features.
- Instance Mask Generation (IMG) Module: Using Class Activation Maps (CAMs), this module creates pseudo-ground-truth masks. It incorporates "feature smoothing" to penalize extreme activation peaks, forcing the model to attend to the entire object region.
- Instance Segmentation Module: This module predicts the final pixel-wise masks based on the refined outputs of the community.
The end-to-end trainability of this system allows for a positive feedback loop where each task informs and improves the others, leading to more robust feature extraction.
Experimental Results
The model was evaluated on the PASCAL VOC 2012 benchmark, using the augmented training set of 10,582 images with only image-level labels. The algorithm achieved a state-of-the-art [email protected] of 38.1 on the validation set.
Qualitative analysis demonstrates that the community learning approach effectively captures full object bodies, such as the entire wingspan of a bird or the torso of a sheep, outperforming baseline weakly supervised detectors that often produce fragmented masks. Notably, the model achieves these results without requiring complex multi-stage retraining processes typical of other WSIS methods.
Conclusion
Deep Multi-Task Community Learning provides a powerful framework for WSIS by leveraging the interactions between detection and segmentation. While the model shows significant improvements, challenges remain in handling heavily overlapping objects and capturing fine details like small limbs. Future research will focus on improving boundary refinement and handling dense scenes.
Summary updated