Abstract
We present a weakly supervised instance segmentation algorithm based on deep community learning with multiple tasks. This task is formulated as a combination of weakly supervised object detection and semantic segmentation, where individual objects of the same class are identified and segmented separately. We address this problem by designing a unified deep neural network architecture, which has a positive feedback loop of object detection with bounding box regression, instance mask generation, instance segmentation, and feature extraction. Each component of the network makes active interactions with others to improve accuracy, and the end-to-end trainability of our model makes our results more robust and reproducible. The proposed algorithm achieves state-of-the-art performance in the weakly supervised setting without any additional training such as Fast R-CNN and Mask R-CNN on the standard benchmark dataset. The implementation of our algorithm is available on the project webpage: https://cv.snu.ac.kr/research/WSIS_CL.
Introduction
Instance segmentation is a fundamental computer vision task that requires both identifying individual objects and providing precise pixel-level masks for each. While supervised models like Mask R-CNN achieve high accuracy, they rely on massive datasets with expensive, hand-drawn masks. Weakly Supervised Instance Segmentation (WSIS) aims to overcome this by using only image-level labels (e.g., "dog," "car").
However, WSIS faces two primary challenges. First, the "Discriminative Part" Problem causes models to focus only on the most distinctive features of an object rather than its entire extent. Second, Noisy Interactions between detection and segmentation tasks often lead to error propagation. This paper introduces Deep Community Learning, a framework that transforms these noisy interactions into a cooperative, self-refining system through a positive feedback loop.
Deep Community Learning Framework
The core innovation is a unified architecture where four distinct modules interact in a sequential and cyclical manner, forming a "community" that learns collectively.
Architecture Components
- Feature Extractor: A shared CNN backbone (ResNet50) provides the base features for all subsequent tasks. It is refined by losses flowing back from all community members.
- Object Detection Module: Based on the Online Instance Classifier Refinement (OICR) architecture, this module identifies object locations. It incorporates a class-agnostic bounding box regressor to improve localization. By being class-agnostic, the regressor avoids overfitting to specific class textures and better captures the full boundaries of objects.
- Instance Mask Generation (IMG) Module: This module generates initial masks from detected boxes using Class Activation Maps (CAMs). To improve mask quality, the authors implement a log-scale feature conversion and a background class map, preventing masks from becoming overly concentrated on small, discriminative parts.
- Instance Segmentation Module: A dedicated sub-network that takes the noisy outputs from the IMG module and learns to predict cleaner, binary masks for each object proposal.
The Feedback Loop
Unlike traditional multi-task learning where tasks run in parallel, Community Learning is sequential. The detection module provides pseudo-ground-truth labels for the mask generator, which in turn provides pseudo-masks for the segmentation network. The errors are corrected through joint training, where the shared feature extractor is updated based on the collective performance of the community.
Experimental Results
The proposed method was evaluated on the PASCAL VOC 2012 benchmark using the trainaug set for training and the val set for evaluation. Performance was measured using mean Average Precision (mAP) and Average Best Overlap (ABO).
- State-of-the-Art Performance: The model achieved an [email protected] of 38.1, significantly outperforming previous weakly supervised methods like PRM (26.8) and IAM (28.3).
- Ablation Study: Results confirmed the synergy of the community approach. A baseline detector achieved only 18.8 mAP, while the full community model reached 38.1 mAP, demonstrating that the interaction between modules is crucial for success.
- Localization Accuracy: The inclusion of the class-agnostic regressor was a key driver in improving localization, helping the model capture the entire object rather than just its most discriminative parts.
Conclusion
Deep Community Learning provides a robust framework for weakly supervised instance segmentation by fostering cooperation between detection and segmentation tasks. While the model currently relies on external object proposals and benefits from post-processing, it sets a new standard for WSIS performance. Future work will explore end-to-end proposal generation and application to more complex datasets like MS-COCO.
Summary updated