Weakly Supervised Instance Segmentation Using Hybrid Network

Shisha Liao, Yongqing Sun, Chenqiang Gao, Pranav Shenoy K. P, Song Mu, Jun Shimamura, Atsushi Sagata

Abstract

Training high-performance instance segmentation models typically necessitates expensive pixel-level annotations. This paper introduces a hybrid network architecture designed to achieve high-quality segmentation using only "cheap" bounding box labels. The approach addresses the inherent noise in weakly supervised masks—particularly for small objects—by employing two specialized branches and an Enhanced Feature Pyramid Network (E-FPN). Experimental results on the PASCAL VOC 2012 dataset demonstrate significant improvements over existing weakly supervised methods.

Introduction: The Weak Supervision Challenge

Instance segmentation requires identifying and outlining every individual object in an image. While models like Mask R-CNN excel at this task, they are data-hungry, requiring massive datasets with pixel-perfect masks. Generating these masks is time-consuming and costly.

Weakly supervised learning offers a solution by using bounding boxes to automatically generate initial "fake" masks via classical algorithms like GrabCut. However, these tools often fail on small or blurry objects. Research indicates that approximately 76% of small object masks generated this way are "invalid," possessing low overlap with the actual object. Training standard models on such noisy data leads to compounded errors and poor performance on small instances.

Methodology: The Hybrid Architecture

To mitigate training data noise, the authors propose a Hybrid Instance Segmentation Network that differentiates between large and small objects.

The Hybrid Instance Segmentation Network architecture featuring specialized branches for large and small objects.

Specialized Branches

  1. Large Object Instance Branch: This branch, based on a modified Mask R-CNN, handles "valid" masks typically associated with larger objects. It performs detection and segmentation simultaneously.
  2. Small Object Instance Branch: Designed for "hard" samples, this branch performs detection first, followed by segmentation. To prevent the model from learning incorrect, jagged shapes from noisy masks, poor-quality generated masks are replaced with a simple ellipse—a geometric prior that serves as a cleaner training signal.

During inference, objects are routed to the appropriate branch based on a 64x64 pixel size threshold.

Enhanced-FPN (E-FPN)

Standard Feature Pyramid Networks (FPN) often lose high-resolution spatial information as low-level details travel a "long path" to the top layers. The proposed Enhanced-FPN shortens this distance, preserving critical localization information necessary for precise boundary definition, especially when ground-truth masks are unavailable.

Experimental Results

The hybrid approach was evaluated on the PASCAL VOC 2012 dataset, using 10,582 images for training (augmented with SBD) and 1,449 images for validation.

Comparison of the proposed hybrid method against other state-of-the-art weakly supervised approaches.

As shown in the table above, the proposed method significantly outperforms previous state-of-the-art weakly supervised methods. It achieved a [email protected] of 51.3%, a notable increase over the 44.8% reached by DeepLabBOX. Furthermore, the [email protected] reached 22.4%, nearly doubling the performance of several earlier box-level and image-level supervised methods. Qualitative analysis confirms that the network produces much cleaner boundaries for small objects compared to single-branch architectures.

Conclusion and Future Work

The Hybrid Instance Segmentation Network demonstrates that specialization based on object scale and the use of geometric priors can effectively overcome the limitations of noisy, weakly supervised training data. By prioritizing localization through an Enhanced-FPN, the model achieves state-of-the-art results with minimal annotation effort.

Future research could explore dynamic routing mechanisms instead of fixed pixel thresholds and investigate ways to reduce the dependence on initial classical segmentation tools like GrabCut. Additionally, optimizing the computational overhead of the multi-branch architecture remains an area for further development.

Summary updated

BibTeX
@inproceedings{liao2019weakly,
  author = {Shisha Liao and Yongqing Sun and Chenqiang Gao and Pranav Shenoy K. P and Song Mu and Jun Shimamura and Atsushi Sagata},
  title = {{Weakly Supervised Instance Segmentation Using Hybrid Network}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  doi = {10.1109/icassp.2019.8682309},
}
Figure