Real-time Instance Segmentation with Discriminative Orientation Maps

Wentao Du, Zhiyu Xiang, Shuya Chen, Chengyu Qiao, Yiman Chen, Tingming BaiICCV 2021 · DOI · arXiv

Abstract

Real-time instance segmentation requires a delicate balance between detection accuracy and inference speed. Traditional methods often rely on computationally expensive Region of Interest (RoI) operations or struggle with overlapping objects. This paper presents OrienMask, a one-stage framework that utilizes discriminative orientation maps to represent object masks. By predicting 2D offset vectors for every pixel—pointing toward object centers for foreground and away for background—OrienMask achieves high-quality segmentation without RoI cropping. Evaluated on the MS COCO benchmark, OrienMask delivers 34.8 mask AP at 42.7 FPS, outperforming existing real-time models like YOLACT in both precision and efficiency.

Introduction

Instance segmentation is a fundamental computer vision task that combines object detection with pixel-level classification. Existing approaches generally fall into two categories: detection-based methods, such as Mask R-CNN, which are accurate but slow due to sequential RoI processing; and segmentation-based methods, which are faster but often fail to distinguish between adjacent or overlapping instances of the same class.

OrienMask addresses these limitations by introducing a truly one-stage, anchor-based approach. It eliminates the need for RoI pooling or alignment, which often introduces quantization errors and blurs boundaries. Instead, it leverages the inherent spatial information in orientation maps to reconstruct masks directly from high-resolution feature maps.

Methodology

Architecture Overview

OrienMask is built upon the YOLOv3 architecture, enhanced with a Feature Pyramid Network (FPN) to handle objects across multiple scales. The primary innovation is the addition of the OrienHead, a lightweight branch that operates alongside the standard bounding box and class prediction heads.

The OrienMask architecture integrates a specialized OrienHead into a YOLOv3-based backbone for efficient mask prediction.

The OrienHead predicts two-channel orientation maps at the highest resolution of the FPN (P2). These maps contain the spatial offset vectors used to define the geometry of each instance.

Discriminative Orientation Maps

The core concept of OrienMask is the use of directional vectors to encode instance information. For every pixel within a predicted bounding box, the model predicts a 2D vector:

  1. Centripetal Vectors: For foreground pixels, vectors point toward the center of the object instance.
  2. Centrifugal Vectors: For background pixels, vectors point away from the center toward the boundaries of a predefined "valid training area."
Discriminative orientation maps use directional vectors to distinguish foreground instances from the background.

This dual-directionality allows the model to inherently filter background noise and separate overlapping instances. During inference, a mask is reconstructed by identifying pixels whose vectors point into a small target region around the predicted object center.

Instance Grouping and Scale Invariance

To handle overlapping objects of different sizes, OrienMask utilizes YOLO's anchor-based logic. Objects are assigned to different orientation maps based on their matched anchor size. This multi-layer approach prevents masks of different scales from interfering with one another, a common issue in global mask prediction frameworks.

Experiments and Results

The authors evaluated OrienMask on the MS COCO dataset, using the train2017 set for training and test-dev for benchmarking.

Performance and Efficiency

OrienMask demonstrates a superior trade-off between speed and accuracy. On an NVIDIA RTX 2080 Ti, it achieves:

  • 34.8 Mask AP
  • 42.7 FPS

In comparison, YOLACT achieves 29.8 AP, and CenterMask reaches 33.1 AP. Beyond raw accuracy, OrienMask is significantly more memory-efficient. Its mask construction requires only 1.27M parameters, compared to 18.44M for BlendMask, making it an ideal candidate for deployment on edge devices.

Qualitative Analysis

Qualitative results show that OrienMask excels at delineating complex boundaries and handling adjacent instances, such as people in a crowd or overlapping objects on a table. By avoiding RoI operations, the model preserves high-resolution details that are often lost in other frameworks.

Conclusion

OrienMask introduces a novel and efficient representation for instance masks through discriminative orientation maps. By leveraging the spatial relationship between pixels and object centers, it provides a robust, real-time solution for instance segmentation. While the model may face challenges in cases of extreme overlap or high-density boundary confrontation, its overall performance and low memory footprint mark a significant advancement in one-stage instance segmentation. Future research will focus on refining vector field predictions to further improve robustness in crowded environments.

Summary updated

BibTeX
@inproceedings{du2021real,
  author = {Wentao Du and Zhiyu Xiang and Shuya Chen and Chengyu Qiao and Yiman Chen and Tingming Bai},
  title = {{Real-time Instance Segmentation with Discriminative Orientation Maps}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00722},
}
Figure