DANCE : A Deep Attentive Contour Model for Efficient Instance Segmentation

Zichen Liu, Jun Hao Liew, Xiangyu Chen, Jiashi FengWACV 2021 · OpenReview

Abstract

Contour-based instance segmentation methods are attractive due to their efficiency. However, existing contour-based methods either suffer from lossy representation, complex pipeline or difficulty in model training, resulting in sub-par mask accuracy on challenging datasets like MS-COCO. In this work, we propose a novel deep attentive contour model, named DANCE, to achieve better instance segmentation accuracy while remaining good efficiency. To this end, DANCE applies two new designs: attentive contour deformation to refine the quality of segmentation contours and segment-wise matching to ease the model training. Comprehensive experiments demonstrate DANCE excels at deforming the initial contour in a more natural and efficient way towards the real object boundaries. Effectiveness of DANCE is also validated on the COCO dataset, which achieves 38.1% mAP and outperforms all other contour-based instance segmentation models. To the best of our knowledge, DANCE is the first contour-based model that achieves comparable performance to pixel-wise segmentation models.

Introduction

Instance segmentation—the task of detecting objects and delineating their exact boundaries—is traditionally dominated by pixel-wise methods like Mask R-CNN. While accurate, these methods are computationally expensive because they classify every single pixel within a bounding box. Contour-based methods offer a faster alternative by representing shapes as a series of vertices (polygons) and deforming an initial shape to fit the object. However, existing contour models often struggle with training instabilities, such as correspondence interlacing where vertex paths cross, and inefficient regression where computation is wasted on vertices already at the boundary.

Methods

DANCE introduces three key innovations to make contour deformation more robust and efficient:

Segment-wise Matching

Instead of matching vertices uniformly, which often leads to "tangled" deformation paths, DANCE identifies intersection points between the initial and target contours. It splits the contour into segments and performs matching locally. This ensures a smoother, more natural deformation path, simplifying the optimization landscape for the neural network.

Comparison of uniform matching vs. segment-wise matching. Segment-wise matching prevents interlacing paths during contour deformation.

Attentive Contour Deformation

The model predicts an "Edge Attention Map" guided by boundary predictions. This map acts as a gate, directing the model to focus on moving vertices that are far from the boundary while leaving "on-boundary" vertices untouched. This effectively decouples the task of locating the edge from the task of moving toward it.

Scale-Invariant Normalization

To handle objects of varying sizes, DANCE normalizes vertex coordinates based on the object's bounding box dimensions, similar to RoIAlign. This ensures stable training regardless of whether the target is a small bird or a large truck.

Key Results

DANCE was evaluated on the MS COCO 2017 and SBD datasets. On COCO test-dev, DANCE achieved a state-of-the-art 38.1% mAP, significantly outperforming previous contour-based leaders like DeepSnake (30.3%) and PolarMask (30.4%).

Performance comparison on MS COCO. DANCE achieves superior Mask AP compared to other contour-based methods like PolarMask and DeepSnake across various inference speeds.

The model also maintains impressive efficiency. The real-time variant, DANCE-RT, runs at approximately 35ms per image (~28 FPS) on a Tesla V100 GPU, making it suitable for high-speed applications. Qualitatively, DANCE demonstrates a superior ability to handle complex, non-convex shapes, such as human limbs, which often pose challenges for simpler star-convex models.

Conclusion and Future Work

DANCE represents a significant step forward for contour-based instance segmentation, achieving performance comparable to pixel-wise models while remaining highly efficient. While it currently relies on an external object detector and a fixed vertex count, future work may involve integrating the attentive mechanism into end-to-end transformer architectures or refining vertex density for extremely complex topologies.

Summary updated

Abstract

Contour-based instance segmentation methods are attractive due to their efficiency. However, existing contour-based methods either suffer from lossy representation, complex pipeline or difficulty in model training, resulting in sub-par mask accuracy on challenging datasets like MS-COCO. In this work, we propose a novel deep attentive contour model, named DANCE, to achieve better instance segmentation accuracy while remaining good efficiency. To this end, DANCE applies two new designs: attentive contour deformation to refine the quality of segmentation contours and segment-wise matching to ease the model training. Comprehensive experiments demonstrate DANCE excels at deforming the initial contour in a more natural and efficient way towards the real object boundaries. Effectiveness of DANCE is also validated on the COCO dataset, which achieves 38.1% mAP and outperforms all other contour-based instance segmentation models. To the best of our knowledge, DANCE is the first contour-based model that achieves comparable performance to pixel-wise segmentation models. Code is available at https://github.com/lkevinzc/dance.
BibTeX
@inproceedings{liu2021dance,
  author = {Zichen Liu and Jun Hao Liew and Xiangyu Chen and Jiashi Feng},
  title = {{DANCE : A Deep Attentive Contour Model for Efficient Instance Segmentation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
  url = {https://openreview.net/forum?id=YNo6aFafgC},
}
Figure