Multi-Class Part Parsing With Joint Boundary-Semantic Awareness

Yifan Zhao, Jia Li, Yu Zhang, Yonghong Tian

Abstract

Multi-class part parsing in "in the wild" scenarios presents significant challenges due to part-level boundary blurring and inter-class semantic ambiguity. This paper introduces BSANet, a framework that integrates lightweight Boundary Awareness and Semantic Awareness modules to address these issues. By leveraging spatial boundary attention and semantic channel selection, BSANet achieves state-of-the-art performance on the Pascal-Part dataset while maintaining high inference speeds, processing images roughly 30x faster than previous leading methods.

Introduction

Traditional semantic part parsing often assumes a single, well-localized object category. However, real-world images typically contain multiple objects of diverse classes. This "multi-class" setting introduces two primary hurdles:

  1. Part-level Ambiguity: Downsampling in deep networks often blurs the precise boundaries between small, adjacent parts (e.g., a bird's beak or a human's hand).
  2. Class-level Ambiguity: Visually similar parts across different categories (e.g., a bus front vs. a car front) lead to inter-class confusion.

The BSANet Framework

BSANet, built upon a DeepLabV3 backbone, introduces two plug-and-play modules to resolve these ambiguities:

Boundary Awareness Module (BAM)

The BAM focuses on spatial localization. It utilizes an auxiliary edge encoder to predict class-agnostic part boundaries. This prediction serves as a spatial attention mask, forcing the network to prioritize transitions between parts. These refined features are fused into a cascade pyramid to preserve fine-grained details.

Semantic Awareness Module (SAM)

The SAM addresses "what" is being parsed. A semantic branch predicts the overall object category at each pixel. This information acts as a prior, performing channel-wise attention to "switch on" feature channels relevant to the detected class and "switch off" irrelevant ones, thereby preventing misclassification between similar-looking parts of different objects.

Experimental Results

The model was evaluated on the Pascal-Part and Pascal-Person-Part datasets.

Performance and Efficiency

BSANet achieved 58.2% mIoU on the full Pascal-Part benchmark, a significant jump from the 54.4% baseline. On the Pascal-Person-Part subset, it reached a record 68.43% mIoU, outperforming specialized models that utilize additional data like human pose skeletons.

Crucially, BSANet is highly efficient. It processes a 512x512 image in under 200ms on an NVIDIA 1080Ti, making it approximately 30x faster than previous state-of-the-art models like HAZN.

Baseline part parsing result showing significant inter-class noise and blurred boundaries.

Conclusion

By treating boundaries as spatial filters and object categories as semantic priors, BSANet effectively tackles the complexities of multi-class part parsing. Its lightweight design proves that high-level structural modeling is not always necessary for state-of-the-art performance, offering a practical and efficient solution for real-world computer vision tasks.

Summary updated

Abstract

Object part parsing in the wild, which requires to simultaneously detect multiple object classes in the scene and accurately segments semantic parts within each class, is challenging for the joint presence of class-level and part-level ambiguities. Despite its importance, however, this problem is not sufficiently explored in existing works. In this paper, we propose a joint parsing framework with boundary and semantic awareness to address this challenging problem. To handle part-level ambiguity, a boundary awareness module is proposed to make mid-level features at multiple scales attend to part boundaries for accurate part localization, which are then fused with high-level features for effective part recognition. For class-level ambiguity, we further present a semantic awareness module that selects discriminative part features relevant to a category to prevent irrelevant features being merged together. The proposed modules are lightweight and implementation friendly, improving the performance substantially when plugged into various baseline architectures. Without bells and whistles, the full model sets new state-of-the-art results on the Pascal-Part dataset, in both multi-class and the conventional single-class setting, while running substantially faster than recent high-performance approaches.
BibTeX
@inproceedings{zhao2019multi,
  author = {Yifan Zhao and Jia Li and Yu Zhang and Yonghong Tian},
  title = {{Multi-Class Part Parsing With Joint Boundary-Semantic Awareness}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00927},
}
Figure