A Part-Based Deep Neural Network Cascade Model for Human Parsing

Yanghong Zhou, P.y. Mok, Shijie ZhouIEEE Access 2019 · DOI

Abstract

Human parsing is essential for image-based human-centric and clothing analyses. While deep neural networks have improved parsing accuracy, localized small regions like sunglasses often remain poorly segmented. We propose the Part-based Human Parsing Cascade (PHPC), a model that mimics the human visual mechanism of scanning an entire image before focusing on specific parts. By integrating specialized head- and body-parsing sub-networks into a cascade architecture, our method achieves significant improvements in fine-grained segmentation, as demonstrated on the ATR dataset.

What Problem Does This Paper Address?

Human parsing involves labeling every pixel in an image as a specific body part or clothing item. It is a fundamental task for fashion AI and human-centric computer vision. While modern Deep Neural Networks (DNNs) excel at identifying large regions such as dresses or trousers, they frequently struggle with "localized small regions."

Example of human parsing labels and a segmented image.

Items like sunglasses, belts, or thin scarves often disappear or are misclassified because they occupy very few pixels in a standard global scan. Standard models often lack the resolution or specific attention required to distinguish these small features from the background or larger adjacent garments.

Methods: The PHPC Architecture

The proposed Part-based Human Parsing Cascade (PHPC) operates in three distinct stages, imitating the human "scan-then-zoom" observational mechanism:

  1. Image-Level Parsing: A baseline Fully Convolutional Network (FCN-8s) performs a global scan of the image to generate an initial parsing map, which is used to locate bounding boxes for the head and body.
  2. Part-Based Sub-networks: The model crops the head and body regions based on the initial scan. These regions are scaled up to double their original size and processed by specialized sub-networks: a head-parsing network and a body-parsing network.
  3. Feature Fusion: High-resolution feature maps from these sub-networks are merged back into the global feature map. This allows the final output to benefit from both global context and localized, high-resolution details.
The PHPC architecture, featuring image-level parsing followed by specialized head and body sub-networks.

The authors also evaluated traditional refinement techniques like Conditional Random Fields (CRFs) and Superpixel grouping to determine their impact on boundary accuracy.

Datasets and Evaluation

The model was trained and evaluated on the ATR dataset, a standard benchmark for human parsing containing 7,702 images (6,898 for training, 804 for testing). The dataset includes 18 semantic categories, such as background, hat, hair, sunglasses, upper-clothing, skirt, dress, belt, trousers, shoes, face, limbs, bag, and scarf. During preprocessing, images were resized to 384x384, with standard data augmentation (mirroring and cropping) applied during training.

Key Insights

  • The "Zoom" Effect: By doubling the resolution of the head and body regions, the sub-networks can extract much more descriptive features for small items that a standard FCN would otherwise ignore.
  • Mimicking Human Observation: The cascade structure reflects how humans naturally process visual information—identifying the person first, then focusing on the face and outfit details.
  • The CRF Paradox: Interestingly, adding CRF or Superpixel refinement actually decreased accuracy for the PHPC model. While these methods typically smooth out noise, in this architecture, they tended to "smooth over" the fine details (like sunglasses) that the sub-networks were specifically designed to capture.

Key Results

The PHPC model outperformed both the baseline FCN-8s and the CRFasRNN model across several metrics:

  • Pixel Accuracy: Reached 93.32%, compared to 92.37% for FCN-8s.
  • Mean IoU (Intersection over Union): Achieved 59.93%, a 4.58% improvement over the baseline.
  • Small Item Success: The most dramatic gains occurred in difficult categories. For example, the IoU for sunglasses increased by over 11% compared to FCN-8s. Significant improvements were also noted for "belt" and "scarf" categories.

Limitations and Future Work

  • Computational Cost: Running three separate networks in a cascade makes the model slower than end-to-end alternatives. Processing an image takes roughly 4.98 seconds, compared to 0.29 seconds for a standard FCN.
  • Cascade Dependency: The sub-networks depend on the accuracy of the initial global scan. If the first network fails to locate the head or body, the subsequent stages will process the wrong regions.
  • Future Directions: Future work may involve developing a unified, end-to-end network that shares parameters between global and part-based stages to improve speed while maintaining high detail resolution.

Summary updated

Abstract

Human parsing is important for image-based human-centric and clothing analyses. With the development of deep neural networks, some deep human parsing methods were recently proposed, which substantially improve the parsing accuracy. However, some localized small regions (such as sunglasses) are not parsed well in these methods. In this paper, we propose a Part-based Human Parsing Cascade (PHPC) to segment human images, imitating the observational mechanism of how people, when first looking at a human image, quickly scan the entire photograph to first locate the face and then the body parts to see what clothing the person is wearing. The observational mechanism of human vision is used to establish a cascade relationship in designing our network, in which a head-parsing sub-network and a body-parsing sub-network are integrated to the cascade of human parsing networks. The head- and body-parsing sub-networks focus on the head and body classes, respectively, and add attention to the head and body in the final neural networks. Comprehensive evaluations on the ATR dataset have demonstrated the effectiveness of our method.
BibTeX
@inproceedings{zhou2019part,
  author = {Yanghong Zhou and P.Y. Mok and Shijie Zhou},
  title = {{A Part-Based Deep Neural Network Cascade Model for Human Parsing}},
  booktitle = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2951182},
}
Figure