Abstract
Human parsing is essential for image-based human-centric and clothing analyses. While deep neural networks have improved parsing accuracy, localized small regions like sunglasses often remain poorly segmented. We propose the Part-based Human Parsing Cascade (PHPC), a model that mimics the human visual mechanism of scanning an entire image before focusing on specific parts. By integrating specialized head- and body-parsing sub-networks into a cascade architecture, our method achieves significant improvements in fine-grained segmentation, as demonstrated on the ATR dataset.
What Problem Does This Paper Address?
Human parsing involves labeling every pixel in an image as a specific body part or clothing item. It is a fundamental task for fashion AI and human-centric computer vision. While modern Deep Neural Networks (DNNs) excel at identifying large regions such as dresses or trousers, they frequently struggle with "localized small regions."
Items like sunglasses, belts, or thin scarves often disappear or are misclassified because they occupy very few pixels in a standard global scan. Standard models often lack the resolution or specific attention required to distinguish these small features from the background or larger adjacent garments.
Methods: The PHPC Architecture
The proposed Part-based Human Parsing Cascade (PHPC) operates in three distinct stages, imitating the human "scan-then-zoom" observational mechanism:
- Image-Level Parsing: A baseline Fully Convolutional Network (FCN-8s) performs a global scan of the image to generate an initial parsing map, which is used to locate bounding boxes for the head and body.
- Part-Based Sub-networks: The model crops the head and body regions based on the initial scan. These regions are scaled up to double their original size and processed by specialized sub-networks: a head-parsing network and a body-parsing network.
- Feature Fusion: High-resolution feature maps from these sub-networks are merged back into the global feature map. This allows the final output to benefit from both global context and localized, high-resolution details.
The authors also evaluated traditional refinement techniques like Conditional Random Fields (CRFs) and Superpixel grouping to determine their impact on boundary accuracy.
Datasets and Evaluation
The model was trained and evaluated on the ATR dataset, a standard benchmark for human parsing containing 7,702 images (6,898 for training, 804 for testing). The dataset includes 18 semantic categories, such as background, hat, hair, sunglasses, upper-clothing, skirt, dress, belt, trousers, shoes, face, limbs, bag, and scarf. During preprocessing, images were resized to 384x384, with standard data augmentation (mirroring and cropping) applied during training.
Key Insights
- The "Zoom" Effect: By doubling the resolution of the head and body regions, the sub-networks can extract much more descriptive features for small items that a standard FCN would otherwise ignore.
- Mimicking Human Observation: The cascade structure reflects how humans naturally process visual information—identifying the person first, then focusing on the face and outfit details.
- The CRF Paradox: Interestingly, adding CRF or Superpixel refinement actually decreased accuracy for the PHPC model. While these methods typically smooth out noise, in this architecture, they tended to "smooth over" the fine details (like sunglasses) that the sub-networks were specifically designed to capture.
Key Results
The PHPC model outperformed both the baseline FCN-8s and the CRFasRNN model across several metrics:
- Pixel Accuracy: Reached 93.32%, compared to 92.37% for FCN-8s.
- Mean IoU (Intersection over Union): Achieved 59.93%, a 4.58% improvement over the baseline.
- Small Item Success: The most dramatic gains occurred in difficult categories. For example, the IoU for sunglasses increased by over 11% compared to FCN-8s. Significant improvements were also noted for "belt" and "scarf" categories.
Limitations and Future Work
- Computational Cost: Running three separate networks in a cascade makes the model slower than end-to-end alternatives. Processing an image takes roughly 4.98 seconds, compared to 0.29 seconds for a standard FCN.
- Cascade Dependency: The sub-networks depend on the accuracy of the initial global scan. If the first network fails to locate the head or body, the subsequent stages will process the wrong regions.
- Future Directions: Future work may involve developing a unified, end-to-end network that shares parameters between global and part-based stages to improve speed while maintaining high detail resolution.
Summary updated