Semi-Supervised Skin Detection by Network With Mutual Guidance

Yi He, Jiayuan Shi, Chuan Wang, Haibin Huang, Jiaming Liu, Guanbin Li, Risheng Liu, Jue Wang

Abstract

Skin detection is a critical task in computer vision but suffers from visual ambiguity and a lack of large-scale annotated datasets. This paper introduces a semi-supervised dual-task network that performs skin and human body detection simultaneously. Through a "mutual guidance" mechanism, the model uses body detection as a semantic guide to distinguish skin from similarly colored background objects. Experimental results demonstrate that this approach achieves state-of-the-art accuracy, significantly outperforming traditional and standard deep learning methods.

Introduction

Skin detection—identifying skin-colored pixels in images—is foundational for applications like face recognition and gesture tracking. However, it faces two primary hurdles: visual ambiguity due to varied lighting and background objects (e.g., wood, leather) sharing skin-like colors, and data scarcity due to the high cost of pixel-level ground truth labeling. Most existing models lack semantic awareness, failing to recognize that skin should typically be part of a human body. This research addresses these gaps by incorporating body detection as a weak semantic guide to help the network distinguish between a person's features and background clutter.

Methodology

The proposed architecture is a Dual-Task Neural Network featuring a shared encoder and two specialized decoders for skin and body detection. The core innovation is the Mutual Guidance strategy, which operates in two stages.

The proposed dual-task network architecture and its decoupled stages for analysis.

In the first stage, the network generates initial probability maps for both skin and body. In the second stage, the skin decoder's output serves as a prior for the body decoder, and vice versa. This creates a refinement loop where the body mask filters background false positives, and the skin mask provides high-frequency details to refine body boundaries. To train on partially labeled data, the authors employ a semi-supervised loss function combining Cross-Entropy (CE), CRF Loss for spatial consistency, and Weighted Cross-Entropy (WCE) to enforce the constraint that skin pixels must reside within the body mask.

Datasets

A new dataset of 10,711 human portraits was curated to facilitate this dual-task learning: 5,000 images are annotated with skin masks, and 5,711 are annotated with body masks. Crucially, most images contain only one label type, necessitating the semi-supervised approach. The Pratheepan Face Dataset was also used for external validation and comparison against existing methods.

Results

The mutual guidance model consistently outperformed Gaussian Mixture Models (GMM), UNet, and DeepLab-v3. Quantitatively, the model achieved an Intersection-over-Union (IoU) of 81.18% on the validation set, compared to 75.59% for a standard UNet.

Qualitative comparison of skin detection results across different models and challenging scenarios.

Qualitative analysis shows the model's robustness in ignoring "skin-like" background elements, such as wooden furniture or brown sofas, which typically cause failures in other state-of-the-art models. The "mutual benefit" is evident: body masks act as a prior to eliminate background noise, while skin masks help the body detector define edges more accurately. The model showed a much higher "IoU Top-1 rate," winning the comparison against other methods on approximately 51% of test images.

Conclusion

While highly effective for portraits, the model's performance in crowded or long-shot scenes remains a subject for future study. The dependency on the body detector suggests that extreme occlusion could impact results. However, the "mutual guidance" philosophy offers a promising framework for other joint segmentation tasks, such as hair and face segmentation or video-based object tracking.

Summary updated

Abstract

In this paper we present a new data-driven method for robust skin detection from a single human portrait image. Unlike previous methods, we incorporate human body as a weak semantic guidance into this task, considering acquiring large-scale of human labeled skin data is commonly expensive and time-consuming. To be specific, we propose a dual-task neural network for joint detection of skin and body via a semi-supervised learning strategy. The dual-task network contains a shared encoder but two decoders for skin and body separately. For each decoder, its output also serves as a guidance for its counterpart, making both decoders mutually guided. Extensive experiments were conducted to demonstrate the effectiveness of our network with mutual guidance, and experimental results show our network outperforms the state-of-the-art in skin detection.
BibTeX
@misc{he2019semi,
  author = {Yi He and Jiayuan Shi and Chuan Wang and Haibin Huang and Jiaming Liu and Guanbin Li and Risheng Liu and Jue Wang},
  title = {{Semi-Supervised Skin Detection by Network With Mutual Guidance}},
  year = {2019},
  eprint = {1908.01977},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1908.01977},
}
Figure