Full body image feature representations for gender profiling

Matthew Collins, Jianguo Zhang, Paul Miller, Hongbin WangWorkshops @ ICCV 2009 · OpenReview

Abstract

In this paper we focus on building robust image representations for gender classification from full human bodies. We first investigate a number of state-of-the-art image representations with regard to their suitability for gender profiling from static body images. Features include Histogram of Gradients (HOG), spatial pyramid HOG and spatial pyramid bag of words etc. These representations are learnt and combined based on a kernel support vector machine (SVM) classifier. We compare a number of different SVM kernels for this task but conclude that the simple linear kernel appears to give the best overall performance. Our study shows that individual adoption of these representations for gender classification is not as promising as might be expected, given their good performance in the tasks of pedestrian detection on INRIA datasets, and object categorisation on Caltech 101 and Caltech 256 datasets. Our best results, 80% classification accuracy, were achieved from a combination of spatial shape information, captured by HOG, and colour information captured by HSV histogram based features. Additionally, to the best of our knowledge, currently there is no publicly available dataset for full body gender recognition. Hence, we further introduce a novel body gender dataset covering a large diversity of human body appearance.

Introduction

While facial recognition has reached high levels of maturity, biometric systems in real-world surveillance scenarios often fail to capture clear, high-resolution facial data. In "in-the-wild" computer vision, subjects are frequently seen from the side or back, or are partially occluded. This research addresses the gap in gender profiling by shifting the focus from facial features to full-body "pedestrian" images typical of CCTV footage. By identifying robust proxies for gender—such as body silhouette, hair length, and clothing—this study demonstrates that gender can be decoded even when the face is a blur.

Methodology

Gender profiling is treated as a supervised learning problem. The researchers evaluated several state-of-the-art feature extraction techniques to determine which holds the most discriminative power:

  • Shape Descriptors: Pyramid Histogram of Gradients (PHOG) and its variants (Canny-based CHOG and Pixel-based PiHOG) were used to capture the silhouette and internal contours of a person at multiple spatial scales.
  • Appearance and Texture: Pyramid Histogram of Visual Words (PHOW) utilized SIFT descriptors to identify local patterns regardless of their spatial location.
  • Color Features: Local HSV Colour Histograms (LHSV) were introduced to capture the distribution of hue and saturation across a grid overlaid on the subject.
  • Classification: These features were processed using a Support Vector Machine (SVM). Despite testing complex non-linear kernels (RBF and Chi-squared), a simple Linear SVM proved most effective due to the high dimensionality of the feature vectors.

Datasets and Data Diversity

As no dedicated full-body gender dataset existed, the authors adapted and manually labeled two pedestrian benchmarks. The MIT Pedestrian Dataset was balanced to include 305 males and 123 females (later adjusted for bias), while the VIPeR Dataset provided a more challenging set of 1,249 images with diverse viewpoints and lighting conditions.

Sample images from the datasets showing the diversity of male (left) and female (right) pedestrian appearances.

Key Insights and Results

The study revealed that the fusion of different feature types is critical. Shape features (HOG) can confuse baggy clothing for body mass, and color features (HSV) are often fooled by gender-neutral attire like blue jeans. However, combining PiHOG and LHSV yielded the best results, achieving 80.62% accuracy on the VIPeR dataset.

Comparison of original and tightly cropped pedestrian images used to evaluate the impact of background context on feature extraction.

An interesting finding was the role of context. "Uncropped" images, which include a small amount of background, actually improved the performance of some shape descriptors by providing better context for the person's boundaries. Furthermore, dense gradient maps (PiHOG) outperformed edge-only maps (CannyHOG), suggesting that subtle gradients in clothing texture are as vital as the hard silhouette outline.

Conclusion

This research highlights that for high-dimensional image features, simple linear boundaries are often more robust and less prone to overfitting than complex kernels. While 80% accuracy is lower than contemporary facial recognition rates, it represents a significant advancement for body-based profiling in low-resolution scenarios. Future work may involve more sophisticated feature weighting through Multiple Kernel Learning (MKL) and improving pose invariance to handle side-profile views more effectively.

Summary updated

Abstract

In this paper we focus on building robust image representations for gender classification from full human bodies. We first investigate a number of state-of-the-art image representations with regard to their suitability for gender profiling from static body images. Features include Histogram of Gradients (HOG), spatial pyramid HOG and spatial pyramid bag of words etc. These representations are learnt and combined based on a kernel support vector machine (SVM) classifier. We compare a number of different SVM kernels for this task but conclude that the simple linear kernel appears to give the best overall performance. Our study shows that individual adoption of these representations for gender classification is not as promising as might be expected, given their good performance in the tasks of pedestrian detection on INRIA datasets, and object categorisation on Caltech 101 and Caltech 256 datasets. Our best results, 80% classification accuracy, were achieved from a combination of spatial shape information, captured by HOG, and colour information captured by HSV histogram based features. Additionally, to the best of our knowledge, currently there is no publicly available dataset for full body gender recognition. Hence, we further introduce a novel body gender dataset covering a large diversity of human body appearance.
BibTeX
@inproceedings{collins2009full,
  author = {Matthew Collins and Jianguo Zhang and Paul Miller and Hongbin Wang},
  title = {{Full body image feature representations for gender profiling}},
  booktitle = {Workshops @ ICCV},
  year = {2009},
  url = {https://openreview.net/forum?id=1wd7zP8Zmw},
}
Figure