What Face and Body Shapes Can Tell About Height

Semih Günel, Helge Rhodin, Pascal FuaarXiv 2018 · arXiv

Introduction

Estimating a person's true height from a single photograph is a classic "ill-posed" problem in computer vision. Due to scale ambiguity, a short person standing close to the camera can appear the same size as a tall person standing further away. Traditionally, solving this required "extrinsic" information: knowing the camera’s focal length, its height from the ground, or having a reference object of known size in the frame.

Figure 1: Automated pose detection and height estimation in a complex, uncalibrated scene.

Researchers from EPFL have developed a deep learning approach to estimate height from a single, uncalibrated RGB image by analyzing anatomical proportions. They hypothesize that human anatomy provides "intrinsic" scale markers—such as the ratio of the head to the shoulders or the length of the tibia relative to the torso—that a neural network can learn to decode.

Methodology: The DeepNet Architecture

The proposed system, DeepNet, moves away from simple linear models and manual keypoints toward a two-stream deep architecture:

  1. Preprocessing: The system automatically detects the subject and their 2D pose, generating two specific crops: a full-body crop and a high-resolution face crop.
  2. Two-Stream Processing:
    • Coarse Stream: A ResNet-50 backbone processes the full-body crop to capture global proportions and articulated pose.
  3. Fine Stream: A VGG-based network processes the face crop to extract subtle biometric features.
  4. Fusion: Features from both streams are concatenated and passed through fully connected layers to regress the final height in centimeters.

The IMDB-100K Dataset

A major contribution of this work is the realization that previous failures in this field were largely due to small datasets. Standard 3D pose datasets have many images but very few unique subjects. To solve this, the authors crawled IMDB for the 100,000 most popular actors, using face recognition to propagate height labels from profile data to movie stills and ceremony photos.

Figure 4: Examples from the IMDB-100K dataset showing the high variability in pose, lighting, and attire for a single individual.

This resulted in the IMDB-100K dataset, containing 101,664 images of 12,104 unique individuals. This diversity is critical for the network to learn the statistical variance of human proportions across different genders, ages, and ethnicities.

Key Results and Insights

The best-performing gender-specific DeepNet model achieved a Mean Absolute Error (MAE) of 5.56cm on the IMDB dataset, significantly outperforming the "Constant Mean" baseline of 8.25cm. Several key insights emerged from the study:

  • The Face as a Ruler: Surprisingly, facial features are highly predictive of height. In evaluations, a model looking only at the face often performed better than one looking only at the body, likely due to consistent head-to-shoulder ratios.
  • Gender Priors: Training gender-specific models significantly boosted accuracy, as men and women have distinct height distributions and skeletal proportions.
  • Generalization: On an independent "Lab-test" of non-celebrities in diverse environments, the model maintained an MAE of approximately 8.5cm, proving it can generalize beyond the training distribution.

Limitations and Future Work

The authors note that "Hollywood heights" in the IMDB data may be slightly exaggerated, creating a noise ceiling for accuracy. Additionally, the current model does not account for height loss due to aging. Future work may involve using group photos to apply relative height constraints, further refining absolute estimates through assignment consistency.

Summary updated

Abstract

Recovering a person's height from a single image is important for virtual garment fitting, autonomous driving and surveillance, however, it is also very challenging due to the absence of absolute scale information. We tackle the rarely addressed case, where camera parameters and scene geometry is unknown. To nevertheless resolve the inherent scale ambiguity, we infer height from statistics that are intrinsic to human anatomy and can be estimated from images directly, such as articulated pose, bone length proportions, and facial features. Our contribution is twofold. First, we experiment with different machine learning models to capture the relation between image content and human height. Second, we show that performance is predominantly limited by dataset size and create a new dataset that is three magnitudes larger, by mining explicit height labels and propagating them to additional images through face recognition and assignment consistency. Our evaluation shows that monocular height estimation is possible with a MAE of 5.56cm.
BibTeX
@misc{gunel2018what,
  author = {Semih Günel and Helge Rhodin and Pascal Fua},
  title = {{What Face and Body Shapes Can Tell About Height}},
  year = {2018},
  eprint = {1805.10355},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1805.10355},
}
Figure