Introduction
Estimating a person's true height from a single photograph is a classic "ill-posed" problem in computer vision. Due to scale ambiguity, a short person standing close to the camera can appear the same size as a tall person standing further away. Traditionally, solving this required "extrinsic" information: knowing the camera’s focal length, its height from the ground, or having a reference object of known size in the frame.
Researchers from EPFL have developed a deep learning approach to estimate height from a single, uncalibrated RGB image by analyzing anatomical proportions. They hypothesize that human anatomy provides "intrinsic" scale markers—such as the ratio of the head to the shoulders or the length of the tibia relative to the torso—that a neural network can learn to decode.
Methodology: The DeepNet Architecture
The proposed system, DeepNet, moves away from simple linear models and manual keypoints toward a two-stream deep architecture:
- Preprocessing: The system automatically detects the subject and their 2D pose, generating two specific crops: a full-body crop and a high-resolution face crop.
- Two-Stream Processing:
- Coarse Stream: A ResNet-50 backbone processes the full-body crop to capture global proportions and articulated pose.
- Fine Stream: A VGG-based network processes the face crop to extract subtle biometric features.
- Fusion: Features from both streams are concatenated and passed through fully connected layers to regress the final height in centimeters.
The IMDB-100K Dataset
A major contribution of this work is the realization that previous failures in this field were largely due to small datasets. Standard 3D pose datasets have many images but very few unique subjects. To solve this, the authors crawled IMDB for the 100,000 most popular actors, using face recognition to propagate height labels from profile data to movie stills and ceremony photos.
This resulted in the IMDB-100K dataset, containing 101,664 images of 12,104 unique individuals. This diversity is critical for the network to learn the statistical variance of human proportions across different genders, ages, and ethnicities.
Key Results and Insights
The best-performing gender-specific DeepNet model achieved a Mean Absolute Error (MAE) of 5.56cm on the IMDB dataset, significantly outperforming the "Constant Mean" baseline of 8.25cm. Several key insights emerged from the study:
- The Face as a Ruler: Surprisingly, facial features are highly predictive of height. In evaluations, a model looking only at the face often performed better than one looking only at the body, likely due to consistent head-to-shoulder ratios.
- Gender Priors: Training gender-specific models significantly boosted accuracy, as men and women have distinct height distributions and skeletal proportions.
- Generalization: On an independent "Lab-test" of non-celebrities in diverse environments, the model maintained an MAE of approximately 8.5cm, proving it can generalize beyond the training distribution.
Limitations and Future Work
The authors note that "Hollywood heights" in the IMDB data may be slightly exaggerated, creating a noise ceiling for accuracy. Additionally, the current model does not account for height loss due to aging. Future work may involve using group photos to apply relative height constraints, further refining absolute estimates through assignment consistency.
Summary updated