On the Feasibility of 3D Model-Based Forensic Height and Weight Estimation

Neerja Thakkar, Hany FaridWorkshops @ CVPR 2021 · DOI

Introduction

Forensic height and weight estimation are critical components of criminal investigations, yet they remain notoriously unreliable. Traditional methods often rely on "ruler-on-the-wall" photogrammetry, which is highly susceptible to perspective distortion, low image resolution, and the fact that suspects are rarely standing perfectly upright. A striking example is the case of George Powell III, who was convicted of robbery based on an expert estimate that the suspect was at least 185 cm tall. Powell is 190 cm, while other experts later estimated the same suspect in the same video at just 171 cm.

Surveillance footage often provides poor conditions for traditional height estimation.

Researchers from UC Berkeley have explored whether modern 3D computer vision can move past these limitations. By using 3D human pose estimation, it may be possible to "reconstruct" a suspect from a single 2D image, stand them up straight in a virtual space, and derive more accurate measurements.

Methodology: The 3D Fitting Pipeline

The proposed forensic pipeline utilizes SMPLify-X, a state-of-the-art method that fits a 3D mesh—essentially a digital "skin"—to a person in a 2D image. The process involves several key steps:

  1. 3D Fitting: Using tools like OpenPose, the system identifies 2D keypoints (joints, face, hands) and optimizes a 3D model to match these points.
  2. The "Neutral Pose" Trick: Once a 3D model is fitted to the suspect's "action" pose (e.g., running or crouching), it is programmatically "reposed" into a neutral, standing position. This allows for a direct measurement of height from the top of the head to the floor.
  3. Weight Proxy: The volume of the resulting 3D mesh is used as a proxy for estimating the individual's weight.
The pipeline from 2D image to keypoint detection and final 3D mesh fitting.

The Challenge of Absolute Scale

A significant hurdle in this approach is "absolute scale"—knowing exactly how large a pixel represents in the real world. A 3D model generated from a 2D image has no inherent units. To address this, the researchers tested three scaling methods:

  • Coherent Point Drift (CPD): Aligning the model to a known ground truth to establish a theoretical upper bound for accuracy.
  • Average IPD: Using the average human inter-pupillary distance (the distance between pupils) as a standard "ruler."
  • Measured IPD: Using the specific pupil distance of a known suspect.

Experimental Setup and Datasets

To validate the approach, the study utilized two primary datasets:

  • Simulated Dataset: 1,584 high-resolution renders of 3D models in various shapes and poses, allowing researchers to isolate variables like camera tilt.
  • IMDB-23K: A subset of 869 celebrity images. Since celebrities have publicly available "ground truth" heights, these images provide a realistic testbed for messy, real-world conditions.

Key Insights and Results

The research yielded several critical insights into the feasibility of 3D forensic modeling:

Height vs. Weight Accuracy

When the scale is known (using CPD alignment), height estimation is remarkably accurate, with an error of only 0.5% (approx. 0.85 cm for a 170 cm person). However, weight estimation remains a significant challenge, with errors hovering around 18-20%. Current 3D pose estimators are biased toward "average" body shapes; because they rely on 2D skeletons, they struggle to distinguish between thin and heavy individuals if their bone structures appear similar.

The IPD Bottleneck

While 3D modeling effectively "undoes" the difficulty of non-upright poses, the reliance on average human eye distance (IPD) for scaling introduces a 5.5% error. This margin—the difference between a 5'7" person and a 6'0" person—is significant enough to potentially lead to wrongful convictions if used in isolation.

Resolution and Compression

Interestingly, image resolution has a minimal impact on height accuracy until it reaches extreme lows. However, JPEG compression is a more significant issue, as artifacts can confuse 2D keypoint detectors and degrade the quality of the 3D fit.

Conclusion and Future Directions

The study demonstrates that while 3D pose estimation can accurately recover height from unconstrained images, scale ambiguity remains the primary bottleneck. Without a reference object in the scene or a known measurement of the suspect, the 3D model lacks a definitive size.

Future advancements must move beyond "skeleton-only" models. Accurate weight estimation will likely require appearance-based models that analyze the actual pixels of the body to estimate volume. Additionally, "human-in-the-loop" systems, where forensic analysts manually refine 3D fits, could improve results, provided that safeguards against human bias are implemented.

Summary updated

BibTeX
@inproceedings{thakkar2021feasibility,
  author = {Neerja Thakkar and Hany Farid},
  title = {{On the Feasibility of 3D Model-Based Forensic Height and Weight Estimation}},
  booktitle = {Workshops @ CVPR},
  year = {2021},
  doi = {10.1109/cvprw53098.2021.00106},
}
Figure