3D Joints Estimation of the Human Body in Single-Frame Point Cloud

Tianxu Xu, Dong An, Zhonghan Wang, Sicheng Jiang, Chengnuo Meng, Yiwen Zhang, Qiang Wang, Zhongqi Pan, Yang YueIEEE Access 2020 · DOI

Abstract

Joint estimation of the human body in point clouds is a key step for tracking human movements. In this work, we present a geometric method to achieve detection of the joints from a single-frame point cloud captured using a Time-of-Flight (ToF) camera. A three-dimensional (3D) human silhouette, as a global feature of the single-frame point cloud, is extracted based on pre-processed data. The angle and aspect ratio of the silhouette are subsequently utilized to perform pose recognition, and then 14 joints of the human body are derived via geometric features of the 3D silhouette. To verify this method, we tested on an in-house captured 3D dataset containing 1,200-frame depth images, categorized into four different poses (upright, raising hands, parallel arms, and akimbo). Furthermore, we tested on a subset of the G3D dataset. By hand-labeling the joints of each human body as the ground truth for validation and benchmarks, the average normalized error of our geometric method is less than 5.8 cm. When the distance threshold from the ground truth is 10 cm, the results demonstrate that our proposed method delivers improved performance with an average accuracy in the range of 90%.

Introduction

Human pose estimation is a cornerstone of computer vision, essential for applications ranging from autonomous driving to elderly care. While traditional 2D video analysis has advanced significantly, it often struggles with lighting changes, complex backgrounds, and occlusions. Low-cost Time-of-Flight (ToF) depth sensors offer a robust 3D alternative that is more resilient to environmental changes. However, many current 3D methods rely on deep learning, which requires enormous amounts of labeled data and high computational power. This paper proposes a "middle way": a lightweight, purely geometric approach that can accurately locate joints in a single frame of a point cloud without prior training or massive datasets.

Methodology

The researchers developed a three-stage pipeline consisting of Data Acquisition, Pre-processing, and Joint Estimation.

The proposed three-stage pipeline: (a) Data Acquisition, (b) Data Pre-processing including orientation alignment, and (c) Joint Estimation.
  1. Pre-processing: The raw point cloud is filtered to remove background noise. A critical step is the automatic adjustment of the human subject's orientation. By projecting the cloud to a "top view" and calculating the rotation angle of the farthest points, the algorithm ensures the subject is correctly aligned with the coordinate system before analysis.
  2. Pose Recognition: Instead of a generic approach, the algorithm first classifies the subject into one of four poses: upright, raising hands, parallel arms, or akimbo (hands on hips). This is achieved using geometric "figures of merit," such as the angle between the head and the farthest points on the x-axis and the aspect ratio of the silhouette.
  3. Joint Estimation: Once the pose is identified, 14 joints (head, neck, waist, and L/R shoulders, elbows, hands, knees, and feet) are derived. The logic adapts based on the pose; for example, in the "parallel arms" pose, hands are identified as the farthest points on the x-axis, whereas in the "akimbo" pose, the hands are assumed to overlap with the waist.

Experimental Results

The method was evaluated using an in-house dataset of 1,200 frames featuring 10 participants and a subset of the public G3D gaming action dataset. All joints in the test sets were hand-labeled to provide a gold standard for error calculation.

Examples of the geometric joint estimation results for three different poses: upright, akimbo, and parallel arms.

Key performance metrics include:

  • Accuracy: The method achieved an average normalized error of less than 5.8 cm.
  • Threshold Performance: With a 10 cm distance threshold from the ground truth, the average accuracy reached approximately 90%.
  • Efficiency: The joint extraction process is fast, taking about 103 ms per frame, while pose classification requires only 2.66 ms.

Discussion and Conclusion

This research demonstrates that a 3D silhouette contains sufficient geometric information to locate joints accurately if orientation is standardized. By using well-defined geometric rules—such as using the ratio of z-coordinate range to y-coordinate range to determine knee positions—the system remains effective and transparent compared to "black box" deep learning models.

The primary limitation is pose variety; the current algorithm is specifically tuned to four discrete poses. Additionally, the computational bottleneck lies in the pre-processing stage, which currently takes nearly 9.5 seconds on a standard CPU. Future work will focus on migrating these processes to a GPU to enable real-time application and expanding the library of recognizable poses.

Summary updated

Abstract

Joint estimation of human body in point cloud is a key step for tracking human movements. In this work, we present a geometric method to achieve detection of the joints from a single-frame point cloud captured using a Time-of-Flight (ToF) camera. Three-dimensional (3D) human silhouette, as global feature of the single-frame point cloud, is extracted based on the pre-processed data, the angle and aspect ratio of the silhouette are subsequently utilized to perform pose recognition, and then 14 joints of human body are derived via geometric features of 3D silhouette. To verify this method, we test on an in-house captured 3D dataset containing 1200-frame depth images, which can be categorized into four different poses (upright, raising hands, parallel arms, and akimbo). Furthermore, we test on a subset of the G3D dataset. By hand-labelling the joints of each human body as the ground truth for validation and benchmarks, the average normalized error of our geometric method is less than 5.8 cm. When the distance threshold from the ground truth is 10 cm, the results demonstrate that our proposed method delivers improved performance with an average accuracy in the range of 90%.
BibTeX
@inproceedings{xu2020d,
  author = {Tianxu Xu and Dong An and Zhonghan Wang and Sicheng Jiang and Chengnuo Meng and Yiwen Zhang and Qiang Wang and Zhongqi Pan and Yang Yue},
  title = {{3D Joints Estimation of the Human Body in Single-Frame Point Cloud}},
  booktitle = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.3027892},
}
Figure