Anatomy and pose estimation from point sets using FAKIR

Tong Fu, Raphaëlle Chaine, Julie DigneHAL 2019

TL;DR

This paper introduces FAKIR, a novel algorithm for estimating the anatomy and pose of human statues from 3D point cloud data. By employing a pliable sphere-mesh anatomical model and a unique forward-backward iterative registration process, FAKIR efficiently reconstructs the underlying skeleton and its pose, even for damaged or artistically stylized statues. This work significantly advances the digital preservation and virtual restoration of cultural heritage artifacts.

What Problem Does This Paper Address?

The digital preservation and virtual restoration of cultural heritage, particularly statues, is a growing area of interest. A crucial step in this process is understanding the underlying anatomical structure and pose of the artifact. Manually identifying anatomical parts and guiding restoration hypotheses is a time-consuming and tedious task. Existing methods often struggle with the inherent inaccuracies and artistic stylizations found in statues, which may not adhere to perfect human proportions. Furthermore, working directly with raw 3D point cloud data, without requiring a prior meshing step, is desirable for preserving the accuracy of the initial scan. This paper addresses the need for an efficient and robust method to automatically estimate the anatomy and pose of human statues from point cloud data, enabling further virtual manipulation and analysis.

Methods

The core of this work lies in the FAKIR (Forward And backward Iterative Registration) algorithm. FAKIR operates on a custom-designed, pliable sphere-mesh anatomical model. This model represents each bone as a segment with spheres at its ends, allowing for efficient distance computations and flexibility in representing non-standard proportions. The model consists of 22 bones organized into 5 chains (spine, arms, legs), with some bones having fixed relative motion.

FAKIR's registration process is inspired by inverse kinematics techniques. It iteratively refines the anatomical model's pose and intrinsic parameters (bone lengths and radii) to best match the input point cloud. The key innovation is the forward and backward iterative loop. This process first moves forward through a bone chain, optimizing parameters, and then reverses direction to further refine the fit. This bidirectional approach, along with a pairwise optimization strategy (considering two consecutive bones at a time), contributes to a more robust estimation of joint positions and bone parameters. The algorithm also incorporates a normal-constrained projection to accurately assign points to bones.

Datasets

The paper evaluates FAKIR on both synthetic data and real-world point clouds derived from digitized statues. Specific examples of statues used include the "Wounded Amazon" and the "Age of Bronze." The authors do not introduce a new benchmark dataset but rather demonstrate the algorithm's performance on existing scanned artifacts.

Key Insights and Ideas

  • Pliable Sphere-Mesh Model: The use of a sphere-mesh model, rather than a rigid skeletal structure, is crucial for handling the often non-anatomical proportions of statues. This flexibility allows the model to adapt to artistic interpretations.
  • Forward and Backward Iterative Registration (FAKIR): The bidirectional refinement process is a significant contribution. It allows for a more comprehensive optimization by considering how adjustments in one part of the chain affect others, leading to a more accurate overall fit.
  • Pairwise Optimization: Focusing on pairs of bones during registration helps to stabilize the estimation of joint positions and common radii, especially in areas where bones connect.
  • Robustness to Imperfections: FAKIR demonstrates resilience to noise, missing data, and initial anchor point inaccuracies, making it practical for real-world scanned artifacts which are rarely perfect.

Key Results

  • Efficiency: FAKIR is computationally efficient. For a 10,000-point cloud and a 22-bone model, the first forward pass takes approximately 5 seconds, and subsequent passes take less than 2 seconds.
  • Accuracy: The algorithm achieves good registration results, even for statues with moderate clothing or missing parts. Qualitative results show a good overlay of the registered model onto the point cloud.
  • Comparison: Compared to the Pinocchio algorithm, FAKIR provides a better skeleton registration, particularly for shoulder and neck bones, and achieves comparable computation times while offering a more refined result.
  • Convergence: The algorithm converges in a small number of iterations (around 7 for synthetic data without noise) and is relatively insensitive to the number of points in the input data.

Limitations and Future Work

The authors acknowledge that FAKIR's current implementation is primarily designed for statues depicting naked or lightly clothed figures. Handling heavily clothed statues, where anatomy is hidden, is a significant limitation. Future work will focus on extending FAKIR to address this challenge, potentially by incorporating methods to infer hidden anatomy or by adapting the model to account for clothing.

Summary updated

Abstract

National audience
BibTeX
@inproceedings{fu2019anatomy,
  author = {Tong Fu and Raphaëlle Chaine and Julie Digne},
  title = {{Anatomy and pose estimation from point sets using FAKIR}},
  booktitle = {HAL},
  year = {2019},
}
Figure