Fast capture of personalized avatar using two Kinects

Yin Chen, Gang Dang, Zhi‐Quan Cheng, Kai XuJournal of Manufacturing Systems 2014 · DOI

TL;DR

This paper introduces a novel system for rapidly capturing personalized 3D avatars using two Microsoft Kinects. By simultaneously capturing front and back views and employing a sophisticated graph matching algorithm for alignment, the system achieves a full-body capture in just 3 seconds, significantly faster than previous methods, making it ideal for casual users.

What Problem Does This Paper Address?

Creating realistic, personalized 3D avatars has been a long-standing goal in computer graphics, enabling more immersive virtual experiences. Traditional methods, like those using laser scanners, are often expensive, complex, and require users to hold static poses for extended periods (e.g., 17 seconds), which is impractical for many. While depth cameras like Microsoft Kinect have lowered the barrier to entry, existing Kinect-based systems still face challenges: some require users to perform multiple poses, others take too long (e.g., 30 seconds), and achieving high-fidelity models with fine details like hairstyles and clothing wrinkles remains difficult. This paper aims to bridge this gap by enabling fast, high-quality, personalized avatar capture in a single, brief moment.

Methods

The core of the system relies on a clever setup and a novel alignment algorithm:

  • Dual Kinect Capture: Two Kinects are positioned opposite each other, capturing the front and back of the person. To overcome Kinect's limited field of view and scanning distance, each Kinect's motor is used to capture three partial scans (upper, middle, and lower body) by adjusting its pitch angle. This results in six partial scans in total.
  • Geometric Calibration: A simple yet effective calibration method is used to align the coordinate systems of the two Kinects. This involves capturing a crumpled A4 paper, which provides rich geometric features for alignment.
  • Scan Denoising: A depth filtering technique, adapted from existing work, is used to remove noise and unreliable data points from the captured scans.
  • Supersymmetric Third-Order Graph Matching: This is the paper's key algorithmic contribution. Instead of relying on traditional methods like ICP (Iterative Closest Points), which struggle with low overlap between scans, this algorithm uses a third-order graph matching approach. It leverages a "supersymmetric affinity tensor" to represent feature matching and a power iteration method to efficiently find the optimal rigid alignment between the partial scans. This method is robust to small overlaps and significantly faster than previous graph matching techniques.
  • Gap Filling: Since the front and back scans have a narrow band of missing data due to occlusion, a spatial quadratic Bézier curve fitting method is used to intelligently fill these gaps. This creates a closed mesh.
  • Poisson Surface Reconstruction: Finally, the filled and aligned scans are used to reconstruct a watertight 3D mesh model.

Datasets

Not discussed in the paper. The system is demonstrated on various individuals, implying real-world capture data rather than a specific benchmark dataset.

Key Insights and Ideas

  • The "Moment" as a Capture Window: The paper frames the ideal capture time as a "moment" (around 3 seconds), grounded in psychological studies of human behavior. This short duration is crucial for user comfort and practicality.
  • Dual Kinects for Full Coverage: Using two Kinects in opposition is a simple yet effective way to capture both sides of the body simultaneously, reducing the need for complex body rotations or multiple capture sessions.
  • Robust Alignment for Low Overlap: The supersymmetric third-order graph matching algorithm is a significant advancement for aligning partial scans that have minimal overlap, a common problem in multi-view capture systems. Its ability to handle mirrored transformations is particularly noteworthy.
  • Gap Filling for Watertight Models: The Bézier curve-based gap filling is essential for creating complete 3D models, preventing artifacts that would arise from direct reconstruction of incomplete scans.

Key Results

  • Capture Speed: The system achieves a capture time of approximately 3 seconds, a substantial improvement over existing methods.
  • Reconstruction Time: The entire process from capture to a textured 3D model takes about 1 minute.
  • Accuracy: Biometric measurements from reconstructed models show an average error comparable to other methods, with specific errors detailed in Table 2 (e.g., height error of 1.0 cm, waist girth error of 6.5 cm).
  • Model Fidelity: The reconstructed avatars capture personalized details like hairstyles, faces, and clothing wrinkles, as demonstrated in Fig. 10.

  • Cost-Effectiveness: The system is low-cost, with two Kinects costing around $300.

  • Space Efficiency: The system requires a small footprint (approximately 3m x 1m x 2m).

Limitations and Future Work

  • Depth Data Quality: The accuracy of the reconstructed mesh is limited by the inherent quality of Kinect's depth data, which degrades with distance. The authors suggest exploring advanced denoising and super-resolution techniques.
  • Occlusion Handling: The current gap-filling method only addresses missing data between front and back scans. It cannot recover geometry occluded by the user's own body. Future work could involve using prior templates or more advanced hole-filling techniques to address general occlusions.
  • Exploiting New SDKs: The authors suggest leveraging new features from updated Kinect SDKs to further improve the system's usability and capabilities.

Summary updated

BibTeX
@article{chen2014fast,
  author = {Yin Chen and Gang Dang and Zhi‐Quan Cheng and Kai Xu},
  title = {{Fast capture of personalized avatar using two Kinects}},
  journal = {Journal of Manufacturing Systems},
  year = {2014},
  doi = {10.1016/j.jmsy.2013.11.005},
}
Figure