A Feature-Based Deformable Model for Photo-Realistic Head Modelling

Yong‐Jin Liu, Matthew M. F. Yuen, Shan XiongOpenalex Unknown 2001 · DOI · OpenAlex

Abstract

We propose a feature-based approach for creating photo-realistic textured 3D head models. First, from a discrete data set of a human head, we generate a generic head model which is feature-based and semi-regular. Then, we take a pair of photos of a human subject from two orthogonal directions. After recovering a set of 3D feature points on the head from photos, we build an individualized geometric model level by level. Finally, we synthesize the photos into a view-independent texture map and automatically generate the texture coordinates. By mapping the texture onto the geometric model, we can efficiently generate highly photo-realistic head models for individuals.

Introduction

In the early 2000s, creating a photo-realistic 3D head typically required expensive, specialized hardware like Cyberware laser scanners or labor-intensive manual modeling. Low-cost alternatives using standard cameras faced a significant technical hurdle: deforming a generic 3D head mesh to fit a specific person's photos without losing fine details or creating artifacts. Traditional parametric surfaces were often too rigid for fine details, while standard polygonal meshes made it difficult to calculate the movement of "non-feature" regions (like cheeks) when "feature" parts (like eye corners) were adjusted.

Methodology

The proposed approach bridges the gap between rigid mathematical models and flexible polygons using subdivision surfaces. The process follows three main stages:

1. Generic Model Generation

The system starts with a high-quality scan of a human head, represented as a discrete 3D data set. This data is converted into a "semi-regular" generic model using a displaced butterfly subdivision scheme.

Discrete 3D data set of a human head used as the basis for the generic model.

This scheme allows the head to be represented as a base mesh ($M^0$) plus a series of "detail sets" ($D^1$ through $D^4$), stored as scalar offsets along surface normals.

2. Individualization

A user captures two photos of a subject (front and side). Key feature lines are identified using a "structure snake" algorithm combined with manual selection. The system then employs structure-from-motion to calculate camera positions and recover 3D coordinates for these features.

The feature-based generic head model shown from profile and frontal views, highlighting the numbered feature points used for deformation.

3. Deformation and Texturing

The generic base mesh is deformed to match the recovered 3D feature points. Because the model is hierarchical, the stored detail sets ($D^i$) can be reapplied to the new shape, preserving the "character" of the face. Finally, the photos are blended into a single view-independent texture map, with texture coordinates assigned automatically based on the mesh's semi-regular structure.

Key Insights

  • Subdivision Surfaces as a Bridge: They provide the mathematical smoothness of parametric surfaces with the detail-handling flexibility of polygonal meshes.
  • Multi-level Displacement: Storing facial details as offsets ($d$) from a smooth surface rather than absolute coordinates makes facial characteristics "portable."
  • Semi-Regularity: This structure ensures a predictable one-to-one correspondence between 2D texture space and the 3D model, eliminating common texture stretching issues.

Results and Discussion

The system generates individualized models efficiently compared to manual methods. Qualitative results demonstrate that the model effectively captures unique silhouettes and skin textures. Furthermore, the system is robust enough to handle non-perfectly orthogonal photos by calculating a rotation matrix $R$ and translation vector $T$.

However, some limitations remain. The "view-independent" texture mapping can result in slight blurring compared to view-dependent methods. Additionally, the process still requires manual intervention to identify key feature points. Future work includes extending the model to facial animation, video tracking, and geometry compression.

Summary updated

Abstract

We propose a feature-based approach for creating photo-realistic textured 3D head model. First, from a discrete data set of human head we generate a generic head model which is feature-based and semi-regular. Then we take a pair of photos of a human subject from two orthogonal directions. After recovering a set of 3D feature points on the head from photos, we build an individualized geometric model level by level Finally we synthesize the photos into a view-independent texture map and automatically generate the texture coordinates. By mapping the texture onto the geometric model, we can efficiently generate highly photo-realistic head models for individuals.
BibTeX
@inproceedings{liu2001feature,
  author = {Yong‐Jin Liu and Matthew M. F. Yuen and Shan Xiong},
  title = {{A Feature-Based Deformable Model for Photo-Realistic Head Modelling}},
  booktitle = {Openalex Unknown},
  year = {2001},
  doi = {10.1007/978-0-306-47002-8\_4},
}
Figure