Abstract
We propose a feature-based approach for creating photo-realistic textured 3D head models. First, from a discrete data set of a human head, we generate a generic head model which is feature-based and semi-regular. Then, we take a pair of photos of a human subject from two orthogonal directions. After recovering a set of 3D feature points on the head from photos, we build an individualized geometric model level by level. Finally, we synthesize the photos into a view-independent texture map and automatically generate the texture coordinates. By mapping the texture onto the geometric model, we can efficiently generate highly photo-realistic head models for individuals.
Introduction
In the early 2000s, creating a photo-realistic 3D head typically required expensive, specialized hardware like Cyberware laser scanners or labor-intensive manual modeling. Low-cost alternatives using standard cameras faced a significant technical hurdle: deforming a generic 3D head mesh to fit a specific person's photos without losing fine details or creating artifacts. Traditional parametric surfaces were often too rigid for fine details, while standard polygonal meshes made it difficult to calculate the movement of "non-feature" regions (like cheeks) when "feature" parts (like eye corners) were adjusted.
Methodology
The proposed approach bridges the gap between rigid mathematical models and flexible polygons using subdivision surfaces. The process follows three main stages:
1. Generic Model Generation
The system starts with a high-quality scan of a human head, represented as a discrete 3D data set. This data is converted into a "semi-regular" generic model using a displaced butterfly subdivision scheme.
This scheme allows the head to be represented as a base mesh ($M^0$) plus a series of "detail sets" ($D^1$ through $D^4$), stored as scalar offsets along surface normals.
2. Individualization
A user captures two photos of a subject (front and side). Key feature lines are identified using a "structure snake" algorithm combined with manual selection. The system then employs structure-from-motion to calculate camera positions and recover 3D coordinates for these features.
3. Deformation and Texturing
The generic base mesh is deformed to match the recovered 3D feature points. Because the model is hierarchical, the stored detail sets ($D^i$) can be reapplied to the new shape, preserving the "character" of the face. Finally, the photos are blended into a single view-independent texture map, with texture coordinates assigned automatically based on the mesh's semi-regular structure.
Key Insights
- Subdivision Surfaces as a Bridge: They provide the mathematical smoothness of parametric surfaces with the detail-handling flexibility of polygonal meshes.
- Multi-level Displacement: Storing facial details as offsets ($d$) from a smooth surface rather than absolute coordinates makes facial characteristics "portable."
- Semi-Regularity: This structure ensures a predictable one-to-one correspondence between 2D texture space and the 3D model, eliminating common texture stretching issues.
Results and Discussion
The system generates individualized models efficiently compared to manual methods. Qualitative results demonstrate that the model effectively captures unique silhouettes and skin textures. Furthermore, the system is robust enough to handle non-perfectly orthogonal photos by calculating a rotation matrix $R$ and translation vector $T$.
However, some limitations remain. The "view-independent" texture mapping can result in slight blurring compared to view-dependent methods. Additionally, the process still requires manual intervention to identify key feature points. Future work includes extending the model to facial animation, video tracking, and geometry compression.
Summary updated