Abstract
HHAvatar is a novel framework that creates ultra-high-fidelity (2K resolution) animatable 3D head avatars using 3D Gaussian Splatting. By combining a fully learned deformation field with a history-aware temporal module, it successfully captures fine-grained facial details and realistic, physics-based dynamic hair motion from sparse-view video.
Introduction
Creating digital doubles that look and move realistically is a cornerstone of VR, telepresence, and film. While Neural Radiance Fields (NeRF) lowered the barrier for reconstruction, they struggle to render at 2K resolution in real-time and often fail to capture "high-frequency" details like individual hair strands or wrinkles. Furthermore, most existing models use Linear Blend Skinning (LBS), which is too rigid to represent exaggerated expressions. A particularly stubborn challenge is hair; unlike facial skin, which moves predictably with bone structure, hair has inertia. Its position depends on previous velocity and head acceleration, leading to "helmet-like" hair in static models.
Methodology
HHAvatar builds on 3D Gaussian Splatting (3DGS), an explicit point-based representation known for high rendering speeds and detail. The authors introduce three key innovations:
- Fully Learned Deformation Field: Instead of relying on a rigid template, an MLP-based deformation field allows Gaussians to move fluidly, capturing subtle skin sliding and extreme expressions.
- Temporal Hair Module: To model hair physics, the authors treat hair separately from the face. They feed the hair's position and speed from the previous two frames ($t-1, t-2$) into a specialized MLP, allowing the model to learn inertia and simulate realistic swinging.
- Geometry-Guided Initialization: The authors first optimize an implicit Signed Distance Function (SDF) and use Deep Marching Tetrahedra (DMTet) to extract a rough mesh. This mesh provides a high-quality "skeleton" to initialize the Gaussians, ensuring stability during training.
Finally, a super-resolution network upscales the 512px neural renders to crisp 2K images.
Datasets and Evaluation
The framework was evaluated on three primary sources: the NeRSemble dataset (10 identities, 16 cameras), the HAvatar Dataset (4K data), and a Self-Captured Dataset specifically designed to test hair dynamics during rapid head movements.
Key Results and Insights
HHAvatar sets a new state-of-the-art for sparse-view head reconstruction. On the NeRSemble dataset, it achieved a PSNR of 27.70 and an LPIPS (2K) of 0.098, significantly outperforming previous leaders.
A core insight of the work is the realization that facial skin is state-dependent (determined by current expression), while hair is history-dependent (determined by motion over time). Decoupling these allows for much more realistic animation where hair realistically "lags" and swings during rapid head turns.
Limitations and Future Work
Despite the high fidelity, the authors note that the mouth interior (tongue and teeth) can occasionally appear blurry due to tracking difficulties. Additionally, the model cannot synthesize expressions significantly different from the training data, and extreme head speeds can cause the underlying pose estimation to fail. Future work may involve integrating better oral tracking or exploring generative priors for unseen expressions.
Summary updated