TL;DR
SurFhead is a new framework for creating high-fidelity, riggable head avatars using 2D Gaussian surfels and affine rigging. By replacing simple similarity transforms with Jacobian-based deformations and a novel "Jacobian Blend Skinning" technique, it achieves state-of-the-art geometric accuracy and handles extreme facial expressions without the "blobby" artifacts common in previous methods.
What Problem Does This Paper Address?
While recent head avatar models based on 3D Gaussian Splatting (3DGS) produce photorealistic images, their underlying geometry is often inaccurate. Most existing methods rely on similarity transformations—which only allow for rotation, translation, and uniform scaling—to move Gaussians along with a face mesh. However, human skin stretches and shears significantly during speech and emotion. Forcing these complex deformations into a rigid similarity framework leads to "over-smoothed" geometry and artifacts in extreme poses.
Furthermore, many models suffer from a "hollow illusion" in the eyes, where the cornea appears concave rather than convex. This happens because the optimization process prioritizes matching the image pixels (photometric loss) by making the eye surface transparent to simulate specularity, rather than reconstructing the actual curved geometry of the eyeball.
Methods
SurFhead introduces three key technical innovations to bridge the gap between classical graphics and modern neural rendering:
- 2D Gaussian Surfels with Affine Rigging: Instead of 3D "blobs," the authors use 2D disks (surfels). These have well-defined normals and depths. They move these surfels using a Jacobian deformation gradient ($J$), which allows for full affine transformations, including the stretch and shear essential for realistic skin movement. Crucially, they use the inverse-transpose of the Jacobian ($J^{-T}$) to transform surface normals, ensuring they remain mathematically consistent with the deformed surface.
- Jacobian Blend Skinning (JBS): To ensure smooth transitions between different parts of the face mesh, the authors propose JBS. Standard linear blending of transformation matrices can cause geometry to "collapse" or distort (see the "Direct Interpolation" in Figure 4). JBS uses Polar Decomposition to split the Jacobian into a rotation component ($U$) and a stretch/shear component ($P$). Rotations are blended in a specialized matrix-logarithm space, while stretch/shear is blended linearly, preventing geometric distortions.
- Eyeball Regularization & ASGs: To fix the "hollow eye" problem, the authors constrain the eyeball region to be convex and opaque. They replace standard Spherical Harmonics (which are too low-frequency to capture sharp glints) with Anisotropic Spherical Gaussians (ASGs). This allows the model to render sharp, realistic specular reflections on the cornea without sacrificing geometric accuracy.
Datasets
The authors evaluated SurFhead on two primary datasets:
- FaceTalk: A monocular synthetic dataset based on the FLAME model. It includes ground-truth surface normals, making it ideal for measuring geometric fidelity.
- NeRSemble: A multi-view real-world dataset featuring 16 cameras and diverse facial expressions. The authors created a custom "held-out" set of extreme expressions to test how well the models generalize to difficult poses.
Key Insights and Ideas
- Normals are the key to realism: By focusing on "geometrically accurate" rigging, the authors show that better normals lead to better relighting and more robust performance in extreme poses.
- Classical Graphics meets 3DGS: The paper demonstrates that "old-school" techniques like Polar Decomposition and Jacobian gradients are still highly relevant for solving the stability issues of modern neural point-based representations.
- The "Hollow Eye" is an optimization trap: The discovery that models "cheat" by using transparency to fake specularity highlights the need for explicit geometric constraints (like convexity) when dealing with highly reflective surfaces like eyes.
Key Results
SurFhead outperforms existing baselines like GaussianAvatars and SplattingAvatar across several metrics:
- Geometric Accuracy: On the FaceTalk dataset, SurFhead achieved a Normal Cosine Similarity (NCS) of 0.983, significantly higher than baselines (e.g., PointAvatar at 0.756).
- Rendering Quality: It maintains high PSNR and SSIM even in extreme poses where other methods produce semi-transparent or "blob-like" artifacts.
- Efficiency: Despite the complex math, the model renders at roughly 90 FPS on a modern GPU, making it suitable for real-time applications.
Limitations and Future Work
The authors acknowledge a few remaining hurdles:
- Expression Bounds: Because the model is tied to the FLAME mesh, it cannot represent expressions that fall far outside the FLAME expression space (e.g., very exaggerated faces that the tracker can't follow).
- Missing Details: The current model lacks explicit representations for hair strands and the tongue, which are not part of the underlying 3D Morphable Face Model (3DMFM).
- Computational Bottleneck: The Polar Decomposition step currently relies on SVD (Singular Value Decomposition), which can be slow for large batches. The authors suggest that specialized 3x3 matrix routines could further boost speed in the future.
Summary updated