When creating digital humans, computer vision and graphics have long relied on parametric templates. Models like SMPL for human bodies or FLAME for faces provide a robust mathematical foundation: a fixed graph structure with a predefined number of vertices and edges. This consistency is incredibly powerful. Because every digital human shares the exact same topology, animating them via linear blend skinning is straightforward, and learning statistical variations across populations becomes a tractable optimization problem.
However, this fixed topology acts as a geometric straightjacket. A template designed for a naked human body or a smooth face cannot naturally adapt to the real world's topological complexity—such as flowing skirts, open jackets, intricate hairstyles, glasses, or changing physical structures like an opening mouth.
To represent these complex geometries, researchers have traditionally applied vertex displacements (offsets) to the base template. But vertex offsets can only stretch or compress the existing surface; they cannot create holes, merge separate components, or cleanly model structures that exist far from the base mesh. Over the past few years, a major shift has occurred to break free from these fixed-topology limitations.
The Failure Modes of Vertex Offsets
To understand why the field is moving away from pure mesh templates, consider what happens when trying to model a person wearing a skirt using a standard body mesh.
A human body template has two distinct leg structures. A skirt, however, is a single continuous surface enclosing both legs. As shown by Ma et al., attempting to force a body template to represent a skirt via vertex offsets results in "shrink-wrapped" or "pants-like" artifacts. The mesh simply lacks the connectivity to bridge the gap between the legs without tearing or distorting.
Similarly, Corona et al. point out that garments like open jackets or button-down shirts change their topology based on how they are worn. A closed shirt is a single tube; an open jacket splits into separate flaps. A fixed-topology mesh cannot transition between these states because its edge connectivity is locked at initialization.
This rigidity extends to facial and head modeling. Traditional 3D morphable models assume a constant number of vertices, which Giebenhain et al. note makes it impossible to capture high-variance structures like long hair, ears, or glasses within a single parametric framework. Furthermore, as Giebenhain et al. observe, when a subject opens their mouth or closes their eyes, surfaces must physically separate or join—a fundamental challenge for a rigid template mesh.
Fixed Mesh Template --------> Vertex Offsets --------> Tearing & Distortion
(Locked Connectivity) (Cannot change (Fails on loose clothes,
topology) hair, open mouths)
To solve these problems, researchers have developed three core strategies: replacing meshes with continuous implicit fields, utilizing unconnected point-based representations, and engineering hybrid models that anchor flexible primitives to rigid templates.
Strategy 1: Continuous Implicit Fields
The most radical way to solve topological rigidity is to eliminate the discrete mesh entirely during the modeling phase. Instead of defining a surface by a set of connected vertices, neural implicit functions represent geometry continuously as the level set of a neural network (e.g., a Signed Distance Function or an Occupancy Field).
In this paradigm, a neural network takes a 3D coordinate $(x, y, z)$ as input and predicts whether that point is inside or outside the surface. Because the surface is extracted implicitly, it can assume any arbitrary topology.
- Full-Head Modeling: Tewari et al. introduced i3DMM, which uses neural implicit functions to capture the full human head. By abandoning mesh templates, the model seamlessly accommodates highly variable geometries like voluminous hair and expressions without requiring dense surface correspondences during training.
- Holistic Avatars: Shen developed X-Avatar, scaling implicit representations to capture body pose, hand gestures, facial expressions, and loose clothing simultaneously. Without a fixed topology, the model naturally captures high-frequency details that fall outside standard human anatomy.
While implicit fields offer total topological freedom, they lack an inherent kinematic structure. Animating a pure implicit field across diverse poses is computationally expensive and prone to artifacts, as it lacks the strong structural guidance that a template mesh provides.
Strategy 2: Point Clouds and Local Patches
Another approach retains explicit geometry but discards the rigid connectivity (edges) between vertices. By representing shapes as point clouds or collections of independent local elements, the geometry can freely separate, cluster, and deform.
Zakharkin et al. demonstrated this by using a cloud transformer architecture to model human clothing. Because points do not have fixed connectivity templates, a single unified framework can transition between distinct garment categories—such as morphing a tight shirt into a loose dress—without manual reconfiguration.
Mesh Representation Point-Based Representation
(Rigid Edges) (No Edge Constraints)
o---o---o o o o
/ \ / \ / \
o---o---o---o o o o o
(Tears if stretched) (Freely separates/reclusters)
To combine the flexibility of points with the structural stability of body models, Ma et al. proposed SCALE, which models clothed humans using hundreds of small, articulated surface patches that "dress" a body. Similarly, Ma et al. developed SkiRT, a template-free point-based framework that replaces rigid body-centric unposing with learned skinning weights, eliminating the "tearing" artifacts common when modeling loose-fitting skirts.
Strategy 3: Hybrid Representations (The Best of Both Worlds)
The most recent trend in the field seeks a compromise: keeping the parametric template mesh to handle macro-motion and skeleton rigging, but attaching flexible, volumetric primitives to it to handle complex, non-conforming geometry.
Rigged 3D Gaussians
With the advent of 3D Gaussian Splatting, researchers realized they could "bind" thousands of independent 3D Gaussians to the facets of a traditional template mesh. The underlying mesh dictates how the avatar moves, while the Gaussians can extend far beyond the mesh boundaries to represent hair, clothing, and accessories.
- Qian et al. introduced GaussianAvatars, rigging 3D Gaussians directly to the triangles of a FLAME mesh. This allows photorealistic head animation while bypassing the topological limits of the underlying face model.
- Xiang et al. developed FlashAvatar, embedding a sparse set of Gaussians onto a parametric face model to render complex features like hair and glasses at 300 FPS.
- Wen proposed GoMAvatar, creating a hybrid "Gaussians-on-Mesh" representation that allows accurate tracking of loose clothing and items that detach from the body, which would normally break a fixed-topology system.
Deformable Tetrahedral Grids
Alternatively, researchers can embed the template within a dynamic volumetric grid. Zhang et al. introduced Dynamic Tetrahedra (DynTet) for talking head synthesis. By anchoring neural implicit fields to a deformable tetrahedral grid, their method maintains the structural consistency of explicit meshes while gaining the topological freedom needed to render internal mouth structures and complex hairstyles.
Summary of Trade-offs
The evolution of this research direction represents a continuous negotiation between structural control and topological flexibility:
| Representation | Topological Flexibility | Animation/Rigging Ease | Key Limitation |
|---|---|---|---|
| Fixed Mesh Templates (e.g., SMPL) | Low | High | Cannot model geometry deviating from base form (skirts, hair). |
| Pure Implicit Fields (SDFs/NeRFs) | High | Low | Computationally expensive to animate accurately across poses. |
| Point Clouds / Patches | High | Medium | Lacks surface continuity; requires dense sampling to avoid holes. |
| Hybrid Models (Gaussians/Tetrahedra on Mesh) | High | High | Stability decreases if spatial offset from the base mesh is too large. |
By decoupling the kinematic structure (the skeleton or proxy mesh) from the geometric topology (the rendered surface), modern 3D human modeling frameworks successfully bypass historical constraints. The reader should see that the field has not abandoned templates; rather, it has repurposed them as structural scaffolding, allowing flexible neural and volumetric representations to handle the messy, non-rigid reality of the real world.
Key Foundational Work
Learning to Dress 3D People in Generative Clothing (2020) — contributed 1 concept elements to this direction.
ARCH: Animatable Reconstruction of Clothed Humans (2020) — contributed 1 concept elements to this direction.
Dynamic Surface Function Networks for Clothed Human Bodies (2021) — contributed 1 concept elements to this direction.