Pose-Dependent Neural Deformation for Animatable Human Avatars

This direction is merged into another direction: pose-dependent appearance modeling. Its last version stays readable and citable here.

Version 1 of 141 papers47 conceptsfrozen 2026-10-04

Traditional computer graphics relies on skeletal animation and Linear Blend Skinning (LBS) to move digital characters. While LBS is computationally efficient, it is fundamentally "rubbery." It treats the human body as a collection of rigid segments, failing to capture the non-linear physical realities of a living person: the way a bicep bulges during a curl, how skin stretches over a knuckle, or how a t-shirt bunches into complex folds when an arm is raised.

A growing body of research is replacing these static geometric rules with pose-dependent neural deformation fields. By conditioning neural networks on skeletal pose, researchers can now synthesize high-fidelity, animatable avatars that exhibit realistic, dynamic surface details from minimal input data.

The Limits of Rigid Transformation

The core challenge in creating a digital twin is disentangling identity (the static shape of the person) from pose (their current movement). Standard parametric models like SMPL provide a good "base" mesh, but they lack the person-specific details—like clothing wrinkles or facial pores—that make an avatar look real.

To solve this, researchers like Hao Zhao and Zerong Zheng propose a coarse-to-fine strategy. In High-Fidelity Human Avatars from a Single RGB Camera, they use a dynamic surface network to predict vertex offsets. Instead of just moving a point based on a bone's rotation, the network asks: "Given that the elbow is bent at 90 degrees, how much should this specific patch of fabric shift to form a wrinkle?"

Conditioning on the Skeleton

The "intuition" behind this research direction is that surface deformation is a function of joint angles. If you know the rotation of the shoulder and elbow, you can theoretically predict the state of the shirt covering them.

This is often implemented by feeding the skeletal pose—represented as a vector of joint rotations—directly into a Multi-Layer Perceptron (MLP). As shown by Andrei Burov and Justus Thies in Dynamic Surface Function Networks, conditioning on these kinematic parameters allows the model to learn the complex relationship between movement and fabric behavior.

This mechanism is not limited to full bodies. In the facial domain, Haiwen Feng and Timo Bolkart developed DECA, which disentangles static identity from dynamic expressions. By conditioning detail generation on expression parameters, they ensure that wrinkles appear and disappear realistically as the face moves, rather than being "baked" into the texture.

From Implicit Fields to Gaussian Splatting

The representation of these avatars has evolved rapidly to balance quality and speed. Early neural avatars used Neural Radiance Fields (NeRFs), which are high-quality but slow to render. Jie Chen introduced NDF, which "unwraps" 3D space into a coordinate system aligned with a body model, allowing the NeRF to focus on pose-dependent appearance variations.

However, the field is shifting toward 3D Gaussian Splatting (3DGS) for real-time performance. Qian Zhi-yin and Siyu Tang demonstrated in 3DGS-Avatar that by attaching Gaussians to a deformable structure and using a pose-dependent MLP to adjust their attributes (like scale and position), they could achieve 50+ FPS rendering. This approach, also explored in GaussianAvatar, allows the model to represent fine-grained, non-rigid deformations like clothing folds that a rigid mesh cannot.

Generalizing the Deformation

The ultimate goal is a model that understands deformation so well it can be applied to new identities or garments. Qianli Ma pioneered this with CAPE, a generative model that "dresses" humans by predicting pose-dependent clothing offsets.

More recently, AvatarVTON by Zicheng Jiang et al. extends this to virtual try-on. By transferring learned deformation patterns from one subject to a new garment, the system can simulate how a specific dress would drape and wrinkle on a moving person without needing a full physics simulation.

Why It Works

The success of pose-dependent neural deformation lies in its ability to treat the human surface as a dynamic manifold. By mapping every point on the body back to a "canonical" (neutral) pose and then applying a learned, pose-conditioned correction, these models bridge the gap between the efficiency of skeletal rigs and the realism of physical reality. As Vladislav Golyanik shows in VINECS, replacing static skinning weights with neural, pose-dependent weights eliminates the "candy-wrapper" artifacts of traditional animation, finally allowing digital characters to move with the fluidity of their human counterparts.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_pose_dependent_neural_deformation_for_an_607534b7,
  title = {Pose-Dependent Neural Deformation for Animatable Human Avatars},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/pose-dependent-neural-deformation-for-animatable-human-avatars/rdr_607534b751b4424fa6c6018cd55ea91c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_pose_dependent_neural_deformation_for_an_607534b7_v1,
  title = {Pose-Dependent Neural Deformation for Animatable Human Avatars},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/pose-dependent-neural-deformation-for-animatable-human-avatars/rdr_607534b751b4424fa6c6018cd55ea91c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}