Decoupling Identity and Motion in Facial Animation

416 concepts276 papers1 proposal

When animating a digital avatar or performing cross-identity facial reenactment, a fundamental tension arises: how do you transfer the dynamic movements of a "driver" onto the face of a "source" without altering the source's unique physical identity?

If you naively copy facial landmarks or pixel movements from a driver to a target, you encounter identity leakage. For instance, if the driving actor has a wider jaw or larger eyes than the target subject, a naive model will inadvertently stretch the target's skull structure to match the driver's anatomy. The resulting animation looks less like the target person expressing an emotion and more like a bizarre hybrid of both individuals.

To solve this, a major research direction focuses on identity-motion decoupling. The core intuition is to separate a portrait or 3D mesh into two distinct, orthogonal components: static identity (the underlying bone structure, skull shape, and intrinsic skin texture) and dynamic motion (non-rigid expressions, rigid head poses, and eye gaze). By isolating these factors, models can manipulate what a face is doing completely independently of what the face looks like.


Un-Posing the Face via Canonical Spaces

One of the most intuitive ways to separate identity from motion is to establish a common ground: a canonical space. Imagine taking any face, regardless of its current expression or orientation, and mathematically "un-posing" it into a standardized, perfectly neutral template.

By mapping a source image into this neutral canonical space, the model strips away all transient motion, leaving behind a pure representation of the subject's identity. To animate this subject, the model calculates how a driving video deforms away from its own neutral state, and then applies that exact deformation field to the source's canonical representation.

[Source Image] ---> Map to Canonical Space ---> [Pure Identity Volume]
                                                       |
[Driver Video] ---> Extract Motion Field   ------------+---> [Animated Output]

Wangbo Yu et al. implement this by mapping source images to a neutral canonical 3D volume, ensuring that deformation fields can be applied consistently across diverse subjects without distorting their underlying facial structures. Similarly, Evangelos Ververas utilizes a 3D key-point estimator that regresses facial landmarks and transforms them into a canonical space, effectively un-posing the face to isolate unique geometry from dynamic movements. To handle highly complex variations, Hyunsoo Cha et al. design a multi-stage hierarchy of canonical spaces, transitioning from a person-agnostic generic space to a subject-specific identity space, and finally to a parametric model space for seamless animation.


Relative Displacements over Absolute Coordinates

Why do traditional 2D landmarks cause identity leakage? Because they are expressed in absolute spatial coordinates. If a driver opens their mouth wide, the absolute coordinate of their chin drops significantly. If applied directly to a target with a naturally short chin, the target's face will warp unnaturally.

To prevent this, modern architectures shift from absolute coordinates to relative displacements. Instead of learning where a facial feature is, the network learns how much it moves relative to its own baseline.

Naima Otberdout et al. demonstrate this by training models to predict vertex displacements relative to a neutral pose rather than absolute spatial coordinates. This representational strategy ensures that motion patterns can be transferred across entirely different facial topologies without distorting the target's structural features. Along the same lines, the authors of this work use a co-attention mechanism to predict paired feature points between the source and driver, capturing relative motion rather than absolute anatomical landmarks to safeguard the source's unique characteristics.


3D Morphable Models as Mathematical Filters

Instead of forcing a neural network to discover the boundary between identity and motion from scratch, many frameworks bake this separation directly into the network's architecture using 3D Morphable Models (3DMMs).

A 3DMM is a statistical model that describes a face using mathematically decoupled vectors: a shape vector ($\alpha$) representing static identity, and an expression vector ($\beta$) representing dynamic muscle movements. By conditioning a generative network on these parameters, the 3DMM acts as a structural scaffold that filters out identity information from the driving source.

Driver Video ---> 3DMM Fitting ---> Expression Vector (β) ---\
                                                              +---> Generative Network ---> Final Video
Source Image ---> 3DMM Fitting ---> Shape Vector (α) --------/

Pravir Singh Gupta et al. integrate a 3D morphable model into the conditioning pipeline, forcing the generator to adopt the mathematical separation of shape, expression, and pose inherent in the 3D model, which prevents attribute cross-talk. When performing high-fidelity face swapping, Hao Shao et al. use 3DMMs to provide a geometric scaffold for pose and expression while explicitly stripping away texture and identity information. This ensures the model captures the target's motion without leaking their original facial features into the final swap.

To make this process robust to "in-the-wild" variations, Zhiliang Xu et al. utilize 3D-projected landmarks instead of raw 2D landmarks, allowing the model to replicate target expressions and poses without distorting identity-defining features like jaw shape or eye spacing. Anastasios Roussos et al. exploit this by creating a hybrid parameter set that retains the target's identity coefficients while adopting the source's movement coefficients.


Forcing Disentanglement Through Feature Shuffling

When explicit 3D models are not used, how do you force a deep neural network to understand that a smile is independent of the person smiling? The answer lies in adversarial shuffling and contrastive learning during training.

If a network is trained to reconstruct a video frame by extracting identity from Image A and motion from Image B (a process called cross-identity training), it cannot cheat by simply memorizing the geometry of Image B. The network is forced to find an abstract representation of motion that applies universally.

Image A (Identity) ---\
                       +---> Shuffled Injection ---> Hybrid Output (Identity A performing Motion B)
Image B (Motion)   ---/

Yuhang Ma et al. implement this via a "shuffled" identity injection strategy, extracting identity features from a reference image and applying them to a target frame with a completely different layout to prevent overfitting to the reference image's geometry. Subin Jeon et al. enforce this separation using a cross-identity reconstruction loss that synthesizes a pseudo-driving frame where one subject mimics the pose of another.

For fine-grained control, Shiwei Jin et al. show that cross-subject training—where the movements of one individual are applied to the identity of another—serves as a robust disentanglement mechanism for manipulating facial Action Units even when expert-labeled data is scarce. This concept extends beyond faces: Xiyuan Kang applies a similar feature-swapping technique to 3D human mesh recovery, swapping pose and shape features between different individuals during training to ensure the model maintains a consistent body shape across diverse movements.

Alternatively, contrastive learning can isolate these features by looking at the same person across time. Taekyung Ki et al. apply contrastive learning to video sequences of the same individual with varying expressions. Because the identity remains constant, the contrastive objective effectively isolates the changing motion features from the static identity-specific traits.


Static-Dynamic Hybrid Architectures

A final, powerful architectural pattern splits the neural network into two distinct pipelines: a static pathway optimized for high-frequency spatial details (like skin pores, moles, and bone structure) and a dynamic pathway optimized for temporal deformations (like wrinkles and muscle stretching).

                  /---> Static Pathway  ---> Tri-Planes / Textures (Identity) \
Input Portrait --+                                                             +---> Rendered Output
                  \---> Dynamic Pathway ---> Deformation Fields / LoRA (Motion) /

Zenghao Chai et al. leverage this by separating person-specific static skin textures from expression-driven dynamic wrinkles. This prevents the transfer of identity-specific traits during facial animation, allowing a target's expressions to be applied to a source identity without introducing unnatural artifacts.

In the context of efficient 3D representations, the authors of MimicTalk propose a static-dynamic hybrid approach where static features like geometry are captured through tri-plane inversion, while person-specific motion patterns are learned separately by fine-tuning low-rank adaptation (LoRA) layers. Similarly, Weichuang Li et al. use a Deformable Neural Radiance Field (NeRF) to explicitly decouple rigid head pose from non-rigid expressions, preventing identity leakage where the facial structure of the driving source inadvertently alters the target subject's appearance.

By combining these architectural splits, canonical mappings, and relative motion constraints, modern generative models achieve the ultimate goal of facial reenactment: total control over expression and pose, with absolute fidelity to the original identity.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 283 papers
  2. Version 1 · 2026-10-04 · 277 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_identity_expression_decoupling_db0c65a4,
  title = {identity-expression decoupling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/identity-expression-decoupling/rdr_db0c65a43c25429a93d1cf7b34f50288},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_identity_expression_decoupling_db0c65a4_v2,
  title = {identity-expression decoupling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/identity-expression-decoupling/rdr_db0c65a43c25429a93d1cf7b34f50288/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake