Preserving Facial Identity Under Extreme Pose Transformations

30 concepts28 papers1 proposal

When a facial synthesis model rotates a face from a frontal view to a sharp 90-degree profile, the available visual information drops precipitously due to self-occlusion. A 2D generative model conditioned on a single reference image must suddenly "hallucinate" the shape of an occluded cheek, the bridge of the nose from an oblique angle, or the precise contours of the jaw.

Without explicit structural constraints, generative networks default to the statistical average of their training data. This causes viewpoint-dependent identity loss: as the rotation angle increases, the unique, fine-grained features of the individual dissolve into a generic, blurry approximation. As noted by Jianmin Bao et al. in Towards Open-Set Identity Preserving Face Synthesis, reconstructing unique facial features under significant perspective shifts inherently risks identity degradation.

Furthermore, 2D motion estimation and warping fields become highly ill-posed at extreme angles. When the angular difference between the source and target poses approaches 90 degrees, the target view shares almost no visual information with the source, leading to severe structural distortions and background warping, as highlighted by Xuemiao Xu et al. in Erratum to "Multi-View Face Synthesis via Progressive Face Flow". To overcome this extreme pose instability, modern architectures rely on three core mechanisms: feature disentanglement, 3D geometric grounding, and global memory compensation.

Disentangling Identity from Perspective

To prevent a model from altering a subject's identity when changing their pose, architectures must enforce strict feature disentanglement. The goal is to separate the latent representation into two orthogonal spaces: an invariant identity embedding (capturing structural constants like bone structure and eye shape) and a transient attribute embedding (capturing pose, expression, and lighting).

In diffusion-based personalization models, this decoupling prevents the network from leaking reference-view perspective details into the target generation. For instance, Siying Cui et al. developed IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Models to decouple identity-specific features from transient attributes, allowing text-to-image models to maintain consistency even across extreme profile views.

Similarly, Wooseok Jang et al. introduced ControlFace: Harnessing Facial Parametric Control for Face Rigging, which utilizes a dual-branch architecture to isolate identity preservation from explicit facial parametric control. By decoupling these factors, the model avoids the identity degradation typically seen when the perspective shifts significantly from the reference image.

Grounding Generative Models with 3D Priors

Purely 2D models lack an understanding of physical depth, meaning large spatial transformations often manifest as flat, physically impossible distortions. To resolve this, modern frameworks bake "3D common sense" directly into the generation pipeline using depth maps or 3D Morphable Models (3DMMs).

One approach is to use self-supervised depth estimation to guide the warping and attention mechanisms. Fa-Ting Hong et al. demonstrated in Depth-Aware Generative Adversarial Network for Talking Head Video Generation that integrating dense 3D geometry maps allows models to preserve structural integrity during 90-degree turns. Alternatively, geometry can act as a training supervisor without adding inference overhead. In G3FA: Geometry-guided GAN for Face Animation, Alain Pagani et al. leverage a pre-trained inverse rendering module to supervise the generator with depth and normal maps, ensuring the network respects 3D facial structures even under sharp lateral rotations.

For latent-space manipulation models like StyleGAN, 3DMMs can map out explicit geometric trajectories. Christos Tzelepis in One-Shot Neural Face Reenactment via Finding Directions in GAN's Latent Space used 3DMMs to discover controllable, structurally sound directions within a pre-trained latent space, allowing for extreme head rotations without breaking photorealism or identity.

Memory Compensation and Continuous Representations

When generating continuous video sequences (such as talking heads), the problem compounds: rapid head movements can cause the boundary between the neural facial representation and the original torso to misalign or warp. To bridge this gap, researchers utilize global memory banks and implicit neural representations.

The authors of Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head Video Generation introduced a global facial memory network designed to retrieve and compensate for missing view-dependent information when the head moves into extreme profiles. By querying this persistent identity memory, the model avoids relying solely on a single, highly occluded source frame.

Furthermore, blending parametric control with implicit neural representations (like NeRFs) provides a continuous coordinate space that naturally handles perspective shifts. Ricong Huang et al. developed Parametric Implicit Face Representation for Audio-Driven Facial Reenactment, conditioning implicit fields with 3D face parameters to eliminate boundary artifacts and maintain high-fidelity lip-sync and structural alignment even under extreme camera angles.

Key Foundational Work

Towards Open-Set Identity Preserving Face Synthesis (2018) — contributed 2 concept elements to this direction.
Photorealistic Facial Synthesis in the Dimensional Affect Space (2018) — contributed 1 concept elements to this direction.
Attribute-Guided Face Generation Using Conditional CycleGAN (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_extreme_pose_instability_18f3e3b9,
  title = {extreme pose instability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/extreme-pose-instability/rdr_18f3e3b9c28e4443adba41bf9ca4e6ef},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_extreme_pose_instability_18f3e3b9_v1,
  title = {extreme pose instability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/extreme-pose-instability/rdr_18f3e3b9c28e4443adba41bf9ca4e6ef/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake