Identity Preservation in Generative Image Synthesis

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2365 papers446 conceptsfrozen 2026-10-04

When a generative model edits a face—whether changing an expression, applying makeup, or aging a subject—it faces a fundamental tension: it must change the image enough to satisfy the edit, but not so much that the person becomes unrecognizable. This problem, often called identity drift, occurs because standard pixel-level or structural losses (like L1 or SSIM) are too rigid to allow for meaningful transformations, while adversarial losses are often too "creative," hallucinating new features that belong to a different person entirely.

To solve this, researchers have converged on a shared strategy: using pre-trained face recognition networks as "identity supervisors" that provide a high-level, semantic anchor for the generation process.

The Face Recognition Model as a Supervisor

The core intuition behind this research direction is that identity is not found in individual pixels, but in the relative geometry and features of a face. Modern face recognition (FR) models are trained on millions of images to map any face into a high-dimensional "embedding space" where different photos of the same person cluster together.

In works like FA-GANs, Jingwu He and colleagues demonstrate that by incorporating a deep identity descriptor into the loss function, a model can significantly warp facial proportions to enhance attractiveness while maintaining high structural similarity to the original subject. The FR model acts as a "critic" that ignores the lighting or the new makeup and focuses only on whether the underlying "fingerprint" of the face remains the same.

This mechanism is remarkably versatile. For instance, in Self-supervised Deformation Modeling for Facial Expression Editing, ShahRukh Athar uses an identity preservation loss to ensure that as facial muscles "deform" to create a smile or a frown, the high-level feature embeddings remain stable.

Steering the Latent Space

In the context of Diffusion Models and GANs, identity preservation acts as a "steering" signal. Rather than just hoping the model stays on track, researchers use the gradient of the identity loss to push the generation toward a specific target identity.

  1. Gradient Guidance: In $\text{ID}^3$, Ailin Deng shows how an ID-preserving loss creates an adjusted gradient vector field. This field steers the diffusion process, ensuring that even under extreme variations in pose or age, the core facial features are pulled back toward the reference identity.
  2. Reward Feedback: ID-Aligner by Weifeng Chen treats identity consistency as a reward signal. By comparing the facial features of a generated image against a reference using a cosine similarity score, the model receives direct feedback on how much "identity" it is losing during training.
  3. One-Step Lookahead: A challenge in Diffusion Models is that the final image isn't visible until the end of the denoising process. PortraitBooth, developed by Peng Xu and others, solves this by using a "one-step reverse formula" to estimate the final face from noisy intermediate states, allowing the identity loss to be applied throughout the entire generation sequence.

Disentanglement and Anchoring

A recurring theme across these papers is the need to disentangle identity from other attributes. If a model confuses "identity" with "pose," then rotating a head will inevitably change the person's face.

Huaibo Huang and the team behind Hierarchical Face Aging Through Disentangled Latent Characteristics address this by explicitly separating the latent space into independent factors for age, identity, and pose. They use knowledge distillation from a pre-trained FR network to "anchor" the identity vector, ensuring it remains constant even as the age vector is manipulated.

Similarly, in masked face recognition, the MEER network by Sufang Zhang uses a generative reconstruction task as a regularizer. By forcing the model to be able to "reconstruct" the unmasked face from a masked version, they ensure the identity embedding retains enough high-quality information for accurate recognition.

Beyond Faces: Appearance Consistency

While much of this work focuses on faces, the same principles are being applied to full-body human animation and urban scene generation. Antonin Vobecky and the creators of DummyNet use an "appearance consistency loss" to ensure that synthetic pedestrians maintain their visual characteristics (like clothing texture and color) when moved into different poses or environments.

Whether it is X2Face preventing a puppeteered face from adopting the driver's bone structure, or T-Person-GAN using manifold mix-up to keep generated people consistent across samples, the goal remains the same: constraining the creative power of generative models with the discriminative precision of recognition models. This synergy allows AI to transform the context of an image while leaving the essence of the subject untouched.

Papers in this version

Showing 60 of 365 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 319 papers
  2. Version 1 · 2026-10-04 · 365 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_identity_preserving_loss_fc393977,
  title = {identity-preserving loss},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/identity-preserving-loss/rdr_fc393977a0cf49e39b03d1a5696c6c52},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_identity_preservation_in_generative_imag_fc393977_v1,
  title = {Identity Preservation in Generative Image Synthesis},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/identity-preservation-in-generative-image-synthesis/rdr_fc393977a0cf49e39b03d1a5696c6c52/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}