When a generative model alters a face—whether by adding wrinkles for age, applying makeup, or changing a hairstyle—it must navigate a fundamental tension known as the identity-transformation trade-off. Standard generative models are highly proficient at synthesizing realistic textures. However, if left unconstrained, they tend to drift toward generating a generic human face that fits the target attribute, erasing the fine geometric and structural nuances that make an individual recognizable. This phenomenon is known as identity drift.
Biometric identity is encoded in subtle spatial relationships and underlying bone structures. When a model applies a heavy transformation—such as translating a thermal image to a visible one (Zhang et al.) or super-resolving a blurry face (Song et al.)—it is easy to synthesize visually pleasing but biometrically incorrect features. For downstream tasks like facial verification, this drift renders the output useless. As demonstrated by Yadav et al., explicitly minimizing this identity gap before running images through a recognition system can dramatically boost verification accuracy.
To solve this, researchers have developed mechanisms that isolate identity from style, enforce geometric consistency, and supervise models using deep perceptual losses.
Decomposing Global Structure and Local Textures
One of the most intuitive ways to preserve identity is to separate who the person is from what is being changed. Instead of forcing a single network to rewrite the entire image, architectures can split the workload.
Li introduced a framework that uses a global network to maintain overall facial structure alongside localized networks dedicated entirely to texture-heavy, attribute-sensitive regions (like the eyes, mouth, and forehead). By learning a residual face—predicting only the changes (such as age-induced wrinkles) rather than generating the face from scratch—the underlying identity remains anchored.
Input Image ───► [ Global Network ] ──────► Base Structure ──┐
│ ▼
└───────► [ Local Networks ] ─────► Texture Residuals ──► Synthesized Face
(Eyes, Mouth, Forehead)
Operating in alternative domains makes this separation even cleaner. By translating images into the wavelet (frequency) domain, Hu et al. showed that models can isolate high-frequency textural changes (aging details) from the low-frequency structural shapes that define identity. Similarly, when modifying complex, highly variable structures like hair, Xie et al. leveraged deep feature fusion to manipulate style attributes independently of the underlying facial identity.
Enforcing Reversibility and Path Consistency
When a face undergoes a massive transformation (e.g., aging 50 years), the accumulation of errors over a single large "jump" often destroys identity. To combat this, models use structural constraints based on continuity and reversibility.
Reversibility (Primal-Dual Learning)
Song et al. proposed a primal-dual conditional GAN architecture based on an elegant intuition: if a model transforms a young face to an old face, it should be able to perfectly invert that process back to the original young face. This bidirectional constraint forces the network to hide the subject's unique identity traits within the latent representation during the forward transformation, ensuring they aren't lost or overwritten by generic aging textures.
Path Consistency
Instead of treating transformations as discrete jumps, Wang et al. and Duong et al. modeled progression as a smooth, continuous sequence of short-term transitions. Taking this further, the authors of Triple-GAN introduced a triple translation loss. This enforces consistency between a direct transformation (Young $\rightarrow$ Old) and a multi-step path (Young $\rightarrow$ Middle $\rightarrow$ Old), penalizing any identity drift that occurs along the intermediate steps.
Direct Path: [ Young ] ──────────────────────────────────► [ Old ]
▲
│ (Must Match)
Multi-step Path: [ Young ] ──► [ Middle-Aged ] ──► [ Advanced ] ─┘
Latent Disentanglement and Explicit Identity Losses
Another core strategy relies on explicitly teaching the network what "identity" means, either through latent space design or external supervision.
Xiao et al. demonstrated that by decomposing latent encodings into distinct identity and attribute paths, you can exchange attribute encodings between two images (e.g., transferring a smile or glasses) while leaving the identity encodings completely untouched.
When explicit decomposition is difficult, external supervision fills the gap. Tang et al. integrated a dedicated identity-preservation module that extracts perceptual features from the intermediate layers of a pre-trained face recognition network. By minimizing the distance between these deep features before and after the transformation, the model is heavily penalized if it alters the biometric signature. This allows models to learn from non-sequential data or alternative references, such as using a secondary image to guide the style while anchoring the identity to the source (Shao et al.).
The Diffusion Paradigm
As generative modeling has shifted from GANs to diffusion models, these identity-preserving principles have adapted to iterative denoising processes. Modern frameworks like the one proposed by Ruan et al. treat complex edits (like makeup transfer and removal) as cross-domain translation problems controlled by precise, learnable embeddings, preventing stylistic changes from bleeding into core identity traits.
Furthermore, training-free diffusion approaches, such as the one developed by Jacobs et al., allow for highly customizable, multi-conditional trajectories (e.g., aging influenced by environmental factors) while maintaining a strict balance between attribute accuracy and biometric consistency.
Key Foundational Work
Recurrent Face Aging (2016) — contributed 1 concept elements to this direction.
Longitudinal Face Modeling via Temporal Deep Restricted Boltzmann Machines (2016) — contributed 1 concept elements to this direction.
Generative Adversarial Network-based Synthesis of Visible Faces from Polarimetric Thermal Faces (2017) — contributed 1 concept elements to this direction.