Face Aging via Diffusion-based Editing

Xiangyi Chen, Stéphane Lathuilière

TL;DR

FADING is a new framework that repurposes large-scale text-to-image diffusion models for high-fidelity face aging. By leveraging the massive visual priors of models like Stable Diffusion, it overcomes the data scarcity and identity-loss issues that plague traditional GAN-based aging methods, enabling realistic transformations across the entire human lifespan.

What Problem Does This Paper Address?

Digital face aging—the art of simulating how a person looks as they grow older or younger—has traditionally been the domain of Generative Adversarial Networks (GANs). However, GANs face two major hurdles:

  1. The Data Bottleneck: They rely on specific facial datasets (like FFHQ or CelebA) which are often biased toward younger, "studio-quality" faces. Consequently, GANs struggle with extreme age gaps (e.g., turning an infant into an 80-year-old) and rare cases like side profiles or faces with heavy accessories.
  2. The Inversion Gap: Mapping a real-world photo into a GAN’s "latent space" is notoriously difficult. Often, the reconstructed image loses the person's unique identity or introduces strange artifacts.

The authors of this paper introduce FADING (Face Aging via DIffusion-based editiNG) to solve these issues by tapping into the "common sense" knowledge of large-scale diffusion models.

Methods

The FADING pipeline consists of three main stages:

  • Specialization (Age-Aware Fine-Tuning): Rather than using a generic model, the authors fine-tune a Latent Diffusion Model (LDM) using a "double-prompt" scheme. They train the model with pairs of prompts: a specific one ("photo of a [age] year old person") and a generic one ("photo of a person"). This helps the model disentangle age-related features from identity-related ones.
  • Null-Text Inversion: To edit a real photo, the model must first "understand" how to perfectly reconstruct it. The authors use Null-Text Inversion, which optimizes the unconditional text embeddings to ensure the diffusion process can recreate the input image with near-perfect fidelity.
  • Localized Editing via Attention Control: During the aging process, the model swaps the original age token for a target age (e.g., "30" to "70"). To ensure the person's structure remains the same, they use attention control. By injecting the spatial attention maps from the original image into the new generation, they ensure that wrinkles and grey hair appear in the right places without changing the background, clothing, or face shape.

Datasets

The researchers utilized two primary high-resolution benchmarks:

  • FFHQ-Aging: An extension of the NVIDIA FFHQ dataset containing 70,000 images labeled into 10 age groups (from 0-2 to 70+ years old). This was used for both fine-tuning and evaluation.
  • CelebA-HQ: A dataset of 30,000 celebrity images, used primarily for evaluating the model's ability to generalize to different identities and "young-to-old" transformations.

Key Insights and Ideas

  • The "Internet Prior" Advantage: Unlike GANs trained on 70k faces, diffusion models trained on billions of images already "know" what aging looks like across diverse ethnicities, lighting conditions, and poses. FADING simply "specializes" this existing knowledge.
  • Disentanglement through Prompts: The "double-prompt" strategy is a clever way to force the model to learn that "age" is a variable attribute, while "person-ness" (identity) is a constant.
  • Robustness to "Rare" Cases: Because the underlying model has seen more than just faces, FADING handles sunglasses, hats, and extreme head tilts much better than previous SOTA (State-of-the-Art) methods, which often hallucinate or distort these features.

Key Results

FADING demonstrates significant improvements over previous leaders like CUSP and HRFAE:

  • Aging Accuracy: On the FFHQ-Aging dataset, FADING achieved a lower Mean Absolute Error (MAE) in predicted age compared to CUSP (9.16 vs 12.63).
  • Identity Preservation: The model achieved the highest gender preservation rate (98.4% on CelebA-HQ), indicating it doesn't accidentally change a person's fundamental attributes while aging them.
  • Visual Quality: Using the Kernel-Inception Distance (KID) metric, FADING showed nearly an order of magnitude improvement in image realism for most age groups.
  • Out-of-Distribution Success: The authors successfully aged "difficult" images, including a photo of Emmanuel Macron and a movie character with heavy makeup (The Joker), proving the model's flexibility.

Limitations and Future Work

The authors noted a slight entanglement between age and skin tone. In some cases, aging a person also slightly altered their skin tone—a bias likely inherited from the pre-trained Stable Diffusion model's training data.

Future Work: The team plans to extend the "enhanced prompt" strategy to preserve other age-agnostic attributes. For example, by integrating a glasses classifier, they could automatically include "wearing glasses" in the prompt to ensure eyewear is perfectly preserved or realistically aged along with the face.

Summary updated

Abstract

In this paper, we address the problem of face aging: generating past or future facial images by incorporating age-related changes to the given face. Previous aging methods rely solely on human facial image datasets and are thus constrained by their inherent scale and bias. This restricts their application to a limited generatable age range and the inability to handle large age gaps. We propose FADING, a novel approach to address Face Aging via DIffusion-based editiNG. We go beyond existing methods by leveraging the rich prior of large-scale language-image diffusion models. First, we specialize a pre-trained diffusion model for the task of face age editing by using an age-aware fine-tuning scheme. Next, we invert the input image to latent noise and obtain optimized null text embeddings. Finally, we perform text-guided local age editing via attention control. The quantitative and qualitative analyses demonstrate that our method outperforms existing approaches with respect to aging accuracy, attribute preservation, and aging quality.
BibTeX
@inproceedings{chen2023face,
  author = {Xiangyi Chen and Stéphane Lathuilière},
  title = {{Face Aging via Diffusion-based Editing}},
  booktitle = {BMVC},
  year = {2023},
}

Loading related content...

Figure