Identity-Preserving Aging of Face Images via Latent Diffusion Models

Sudipta Banerjee, Govind Mittal, Ameya Joshi, Chinmay Hegde, Nasir D. Memon

Abstract

The performance of automated face recognition systems is inevitably impacted by the facial aging process. However, high quality datasets of individuals collected over several years are typically small in scale. In this work, we propose, train, and validate the use of latent text-to-image diffusion models for synthetically aging and de-aging face images. Our models succeed with few-shot training, and have the added benefit of being controllable via intuitive textual prompting. We observe high degrees of visual realism in the generated images while maintaining biometric fidelity measured by commonly used metrics. We evaluate our method on two benchmark datasets (CelebA and AgeDB) and observe significant reduction (~44%) in the False Non-Match Rate compared to existing state-of the-art baselines.

Introduction

Automated face recognition (AFR) systems are increasingly integrated into security and personal technology, yet they remain notoriously sensitive to the natural process of facial aging. As individuals age, their features undergo complex transformations influenced by genetics, lifestyle, and environment. These changes often lead to system failures when comparing a current probe image against an older enrollment image.

The primary obstacle in developing age-invariant AFR systems is the scarcity of "longitudinal data"—large-scale, high-quality datasets tracking the same individuals over multiple decades. While Generative Adversarial Networks (GANs) have been employed to simulate aging, they frequently encounter a fundamental trade-off: they either achieve realistic aging at the cost of the subject's identity or preserve the identity while failing to produce convincing age-related changes. Furthermore, GANs typically require massive datasets to achieve stability. This research addresses these challenges by leveraging Latent Diffusion Models (LDMs) to generate high-fidelity, identity-preserving aged images with minimal training data.

Methodology

The proposed framework builds upon DreamBooth, a technique designed to fine-tune text-to-image diffusion models to recognize specific subjects using a "few-shot" approach (approximately 20 images). To adapt this for biometric aging, the researchers introduced three critical components:

1. Identity-Preserving Losses

To ensure that the generated "aged" version of a person remains biometrically identical to the original, the model integrates two specialized loss functions:

  • Biometric Loss: Utilizes a pre-trained VGGFace extractor to penalize deviations from the subject's unique facial "fingerprint."
  • Contrastive Loss: Based on the SimCLR framework, this loss ensures that the model maintains the distinctiveness of the individual's features across different generated iterations.

2. Age-Aware Regularization

To prevent the model from losing the general concept of human aging while focusing on a specific identity, the authors utilized an auxiliary regularization set. This set consists of approximately 600 image-caption pairs from the CelebA-Dialog dataset, categorized into six distinct age groups: child, teenager, young adult, middle-aged, elderly, and old.

3. Text-Guided Inference

One of the primary advantages of using an LDM backbone is controllability. By associating a unique identifier (a "rare token" such as sks) with the subject during training, users can generate aged images through simple natural language prompts, such as: "photo of a sks person as elderly."

Experimental Results

The method was evaluated using benchmark datasets including CelebA, AgeDB, and the Cross-Age Celebrity Dataset (CACD).

Biometric Fidelity and Performance

The LDM-based approach significantly outperformed existing baselines like IPCGAN and AttGAN. On the CelebA dataset, the model achieved a 44% average reduction in the False Non-Match Rate (FNMR) compared to the previous state-of-the-art, Talk-to-Edit. This suggests that the synthetic images are not only visually convincing but also biometrically accurate enough to be used for training and "hardening" AFR systems against real-world aging.

Qualitative Realism and User Study

Qualitatively, the generated images exhibited higher resolution and more natural textural details—such as realistic wrinkles and skin tone shifts—than GAN-based approaches, which often suffer from blurring or ghostly artifacts. A user study involving 26 volunteers confirmed these findings; participants correctly identified the original subjects in the generated images with high accuracy and correctly categorized the intended age groups (e.g., 99.6% accuracy for the 'child' group).

Discussion and Limitations

While the model represents a significant leap forward, the authors identified specific failure cases. Extreme de-aging remains a challenge; for example, the model struggles to generate a convincing "child" version of an individual if the training set consists only of elderly images or subjects with heavy facial hair. Additionally, the model inherits certain demographic biases from the underlying pre-trained Stable Diffusion model, such as a tendency to add makeup when aging women compared to men.

Conclusion

This work demonstrates that Latent Diffusion Models can effectively bridge the longitudinal data gap in face recognition research. By combining few-shot learning with biometric and contrastive losses, the researchers have created a controllable, high-fidelity system for facial aging. Future work will focus on zero-shot age editing—eliminating the need for fine-tuning images entirely—and utilizing composable diffusion models for more granular control over specific facial features.

Summary updated

Abstract

The performance of automated face recognition systems is inevitably impacted by the facial aging process. However, high quality datasets of individuals collected over several years are typically small in scale. In this work, we propose, train, and validate the use of latent text-to-image diffusion models for synthetically aging and de-aging face images. Our models succeed with few-shot training, and have the added benefit of being controllable via intuitive textual prompting. We observe high degrees of visual realism in the generated images while maintaining biometric fidelity measured by commonly used metrics. We evaluate our method on two benchmark datasets (CelebA and AgeDB) and observe significant reduction (~44%) in the False Non-Match Rate compared to existing state-of the-art baselines.
BibTeX
@inproceedings{banerjee2023identity,
  author = {Sudipta Banerjee and Govind Mittal and Ameya Joshi and Chinmay Hegde and Nasir D. Memon},
  title = {{Identity-Preserving Aging of Face Images via Latent Diffusion Models}},
  booktitle = {IJCB},
  year = {2023},
  url = {https://openreview.net/forum?id=wEXKmj7pfH},
}
Figure