CG-GAN: An Interactive Evolutionary GAN-Based Approach for Facial Composite Generation

Nicola Zaltron, Luisa Zurlo, Sebastian Risi

Abstract

Facial composite generation is a critical forensic task that traditionally relies on the cognitively demanding process of mental recall. This paper presents CG-GAN (Composite Generating GAN), an interactive tool that leverages the photorealistic capabilities of Progressive Growing GANs (pg-GAN) and Interactive Evolutionary Computation (IEC) to facilitate facial generation through recognition. By incorporating "Smart Locks" to manage feature entanglement in the latent space, CG-GAN enables non-experts to iteratively refine facial likenesses. Evaluation results show a 66.07% correct identification rate in suspect lineups, demonstrating the system's effectiveness in producing high-quality, recognizable facial composites.

Introduction

The creation of facial composites—commonly known as police sketches—is hindered by the human brain's limited capacity for feature-based recall. Psychological research indicates that humans are significantly better at holistic recognition than at describing individual facial components. Existing digital tools often fail to bridge this gap, being either too rigid (constructive) or too difficult to control (holistic). CG-GAN addresses these limitations by providing a photorealistic, evolutionary framework that prioritizes user recognition while maintaining granular control over specific facial features.

Methodology

CG-GAN integrates three primary technologies:

  1. Generative Foundation: The system utilizes a pg-GAN pre-trained on the CelebFaces Attributes (CelebA) dataset, capable of producing high-resolution, realistic human faces from latent vectors.
  2. Interactive Evolution: Users engage in Latent Variable Evolution (LVE), where they select the most accurate faces from a generated population. These selections serve as "parents" for subsequent generations, which are produced through crossover and mutation.
  3. Feature Disentanglement: To provide control, the system maps specific attributes (e.g., age, gender) to axes in the latent space. "Feature Locks" allow users to preserve specific traits, while "Smart Locks" automatically identify and freeze correlated attributes to prevent unintended alterations during the evolutionary process.
Examples of photorealistic facial composites generated using the CG-GAN interactive evolutionary process.

Results

The efficacy of CG-GAN was tested with 13 constructors and 55 evaluators. The primary metric was the recognition rate in a five-person suspect lineup. Evaluators correctly identified the target face in 66.07% of trials. Furthermore, nearly half of the generated composites (43.75%) achieved a recognition rate of at least 75%.

A notable success was the generation of a likeness for actor Chris Hemsworth. Through 46 generations of evolution, a user was able to produce a highly accurate composite in approximately 35 minutes, illustrating the system's efficiency and accessibility for non-experts.

Comparison between the target image of actor Chris Hemsworth (left) and the final composite generated by CG-GAN (right).

Discussion and Conclusion

CG-GAN successfully shifts the burden of facial composite generation from recall to recognition. While the system is currently constrained by the biases and attribute limitations of the CelebA dataset, it represents a significant advancement in forensic tools. Future developments aim to expand the available attribute axes and explore multi-witness consensus models to further enhance the accuracy of generated composites.

Summary updated

Abstract

Facial composites are graphical representations of an eyewitness's memory of a face. Many digital systems are available for the creation of such composites but are either unable to reproduce features unless previously designed or do not allow holistic changes to the image. In this paper, we improve the efficiency of composite creation by removing the reliance on expert knowledge and letting the system learn to represent faces from examples. The novel approach, Composite Generating GAN (CG-GAN), applies generative and evolutionary computation to allow casual users to easily create facial composites. Specifically, CG-GAN utilizes the generator network of a pg-GAN to create high-resolution human faces. Users are provided with several functions to interactively breed and edit faces. CG-GAN offers a novel way of generating and handling static and animated photo-realistic facial composites, with the possibility of combining multiple representations of the same perpetrator, generated by different eyewitnesses.
BibTeX
@inproceedings{zaltron2020cg,
  author = {Nicola Zaltron and Luisa Zurlo and Sebastian Risi},
  title = {{CG-GAN: An Interactive Evolutionary GAN-Based Approach for Facial Composite Generation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i03.5637},
}
Figure