StyleGene: Crossover and Mutation of Region-level Facial Genes for Kinship Face Synthesis

Hao Li, Xianxu Hou, Zepeng Huang, Linlin ShenCVPR 2023 · OpenReview

Abstract

High-fidelity kinship face synthesis has many potential applications, such as kinship verification, missing child identification, and social media analysis. However, it is challenging to synthesize high-quality descendant faces with genetic relations due to the lack of large-scale, high-quality annotated kinship data. This paper proposes the RFG (Region-level Facial Gene) extraction framework to address this issue. We utilize an Image-based Gene Encoder (IGE), a Latent-based Gene Encoder (LGE), and a Gene Decoder to learn the RFGs of a face image and their relationship to the StyleGAN2 latent space. By employing cycle-like losses, the framework can be trained using only individual face images, eliminating the need for paired kinship data. A crossover and mutation module, supported by a diverse "Gene Pool," simulates biological inheritance to generate realistic and varied descendant faces. Experiments on FIW, TSKinFace, and FF-Databases demonstrate that StyleGene significantly outperforms existing state-of-the-art methods in both quality and diversity.

Introduction

Kinship face synthesis—predicting the appearance of a child based on their parents—is a complex task with significant real-world utility. However, the field faces a major "data bottleneck": high-quality, large-scale datasets of parent-child image pairs are extremely rare. Existing methods often struggle with overfitting, producing blurry "average" faces, or lack fine-grained control over trait inheritance.

High-fidelity synthesis of a child's face using StyleGene.

StyleGene addresses these challenges by mimicking biological genetics within the StyleGAN2 latent space. By decomposing faces into regional "genes," the model can simulate the way specific traits are inherited independently, producing high-fidelity results without requiring paired training data.

Methodology

The StyleGene framework abstracts the facial genetic process into three core components:

Region-level Facial Genes (RFGs)

Instead of representing a face as a single latent vector, the model decomposes it into 34 independent semantic regions (e.g., eyes, nose, mouth). This disentanglement allows for precise control over inheritance.

Dual-Encoder Architecture

  • Image-based Gene Encoder (IGE): Extracts RFGs from images using facial parsing masks during the training phase.
  • Latent-based Gene Encoder (LGE): Learns to map StyleGAN2’s latent space ($w$) directly to RFGs, enabling gene extraction during inference without requiring masks.
  • Gene Decoder: Maps the assembled "child genes" back into the $w+$ latent space of a pre-trained StyleGAN2 generator.

Crossover and Mutation

Inheritance is simulated through two primary operations:

  1. Crossover: A weighted linear combination of parental RFGs is performed for each of the 34 regions.
  2. Mutation via Gene Pool: To avoid the "averaging" effect common in morphing, the model randomly injects genes from a "Gene Pool" (built from 70,000 FFHQ images). This introduces realistic variations not present in either parent, mirroring natural genetic diversity.

Experiments and Results

StyleGene was evaluated against state-of-the-art baselines like StyleDNA and ChildPredictor using standard kinship datasets (FIW, TSKinFace, and FF-Database).

Kinship face synthesis result showing parents (top) and the generated child (bottom).

Key Performance Metrics

  • Kinship Verification Accuracy: On the TSKinFace dataset, StyleGene achieved an accuracy of 81.74%, significantly higher than StyleDNA's 53.15%. This indicates that the generated children are much more recognizable as belonging to their parents.
  • Diversity (LPIPS): The model produced highly diverse "siblings." On the FIW dataset, StyleGene’s diversity score was 0.3279, nearly doubling that of ChildPredictor (0.1750).
  • Visual Quality: Qualitative results demonstrate superior handling of hair, skin texture, and lighting, leveraging the robust generative capabilities of the StyleGAN2 backbone.

Conclusion

StyleGene introduces a novel, unsupervised approach to kinship face synthesis. By leveraging region-level facial genes and biological heuristics like crossover and mutation, it overcomes the limitations of small paired datasets. The resulting framework produces high-fidelity, diverse descendant faces that maintain strong genetic resemblances to their parents. Future work may explore more complex, non-linear genetic rules to further refine the simulation of trait inheritance.

Summary updated

Abstract

High-fidelity kinship face synthesis has many potential applications, such as kinship verification, missing child identification, and social media analysis. However, it is challenging to synthesize high-quality descendant faces with genetic relations due to the lack of large-scale, high-quality annotated kinship data. This paper proposes RFG (Region-level Facial Gene) extraction framework to address this issue. We propose to use IGE (Image-based Gene Encoder), LGE (Latent-based Gene Encoder) and Gene Decoder to learn the RFGs of a given face image, and the relationships between RFGs and the latent space of Style-GAN2. As cycle-like losses are designed to measure the $\mathcal{L}_{2}$ distances between the output of Gene Decoder and image encoder, and that between the output of LGE and IGE, only face images are required to train our framework, i.e. no paired kinship face data is required. Based upon the proposed RFGs, a crossover and mutation module is further designed to inherit the facial parts of parents. A Gene Pool has also been used to introduce the variations into the mutation of RFGs. The diversity of the faces of descendants can thus be significantly increased. Qualitative, quantitative, and subjective experiments on FIW, TSKinFace, and FF-Databases clearly show that the quality and diversity of kinship faces generated by our approach are much better than the existing state-of-the-art methods.
BibTeX
@inproceedings{li2023stylegene,
  author = {Hao Li and Xianxu Hou and Zepeng Huang and Linlin Shen},
  title = {{StyleGene: Crossover and Mutation of Region-level Facial Genes for Kinship Face Synthesis}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  url = {https://openreview.net/forum?id=UKknhHnv9h},
}
Figure