Abstract
Generating high-quality facial caricatures requires a delicate balance between artistic exaggeration and the preservation of a subject's identity. This paper introduces a novel framework centered on a "four-stream" Disentangled Feature Network (DFN) that separates a face into its core components: geometry, texture, specific organs, and artistic style. By disentangling these features, the model can manipulate them independently to produce diverse, stylized portraits. Furthermore, an Adversarial Portrait Mapping Module (APMM) is proposed to automate the generation process by mapping random noise to specific geometric and stylistic latent spaces, allowing for the creation of varied caricatures from a single input image without manual intervention.
Introduction
Caricature generation is a complex task in computer vision due to the significant domain gap between natural photographs and artistic drawings. Unlike standard image-to-image translation, caricatures involve non-linear geometric warping alongside stylistic changes. Traditional methods often struggle with unpaired data and the complex interplay between facial features, where changing the shape of a feature (like a nose) often inadvertently degrades the skin texture or background style.
To address these challenges, we propose a framework that treats caricature generation as a disentanglement and reconstruction problem. By learning independent representations for geometry and style, the system gains fine-grained control over the final output, ensuring that identity-defining "anchors" remain intact even under heavy exaggeration.
Methods
Disentangled Feature Network (DFN)
The core of the proposed system is the DFN, which processes an input image through four distinct streams to capture essential facial information:
- Landmarks: This stream captures the overall geometric structure and expression, providing the framework for exaggeration.
- Face Texture: This component handles skin tone and general color information, separate from the underlying structure.
- Prominent Organs: By focusing specifically on the eyes, nose, and mouth, the network ensures these critical identity markers remain recognizable during the warping process.
- Artistic Style: This stream extracts stylistic information (e.g., brushstrokes, color palettes) from a reference portrait.
As shown in the architecture diagram, these features are encoded into a latent space and then recombined by a decoder to reconstruct the final caricature. This modular approach prevents the "ghosting" artifacts common in entangled models.
Adversarial Portrait Mapping Module (APMM)
To enable fully automatic generation, the APMM learns to map simple Gaussian noise into the latent spaces of styles and geometric exaggerations. This allows the system to "sample" different artistic choices, generating a wide variety of results from a single natural photo. This module effectively simulates the creative decisions an artist might make regarding the degree of exaggeration and the specific drawing style.
Datasets and Evaluation
The model was trained and evaluated using WebCaricature, the largest available caricature dataset (6,042 caricatures and 5,974 natural images), and CelebA, which provided a foundation for understanding general human facial geometry.
Qualitative comparisons against state-of-the-art methods like CycleGAN, CariGAN, and WarpGAN demonstrate that the DFN produces portraits with richer textures and more plausible exaggerations. Ablation studies confirmed that removing either the "Landmark" or "Organ" streams significantly degraded visual quality and identity preservation, proving that granular disentanglement is essential for this task.
Conclusion and Future Work
The Disentangled Feature Network offers a robust solution for automated caricature generation by separating identity, geometry, and style. While the system produces high-quality results, it currently faces a "Missing Hair Problem" due to standard face detection cropping, which often ignores complex hairstyles or the top of the head. Future research will focus on improving cropping robustness and exploring even more granular disentanglement, such as separating environmental lighting from intrinsic facial texture.
Summary updated