TL;DR
This paper introduces a novel method for generating automatic caricatures by considering not just how much a facial feature deviates from the average, but also how variable that feature is across a population. By incorporating feature variance, their approach produces more likeness-preserving and aesthetically pleasing caricatures compared to previous methods that solely relied on deviation from the mean.
What Problem Does This Paper Address?
Creating realistic and effective caricatures automatically has been a long-standing challenge. Traditional methods, like the "Exaggerating the Difference from the Mean" (EDFM) approach, focus on amplifying features that are statistically different from an average face. However, this can lead to unnatural results because it doesn't account for the fact that some features are naturally more variable than others. For instance, a slight change in eye size might be highly noticeable and distinctive, while the same change in face width might be less impactful because face width naturally varies more across individuals. The paper addresses this by proposing a more nuanced approach that considers feature variance, aiming to create caricatures that are both exaggerated and retain likeness.
Methods
The authors propose a method that builds upon the EDFM concept by incorporating feature variance. The core of their approach involves:
- Learning a "Face Space": They use Non-negative Matrix Factorization (NMF) on a dataset of 300 frontal faces (each with 94 hand-labeled feature points) to learn a "face space." This space is defined by basis vectors representing abstract facial features and their distributions (mean and standard deviation).
- Feature Normalization and Exaggeration: For a new face, its shape is represented within this learned face space. The key innovation is how features are exaggerated. Instead of simply scaling the difference from the mean, they scale the difference by a factor
konly for dimensions where the feature's deviation from the mean is significantly larger than its standard deviation (specifically, more than 2 standard deviations away). Features that are not significantly different from the mean are scaled by a factor of 1, and the residual (features not well-represented by the face space) is also exaggerated but at a reduced scale. - Image Warping and Rendering: The exaggerated shape is then used to warp the original image, creating a photorealistic caricature. For stylized line-drawing caricatures, they employ a Non-Photorealistic Rendering (NPR) technique that leverages a generic face model to guide detail placement and omission.
Datasets
- FERET database: Used to train the face space model. 300 frontal faces from this database were selected, and 94 feature points were hand-labeled on each.
Key Insights and Ideas
- Variance Matters for Distinctiveness: The most significant insight is that a feature's distinctiveness isn't solely determined by its deviation from the mean, but crucially by its variance within the population. Features with higher variance can tolerate larger deviations before appearing unusual, while features with lower variance are more sensitive to even small changes.
- EDFM's Limitation: The paper clearly demonstrates the drawback of the standard EDFM approach, showing how it can over-emphasize features with high natural variance (like face width), leading to a loss of likeness.
- A More Nuanced Exaggeration: By selectively exaggerating features based on their statistical significance relative to their variance, the proposed method achieves a better balance between exaggeration and likeness.
Key Results
The paper presents qualitative results demonstrating the effectiveness of their method. Figure 3 shows a comparison where their method produces a more natural caricature by emphasizing the eyes (which are more distinctive) without over-exaggerating the face width (which has higher variance). Figure 2 showcases both photorealistic and stylized line-drawing caricatures generated by their system.
Limitations and Future Work
- Limitations:
- The paper acknowledges that the Active Appearance Model (AAM) might fail in certain cases (e.g., non-frontal faces, complex lighting), requiring manual placement of shape points, which takes several minutes.
- The NPR aspect is mentioned as not being the core focus, suggesting that further details and optimizations in this area are not discussed.
- Future Work: Not explicitly detailed in the provided text, but the NPR section implies that further development in stylized rendering could be a direction. The core method itself is presented as an improvement, suggesting its application to a wider range of facial poses and expressions could be a natural next step.
Summary updated