Abstract
This paper introduces a novel framework employing case-based reasoning (CBR) to capture the personal styles of individual artists and automatically generate facial sketches from photographs. By iteratively retrieving and adapting "cases"—pairs of source facial photos and hand-drawn sketches—the system transforms stylization into a process of iterative refinement. Two learned models, a fitness evaluation model for case retrieval and a parameter estimation model for case adaptation, guide the synthesis. Experimental results demonstrate that this CBR-based method effectively reproduces both stroke-level stylization and geometric exaggeration, outperforming state-of-the-art example-based algorithms and commercial software.
Introduction
Generating artistic sketches from photographs, a subfield of Image-Based Artistic Rendering (IB-AR), typically relies on Example-Based Artistic Rendering (EBAR). Traditional EBAR methods are often either model-based, which use parametric rules that may lose subtle artistic nuances, or model-free, which reuse patches from examples but are limited by the specific data provided.
Human artists do not draw by applying global filters or blindly copying patches; they work progressively and often incorporate personalized geometric exaggerations. Existing algorithms struggle to replicate this multi-step, highly personalized process. This research addresses this gap by teaching a system to emulate the specific stylistic and exaggerative habits of individual artists through a Case-Based Reasoning framework.
Methodology
The proposed approach frames sketch generation as a CBR problem, where new sketches are synthesized by retrieving and adapting similar past cases. A "case" consists of a source photo, its corresponding hand-drawn sketch, a "neutralized" sketch (with exaggeration removed), and extracted facial landmarks.
The pipeline operates iteratively region-by-region, based on 67 facial feature points:
- Fitness Evaluation (FE) Model: This model predicts which candidate case will best help synthesize the final desired sketch by hypothesizing the "ground truth" in the artist's mind.
- Parameter Estimation (PE) Model: This model automatically calculates optimal blending weights to fuse the retrieved case with the current interim sketch.
- Exaggeration Field (EF): To handle caricature-like features, the authors decouple texture from geometry. A linear system learns a 2D vector field representing an artist's specific warping style, which is applied as a post-processing step to the final sketch.
Experimental Setup
The model was evaluated using three datasets, each containing 49 photo-sketch pairs:
- $Π_1$ (CUHK Student Dataset): A standard benchmark with realistic sketches.
- $Π_2$ (Professional Artist): Sketches by an artist with over 20 years of experience, featuring distinct personal exaggeration.
- $Π_3$ (Freelance Illustrator): Sketches by a digital art PhD student, featuring a highly stylized caricature approach.
Results and Discussion
The CBR method demonstrates high data efficiency, successfully capturing an artist's style with fewer than 50 examples. By decoupling style and geometry, the system avoids confusing texture with structural exaggeration.
A large-scale user study involving over 2,600 participants revealed that 62% of respondents preferred the sketches generated by this CBR method over those from state-of-the-art peer algorithms. Qualitatively, the method preserves crucial stylistic details, such as nasolabial folds, and accurately reproduces the geometric exaggerations characteristic of specific artists, which competing methods and commercial software often fail to capture. Furthermore, the algorithm is computationally efficient, generating a sketch in approximately one minute on a standard CPU.
Conclusion and Future Work
While the system excels at facial features, it currently cannot synthesize human hair due to the lack of consistent anatomical landmarks. Future research may explore cross-modal face matching or manual keypoint indication to address this. Additionally, parallelizing the regional models could significantly reduce training time, which currently stands at approximately eight hours. Overall, the CBR framework provides a robust and efficient means of mimicking personalized artistic styles in facial portraiture.
Summary updated