TL;DR
This paper introduces a novel method for automatically assessing the accuracy of portrait drawings within a learning assistant system. By employing localized Normalized Cross-Correlation (NCC) on specific facial features, the system provides detailed feedback to help beginners improve their drawing skills. The research also includes an improvement to the auxiliary line generation for noses, demonstrating enhanced drawing accuracy.
What Problem Does This Paper Address?
Learning to draw portraits is a valuable skill, but beginners often struggle with accurately capturing facial proportions and feature placement. While digital tools can assist in the drawing process, a significant gap exists in providing objective, automated feedback on the exactness of a user's drawing. Existing portrait drawing learning systems primarily focus on providing visual guidance (like auxiliary lines) but lack a mechanism to quantitatively evaluate the user's output. This paper addresses this by developing a system that can assess how closely a user's drawing matches the original reference, thereby offering targeted feedback for improvement.
Methods
The core of this work is the development of a Drawing Exactness Evaluation Method using Localized Normalized Cross-Correlation (NCC).
- Normalized Cross-Correlation (NCC): This standard image processing technique measures the similarity between two images (or image patches) by calculating the correlation of their brightness distributions. The NCC score ranges from -1 (complete dissimilarity) to 1 (complete similarity).
- Localization: To provide precise feedback, the NCC is not applied to the entire face but is localized to specific facial components (eyes, nose, mouth, eyebrows). This is achieved by:
- Keypoint Extraction: Using OpenPose, 70 facial keypoints are extracted from the reference face image.
- Bounding Box Generation: These keypoints are used to define rectangular bounding boxes around each facial component.
- Component-wise NCC: The NCC calculation is then performed only within these localized bounding boxes, comparing the user's drawing of that component to the reference.
- Auxiliary Line Improvement: The paper identifies that the nose component consistently receives low NCC scores. To address this, they propose an improvement to the auxiliary line generation for the nose by incorporating Bezier curves. This allows for a more accurate representation of the nose's complex 3D structure and boundaries, aiming to guide users more effectively.
Datasets
The paper utilizes face photos for the drawing tasks. The evaluation was conducted with drawings created by students at Okayama University, Japan. For broader evaluation, a dataset of diverse face photos from various countries, ages, and skin colors was collected.
Key Insights and Ideas
- Component-wise feedback is crucial: Applying NCC to individual facial features, rather than the entire face, provides much more granular and actionable feedback for learners.
- Nose is a challenging feature: The consistently low NCC scores for the nose highlight its inherent complexity in drawing and the limitations of basic auxiliary lines.
- Bezier curves enhance guidance: The improvement in nose auxiliary lines using Bezier curves significantly boosted NCC scores, demonstrating the value of more sophisticated geometric guidance.
- System usability is high: The System Usability Scale (SUS) results indicate that the PDLAS and the evaluation method are user-friendly and well-received by participants.
Key Results
- The proposed localized NCC method successfully assessed drawing exactness for individual facial features.
- The improvement in nose auxiliary lines led to an average increase of 222.22% in the nose NCC score.
- Evaluations with diverse face photos showed generally higher NCC scores with the improved auxiliary lines, indicating general effectiveness.
- The System Usability Scale (SUS) yielded an average score of 84, suggesting good usability.
Limitations and Future Work
- Sensitivity to local changes: NCC is sensitive to small displacements, scaling, or rotations, which can lead to significant drops in scores, especially for small features like eyes and noses.
- Insufficient shading evaluation: NCC struggles to evaluate shading, which is crucial for representing 3D structures like the nose.
- Single evaluation dimension: The NCC method primarily focuses on structural similarity and does not evaluate other artistic dimensions like creativity or emotional expression.
- Limited user base: The current study was conducted with a limited number of young students from Asia.
- No user feedback mechanism: The system currently lacks a way to provide specific suggestions for improvement beyond the NCC scores.
Future work includes addressing these limitations by:
- Exploring AI-based evaluations and incorporating feedback from professional artists.
- Expanding the system to evaluate other facial features like glasses and accessories.
- Testing the system with a wider and more diverse user population.
- Implementing a user feedback mechanism to provide more specific drawing improvement suggestions.
Summary updated