TL;DR
This paper introduces DrawAFriend, a mobile game designed to crowdsource a large dataset of drawings. They leverage this data to build a "stroke consensus" model that enables real-time, invisible stroke correction, making drawing on touch devices feel more natural and less error-prone.
What Problem Does This Paper Address?
Drawing is a fundamental form of human expression and communication, yet analyzing and assisting this process is hampered by a lack of large-scale, detailed drawing data. Existing datasets are often rasterized, lack stroke-level information, and have minimal metadata. Furthermore, drawing on touch devices like smartphones presents challenges due to the "fat finger" problem, where imprecise input leads to strokes that don't match the user's intent. This paper aims to solve both the data scarcity problem and the touch-input inaccuracy problem by creating a system that collects drawing data and uses it to provide real-time drawing assistance.
Methods
The core of the paper lies in two main components: data collection and stroke correction.
- Data Collection (DrawAFriend Game): They developed a mobile game, DrawAFriend, where players trace celebrity photos. This tracing paradigm ensures that drawings of the same subject are pre-aligned, making it easier to compare and aggregate strokes. The game intrinsically motivates users through a "guess the drawing" mechanic, similar to Pictionary or Draw Something, leading to a large volume of drawings with associated metadata (strokes, timing, guesses).
- Stroke Correction:
- Artistic Consensus Model: They build a spatially varying model of artistic consensus at the stroke level. This involves finding the "average" or "typical" stroke for a given area across many drawings of the same subject.
- Correction Vector Field: This consensus is represented as a correction vector field. For any point on a stroke being drawn, this field indicates the direction and magnitude to move that point to align with the consensus. They use a modified mean shift algorithm to robustly find this consensus, avoiding outliers.
- Real-time Correction: When a user draws a stroke, the system samples the correction vector field and applies a correction in real-time. Crucially, this correction is designed to be "invisible" to the user, meaning it doesn't require new interaction paradigms. They achieve this by solving an over-constrained linear system that balances faithfulness to the correction vector field with preserving the original stroke's shape and style. A continuation method with decreasing stiffness is used to fine-tune the correction.
Datasets
- DrawAFriend Dataset: This is a newly introduced, large-scale dataset of drawings collected through the DrawAFriend game. It contains stroke-level data, timing information, and user interaction data (guesses). The paper focuses on a subset of celebrity drawings for training and evaluation.
- Existing Web Databases/Image Corpora: Mentioned as a contrast to their approach, but not directly used for training their model.
Key Insights and Ideas
- Crowdsourcing for Detailed Drawing Data: A mobile game can be a highly effective and scalable platform for collecting rich, stroke-level drawing data, overcoming the limitations of traditional image datasets.
- "Artistic Consensus" as Intent: The paper posits that the consensus of strokes across many drawings of the same subject represents the user's underlying artistic intent, even if their physical execution is imprecise.
- Invisible Stroke Correction: Real-time stroke correction can be seamlessly integrated into drawing interfaces without disrupting the user's workflow, making touch-based drawing more forgiving and enjoyable.
- Preserving Artistic Style: The correction method is designed not just to align strokes with a consensus but also to maintain the stylistic nuances of the original drawing, preventing a "homogenized" look.
- Drawing Quality and User Effort: Auto-correction allows users to be less precise in their strokes, reducing the need for undoes and making the drawing process more efficient, without necessarily sacrificing the final drawing's recognizability.
Key Results
- Data Collection Scale: DrawAFriend generated over 1,500 images per day in its first week, quickly amassing a substantial dataset.
- Undo Rate Reduction: The study showed that with auto-correction enabled, users had significantly lower undo rates, especially for more careful drawers.
- Stroke Precision: Artists using auto-correction did not need to draw as accurately, as evidenced by a significant increase in the "distance" between their strokes and the consensus.
- Recognition Rate: The auto-correction did not significantly alter the recognizability of the drawings, suggesting that the final quality is maintained.
- Convergence: The correction vector field converges to a stable state with approximately 25 drawings, though more drawings continue to improve it.
Limitations and Future Work
- Manual Filtering of Initial Data: The initial correction vector fields were primed with manually filtered drawings, which is not scalable. Future work includes developing automated filtering methods.
- Isotropic Correction Field: The current correction vector field is isotropic (uniform in all directions). Future work aims to develop anisotropic fields that consider stroke orientation for more nuanced correction.
- Single Consensus Model: The system currently relies on a single consensus model. Future work suggests identifying and clustering different artistic "styles" to provide style-specific corrections.
- Generalization: Extending the correction method to be more general, perhaps using image-analogy-like approaches, is a future direction.
- Further Data Analysis: The paper highlights the potential of the DrawAFriend dataset for other research, such as identifying salient strokes for recognition, understanding drawing errors, and predicting user actions.
Summary updated