Animated construction of line drawings

Hongbo Fu, Shizhe Zhou, Ligang Liu, Niloy J. Mitra

TL;DR

This paper introduces a method to automatically generate animated line drawings from static images, mimicking how an artist would draw them. By analyzing drawing cognition principles and geometric cues, the system infers a plausible stroke order and direction, enabling applications like video scribing and gesture drawing synthesis.

What Problem Does This Paper Address?

Creating animated line drawings, often used for storytelling or instructional videos (video scribing), traditionally requires artists to meticulously record their drawing process. This is time-consuming and limits the reuse of existing static line art. The paper addresses the challenge of automatically inferring a visually plausible drawing order and direction from a static line drawing, making it possible to animate existing artwork without manual recording.

Methods

The core of the method involves several key steps:

  • Hierarchy Reconstruction: The system first builds a coarse-to-fine hierarchical representation of the input line drawing. This is achieved through an iterative simplification process that alternates between clustering lines and reducing their shape complexity. This hierarchy serves as a level-by-level drawing order, reflecting how drawers often start with a rough sketch and add details.
  • Stroke Ordering: Based on principles from drawing cognition, the system orders the "significant" lines (those forming the overall structure) by formulating it as a global energy minimization problem. This problem is then approximated by finding a Hamiltonian path on a directed graph. The graph's nodes represent lines, and edges encode proximity and anchoring guidelines. Anchoring constraints (e.g., drawing a T-junction crossbar before the stem) are handled by directing graph edges.
  • Stroke Direction Determination: Two schemes are proposed: one based on mechanical movement and preferred directions (downward/left-to-right for right-handers), and another based on motor convenience and economy, prioritizing strokes whose endpoints are closer to the previously drawn line.

Datasets

The paper does not explicitly mention training datasets. For evaluation, they used a range of input line drawings with varying complexity and structure, including examples from their own work and those created by artists. They also conducted a user study involving 19 drawing examples.

Key Insights and Ideas

  • Drawing Order Reflects Cognitive Principles: The paper highlights that drawing order isn't arbitrary but is guided by cognitive principles like hierarchical organization (coarse-to-fine) and geometric cues (proximity, collinearity, anchoring).
  • Anchoring is Crucial: Explicitly modeling anchoring constraints (e.g., T-junctions) significantly improves the plausibility of the inferred drawing order.
  • Hamiltonian Path for Ordering: Formulating the stroke ordering problem as finding a Hamiltonian path on a graph encoding geometric relationships is an effective way to find a globally optimal (or near-optimal) drawing sequence.
  • Plausibility Over Exact Recovery: The goal is to find a plausible drawing order, not necessarily the exact one used by an artist, acknowledging the subjective nature of drawing.

Key Results

  • User Study I: The proposed method significantly outperformed other baseline methods (random ordering, longest-first, nearest-first, smoothest-first) in terms of visual plausibility, as judged by human participants.
  • User Study II: A second user study, comparing the system's output to artist-drawn orders, showed that the system's results were often difficult to distinguish from human-drawn orders, with some examples being statistically indistinguishable from artist-generated orders.
  • T-junction Constraints: Incorporating T-junction constraints led to more plausible orderings and reduced computation time.
  • Applications: The system successfully demonstrated applications in gesture drawing synthesis and animated storytelling (video scribing).

Limitations and Future Work

  • Limitations:
    • The current implementation does not explicitly detect or encode symmetry or similarity patterns, although these are implicitly handled to some extent by other guidelines.
  • The system assumes relatively clean input drawings, free of hatching strokes.
  • It lacks a quantitative understanding of algorithmic limits or ambiguities.
  • For complex scenes with predefined semantic orders (e.g., human figures), the system might not always match human intuition.
  • Future Work:
    • Explore incorporating additional cues like color information, depth, or segmentation from 3D models to improve line ordering.
  • Investigate adaptive rendering styles and stylization techniques to enhance the animated output.
  • Develop methods to handle more complex inputs, including those with hatching or stylized sketches.
  • Explore interactive tools to allow users to guide the ordering process or specify semantic importance.

Summary updated

Abstract

Revealing the sketching sequence of a line drawing can be visually intriguing and used for video-based storytelling. Typically this is enabled based on tedious recording of artists' drawing process. We demonstrate that it is often possible to estimate a reasonable drawing order from a static line drawing with clearly defined shape geometry, which looks plausible to a human viewer. We map the key principles of drawing order from drawing cognition to computational procedures in our framework. Our system produces plausible animated constructions of input line drawings, with no or little user intervention. We test our algorithm on a range of input sketches, with varying degree of complexity and structure, and evaluate the results via a user study. We also present applications to gesture drawing synthesis and drawing animation creation especially in the context of video scribing.
BibTeX
@article{fu2011animated,
  author = {Hongbo Fu and Shizhe Zhou and Ligang Liu and Niloy J. Mitra},
  title = {{Animated construction of line drawings}},
  journal = {ACM Trans. Graphics},
  year = {2011},
  doi = {10.1145/2070781.2024167},
}
Figure