Curve Synthesis from Learned Refinement Models

Saul Simhon, Gregory Dudek

Introduction

Professional digital illustration often requires a level of precision and artistic flair that is difficult for non-experts to achieve. While standard "beautification" tools can smooth out shaky lines, they typically lack an understanding of the artist's intent. A simple vertical stroke might be intended as a tree trunk in one context or a building wall in another. This paper presents a hierarchical statistical framework designed to bridge this gap, transforming rough, hand-drawn sketches into polished, stylized illustrations by leveraging learned context and artistic models.

The Hierarchical Framework

The core innovation of this research is the Two-Level Hierarchical Hidden Markov Model (HHMM). This architecture allows the system to interpret drawings at two distinct scales: the individual curve and the overall scene.

The Curve Level

At the bottom level, individual HMMs are trained on specific ensembles of strokes, such as various leaf shapes or geometric primitives. In this model, the user's rough input is treated as the observable state—a noisy version of an intended "Platonic ideal." The system's goal is to recover the hidden state, which represents the refined artistic stroke. This state space is multi-dimensional, encompassing not just geometry (tangent angles) but also artistic attributes like line thickness and color.

The Scene Level

The top level of the hierarchy encodes the "grammar" of a drawing. It uses a graph-based structure to define allowable sequences and spatial relationships between different elements. For instance, a "landscape" scene might dictate that "clouds" are likely to appear above "grass" or "water."

Scene-level grammar graph defining spatial and sequential relationships between drawing elements.

This contextual awareness allows the system to perform semantic recognition. If a user draws a curve in the upper portion of the canvas, the scene level increases the probability that the stroke belongs to a "cloud" model rather than a "terrain" model, even if the shapes are geometrically similar.

Global Optimization and Synthesis

Unlike many sketching tools that make "greedy," stroke-by-stroke decisions, this framework employs the Viterbi algorithm for global optimization. By evaluating the entire sequence of strokes, the system can find the most likely global interpretation of the drawing. This prevents inconsistencies where the beginning of an illustration might not align with the stylistic choices made at the end.

Once the optimal path through the HHMM is identified, the system synthesizes the refined curves. It propagates information back down the hierarchy, augmenting the original rough lines with learned artistic styles. This process goes beyond simple smoothing; it can synthesize varying pen pressure, specific color palettes, and even provide "seeds" for texture-filling algorithms to create a complete, colored illustration from a sparse outline.

Experimental Results

The authors demonstrated the versatility of their system using several specialized training ensembles, including leaf shapes, coastlines, and geometric primitives.

Refinement of geometric shapes, demonstrating the synthesis of color and style from rough outlines.

Key findings from these experiments include:

  • Semantic Accuracy: The system successfully distinguished between similar shapes based on their relative positions within a scene.
  • Coherence: Qualitative comparisons showed that the Viterbi-based global decoding produced significantly more coherent stylization than greedy methods, which frequently misclassified strokes mid-drawing.
  • Real-Time Performance: Despite the underlying mathematical complexity, the system achieved real-time performance on consumer-grade hardware of the era, making it suitable for interactive applications.

Limitations and Future Work

While the framework is powerful, it faces challenges regarding the "state space explosion" caused by high-dimensional data (angles, colors, thickness). To manage this, the authors utilized a candidate-based approach rather than storing full probability matrices. Additionally, the system currently requires paired training data (rough vs. refined), which can be labor-intensive to produce for new styles. Future work may focus on integrating texture synthesis more deeply into the probabilistic framework and providing users with more granular control over the refinement process.

Summary updated

Abstract

We present a method for generating refined 2D illustrations from hand drawn outlines consisting of only curve strokes. The system controllably synthesizes novel illustrations by augmenting the hand drawn curves’ shape, thickness, color and surrounding texture. These refinements are learned from a training ensemble. Users can select several examples that depict the desired idealized look and train the system for that type of refinement. Further, given several types of refinements, our system automatically recognizes the curve and applies the appropriate refinement. Recognition is accomplished by evaluating the likelihood the curve belongs to a particular class based on both its shape and context in the illustration.
BibTeX
@inproceedings{simhon2003curve,
  author = {Saul Simhon and Gregory Dudek},
  title = {{Curve Synthesis from Learned Refinement Models}},
  booktitle = {Eurographics (posters)},
  year = {2003},
  doi = {10.2312/egp.20031032},
}
Figure