OverCoat: an implicit canvas for 3D painting

Johannes Schmid, Martin Sebastian Senn, Markus H. Gross, Robert W. SumnerACM Trans. Graphics 2011 · DOI

TL;DR

This paper introduces OverCoat, a novel system that extends the familiar 2D painting metaphor into 3D space. It allows artists to paint "in space" rather than just on surfaces, enabling a new class of expressive 3D artwork that was previously difficult or impossible to achieve.

What Problem Does This Paper Address?

Traditional 3D painting often falls into two categories: texture painting on existing 3D models or methods that force strokes to conform to object surfaces. This "gift-wrapped" appearance limits artistic freedom, making it hard to create effects like fur, hair, or smoke that naturally extend beyond surfaces. Existing systems struggle to capture the expressive, off-surface qualities of traditional 2D painting, hindering artists from realizing their full creative vision in 3D.

Methods

OverCoat tackles this by introducing an "implicit canvas" defined by a 3D scalar field. Artists first create approximate 3D proxy geometry, which implicitly defines this scalar field (initially as a signed distance field). The core innovation lies in an optimization procedure that embeds painted strokes into this 3D space. This procedure allows strokes to be placed not just on specific level sets (isosurfaces) of the scalar field but also to traverse between them.

Key technical contributions include:

  • Implicit Canvas: A 3D scalar field that acts as the painting space, shaped by proxy geometry.
  • Optimization-based Stroke Embedding: Strokes are embedded by solving an optimization problem that satisfies various objective criteria (e.g., staying on a level set, smoothness, straightness). This offers flexibility beyond simple projection.
  • Unified Painting/Sculpting Metaphor: A sculpting tool allows artists to directly modify the scalar field, creating protrusions or indentations that influence subsequent painting. This provides a cohesive workflow for shaping and painting.
  • Efficient Rendering: A depth offsetting method is used to resolve visibility ambiguities inherent in 3D stroke rendering, enabling interactive performance.

Datasets

Not discussed in the paper. The paper focuses on a novel system and demonstrates its capabilities with artist-created examples.

Key Insights and Ideas

  • Blurring Primary and Secondary Space: The system fundamentally shifts the paradigm by treating strokes as primary 3D elements and 3D objects as mere helpers for defining the painting space. This is a crucial conceptual leap for 3D art creation.
  • Optimization as a Flexible Tool: Framing stroke embedding as an optimization problem unlocks a wide range of expressive possibilities, allowing artists to define strokes that adhere to complex criteria beyond simple surface constraints.
  • Sculpting as Canvas Shaping: Integrating sculpting directly into the painting workflow, where it modifies the underlying scalar field, provides a powerful way to iteratively refine the 3D canvas before or during painting.
  • Expressiveness Beyond Surfaces: The ability to paint "in space" is key to achieving stylized effects like the rough tail of the cat in Figure 1, which would be extremely difficult with traditional surface-based methods.

Key Results

The paper demonstrates OverCoat's capabilities through five complete 3D paintings created by artists. These examples showcase the system's ability to:

  • Create expressive, non-surface-conforming strokes (e.g., the cat's tail, fur, hair).
  • Utilize sculpting to add details like beards and eyebrows.
  • Render complex scenes with tens of thousands of strokes at interactive frame rates (e.g., "Wizard vs. Genie" with 24,000 strokes rendered in 610ms).
  • Highlight the benefits of their embedding method by comparing it to a projection-based approach, showing how OverCoat preserves stylization and expressive qualities.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Manual Lighting and Texture: Artists must paint all lighting and texture information by hand, which is labor-intensive. Future work could explore transferring scene lighting and shading information to painted strokes.
  • Splat-based Model Fidelity: The current brush model is simple. More sophisticated splat-based models could enhance realism.
  • View-Dependent Effects: The current system doesn't inherently support view-dependent shape changes common in stylized 2D art. Future work could incorporate view-dependent geometry.
  • Layering and Compositing: The renderer lacks the advanced layering and compositing controls found in 2D software.
  • Stroke Order Ambiguities: While addressed, the splat ordering method can lead to occasional popping artifacts. More robust solutions for stroke-order ambiguities are needed.
  • Integration with Modeling: While artists used external tools, integrating sketch-based modeling and deformation directly into OverCoat would streamline workflows.
  • Animation: The system currently supports static paintings. Extending it to animate paintings, perhaps by deforming strokes with animated proxy objects or using space deformation, is a significant future direction.

Summary updated

BibTeX
@article{schmid2011overcoat,
  author = {Johannes Schmid and Martin Sebastian Senn and Markus H. Gross and Robert W. Sumner},
  title = {{OverCoat: an implicit canvas for 3D painting}},
  journal = {ACM Trans. Graphics},
  year = {2011},
  doi = {10.1145/2010324.1964923},
}
Figure