TL;DR
This paper introduces a novel compositing method for 3D paintings that elegantly blends stroke order and depth order, resolving a long-standing conflict in rendering. Their approach ensures smooth transitions and avoids visual artifacts, making 3D painting more intuitive and visually appealing.
What Problem Does This Paper Address?
Traditional 2D painting offers artists a natural way to layer strokes, where newer strokes obscure older ones. Translating this to 3D painting, where strokes exist in space, presents a challenge: should strokes be composited based on their painted order (stroke order) or their spatial position (depth order)? Compositing purely by stroke order loses the 3D sense of depth, while compositing purely by depth order prevents artists from painting over existing strokes, breaking the familiar 2D painting metaphor. Existing methods often resort to ad-hoc solutions that can lead to artifacts like popping or unnatural blending. This paper tackles this "mixed-order compositing" problem, aiming for a method that respects both the artist's intent (stroke order) and the physics of 3D space (depth order) in a coherent and visually pleasing manner.
Methods
The core of the paper is a new compositing function that formalizes the desired properties for mixed-order rendering. The key idea is to replace the color of each fragment with a "smoothed" composite of nearby fragments in stroke order, and then composite these modified fragments in depth order.
- Stroke Order Compositing Window: For each fragment, they define a "compositing window" around its depth. Fragments within this window (defined by a parameter
d) are considered to be on the same surface and are composited using stroke order. Fragments outside this window are composited using depth order. - Depth-Based Smoothing: To ensure temporal coherence and avoid popping artifacts during animation, the stroke-order composite within the window is smoothed using a box filter of width
γd. This results in a piecewise-constant functionS(z)that represents the stroke-order composite at a given depth. - Replacement Colors: Each fragment's color is then replaced by the smoothed composite
S(z)evaluated at the fragment's depth, weighted by the fragment's alpha. This effectively "pre-blends" fragments that are close in depth and stroke order. - Depth Order Compositing: Finally, these fragments with their new replacement colors are composited in strict depth order. This ensures that depth order is respected for fragments far apart in depth, while the pre-blending handles the stroke order for fragments on the same surface.
- Efficient Algorithm: The authors develop an efficient O(n log n) algorithm using a binary tree data structure to compute the stroke-order composites and a sweep-line approach to calculate the replacement colors.
Datasets
Not discussed in the paper. The authors demonstrate their method using 3D paintings generated in the "OverCoat" system, implying these are custom scenes rather than standard datasets.
Key Insights and Ideas
- Formalizing Mixed-Order Compositing: The paper provides a clear set of four properties (stroke order for same depth, depth order for different depths, transparency handling, and temporal continuity) that a good mixed-order compositing function should satisfy. This formalization is crucial for understanding and evaluating solutions.
- The Power of Smoothing: The introduction of a smoothed compositing function
S(z)is key to achieving temporal coherence. By blurring the transitions between stroke order and depth order, they eliminate the jarring "popping" artifacts common in previous methods. - Bridging 2D and 3D Metaphors: The method successfully reconciles the artist's intuitive stroke-layering in 2D with the spatial realities of 3D, offering a more natural workflow for 3D painting.
- Efficiency is Achievable: Developing an efficient O(n log n) algorithm for this complex compositing problem is a significant technical achievement, making it practical for real-time applications.
Key Results
- The proposed method produces visually pleasing 3D paintings with no noticeable popping or other artifacts, as demonstrated in their video and figures.
- Artifacts present in previous methods like depth offsetting and clustering are effectively resolved.
- The algorithm runs efficiently in O(n log n) time, making it suitable for interactive applications.
- The method is demonstrated on a variety of 3D paintings, showcasing its versatility.
Limitations and Future Work
- Parameter Tuning: The method relies on two parameters,
d(stroke order window size) andγ(smoothing factor), which need to be chosen appropriately for each scene. While they provide guidelines, finding optimal values can still require artist input. - Artistically Undesirable Blending: While temporal coherence is improved, the blending introduced by
γmight not always align with a specific artistic style. The parameterγallows a trade-off between temporal coherence and blending. - Surface Identification: The method assumes fragments close in depth belong to the same surface. This assumption can break down in cases of complex interpenetrating surfaces or when artists interleave drawing on different surfaces. Future work could explore smoothly incorporating explicit surface information.
- Performance for Large Scenes: For very large scenes where
dis small relative to the depth range, further optimization to O(n log m) is possible by using dynamic binary trees. - Memory Optimization: Memory usage can be reduced to O(m) by interleaving algorithm stages.
- Error Bounded Compositing: Exploring compositing nearby fragments in stroke order with bounded error is a promising direction.
- Discarding Obscured Fragments: Optimizing by discarding fragments that are fully obscured by others closer to the viewer is also suggested.
Summary updated