Making Japenese Ukiyo-e Art 3D in Real-Time

Innes Brown, Ognjen ArandjelovićSci 2020 · DOI

TL;DR

This paper introduces the first real-time 3D rendering framework to emulate the distinctive visual style of Japanese Ukiyo-e art. By analyzing the physical processes behind Ukiyo-e prints, the authors developed novel algorithms for line rendering and color application that can transform standard 3D models into visually compelling Ukiyo-e-style imagery, suitable for interactive applications like video games.

What Problem Does This Paper Address?

Traditional 3D computer graphics often strives for photorealism. However, many artistic styles, particularly those from non-Western traditions, employ distinct visual conventions that deviate from realism to convey information or evoke emotion. Ukiyo-e, a traditional Japanese woodblock printing style, is characterized by bold line work, vibrant colors, and a unique non-perspective projection. While research has explored emulating various painting styles in 2D, creating a real-time 3D rendering system that captures the essence of Ukiyo-e, including its characteristic line quality, color gradation (bokashi), and flat aesthetic, has been an underexplored challenge. This paper aims to bridge this gap by developing a framework that can automatically render existing 3D models in this specific artistic style.

Methods

The authors propose a comprehensive framework comprising several key components:

  • Line Rendering: This process mimics the creation of Ukiyo-e keyblocks. It involves:
  • Important Edge Detection: Identifying silhouettes, creases, and boundary edges of 3D models.
  • Reference Image Generation: Creating a 2D image where edges are assigned unique colors to aid in analysis.
  • Polyline Construction: Analyzing the reference image to build polylines that trace the visible edges. This involves sophisticated segment linking and identification to ensure smooth, coherent lines.
  • Stroke Rendering: Building polygon strips along the polylines and rendering stroke textures to create the final 2D lines.
  • Color Rendering: This component aims to replicate the vibrant and often imperfect coloring of Ukiyo-e prints:
  • Pseudo-Uniform Colors: Applying a base color with added noise texture in screen space to simulate the subtle inconsistencies found in traditional prints. This noise texture is adjusted based on object depth and screen position to maintain temporal coherence.
  • Pattern Textures: Extending the noise texture approach to incorporate patterns, mimicking designs found on clothing or other elements in Ukiyo-e art.
  • Bokashi Effect: Two algorithms are proposed to emulate the characteristic color gradation.
  • Bokashi Algorithm 1: Mimics the physical printing process by creating a linear gradation in screen space, influenced by a light source and object center. This algorithm prioritizes a flat, 2D appearance.
  • Bokashi Algorithm 2: Based on traditional 3D shading techniques (Gouraud/Phong), it treats objects as spheres for shading calculations, resulting in a more 3D-like gradation that approximates object shape while still maintaining a stylized look.

The entire framework is designed for computational efficiency to achieve real-time performance.

Datasets

Not explicitly discussed in the paper. The authors mention using "conventional 3D models" and provide links to online repositories for freely available 3D models (e.g., from MIT and Florida State University) used for demonstration and evaluation.

Key Insights and Ideas

  • Deconstructing the Art Form: The core innovation lies in analyzing the physical process of Ukiyo-e printmaking (woodblock carving, inking, printing) and translating those steps into computational algorithms. This is a powerful approach for emulating artistic styles.
  • Screen-Space Texturing for 2D Emulation: For color and texture, applying them in 2D screen space rather than 3D world space is crucial for achieving the characteristic flat, print-like appearance of Ukiyo-e, avoiding the "swimming" effect of textures in motion.
  • Bokashi as a Shading Problem: Recognizing that bokashi is not just simple shading but a specific artistic effect, the authors developed two distinct algorithms: one prioritizing the 2D print aesthetic and another offering more 3D-like gradation for temporal coherence.
  • Performance Bottleneck Identification: The analysis clearly shows that the reference image analysis (specifically edge detection and polyline construction) is the primary performance bottleneck, rather than polygon count or the rendering of strokes themselves. This highlights the importance of optimizing the initial image processing steps.
  • Resolution Trade-off: The paper demonstrates that reducing the resolution of the reference image can significantly boost performance with minimal visual impact on the final output, offering a practical way to achieve real-time frame rates.

Key Results

  • Real-time Performance: The framework achieves interactive frame rates (above 30 fps) across various scenes, even with complex models.
  • Visual Emulation: The rendered images successfully capture the distinct visual characteristics of Ukiyo-e prints, including line work, color, and gradation.
  • Performance Analysis:
    • Line rendering is identified as the performance bottleneck.
  • Color rendering algorithms scale linearly with polygon count.
  • Reducing reference image resolution from 1080p to 720p can nearly double frame rates (e.g., from 22 fps to 40 fps) with visually similar results.
  • An empty scene at 1080p resolution limits frame rate to under 30 fps due to the initial reference image analysis.

Limitations and Future Work

  • Perspective: The current implementation uses conventional perspective projection, which differs from the non-perspective or multi-viewpoint style often seen in traditional Ukiyo-e. The authors note this makes their renderings more akin to the early, short-lived Ukiyo-e movement that was influenced by Western paintings.
  • Texture Rotation: The 2D nature of texture application makes it difficult to maintain temporal coherence during significant 3D object rotation, as the texture orientation doesn't change with the object.
  • Bokashi Algorithm 1 Limitations: This algorithm produces linear gradations and cannot accurately represent multiple light sources or complex curved gradations. Its screen-space nature also makes it resolution-dependent.
  • Future Work:
    • Exploring non-conventional perspective projections to better match the historical Ukiyo-e style.
  • Investigating ways to improve texture handling for 3D rotations.
  • Further refining the bokashi algorithms for more complex lighting scenarios and shapes.
  • Developing the framework for broader applications, potentially using industry-standard graphics APIs like OpenGL.

Summary updated

Abstract

Ukiyo-e is a traditional Japanese painting style most commonly printed using wood blocks. Ukiyo-e prints feature distinct line work, bright colours, and a non-perspective projection. Most previous research on ukiyo-e styled computer graphics has been focused on creation of 2D images. In this paper we propose a framework for rendering interactive 3D scenes with ukiyo-e style. The rendering techniques use standard 3D models as input and require minimal additional information to automatically render scenes in a ukiyo-e style. The described techniques are evaluated based on their ability to emulate ukiyo-e prints, performance, and temporal coherence.
BibTeX
@inproceedings{brown2020making,
  author = {Innes Brown and Ognjen Arandjelović},
  title = {{Making Japenese Ukiyo-e Art 3D in Real-Time}},
  booktitle = {Sci},
  year = {2020},
  doi = {10.3390/sci2010006},
}

Loading related content...

Figure