A Flexible Neural Renderer for Material Visualization

Aakash K. T., Parikshit Sakurikar, Saurabh Saini, P. J. Narayanan

Introduction

The creation of photorealistic computer-generated imagery is a highly iterative process. Artists must meticulously tune material parameters—such as roughness, specular, and diffuse colors—to achieve a desired look. Traditionally, this requires waiting for computationally expensive ray-traced previews. While denoising and direct neural rendering have emerged as alternatives, they often struggle with spatially-varying materials (SVBRDFs) or lack interactive lighting control. This paper introduces a lightweight, U-Net-based neural renderer that provides high-quality material visualizations on a standard shaderball in under 3 milliseconds, supporting both uniform and complex materials under dynamic lighting.

Methods

The core of the system is a convolutional autoencoder based on the U-Net architecture. Unlike methods that operate on raw textures, this renderer utilizes screen-space maps of Cook-Torrance parameters (Diffuse, Specular, Roughness, and Normal) as input. These maps are generated by rasterizing a fixed 3D shaderball geometry.

To enable interactive lighting, the model incorporates a separate fully-connected encoder for environment parameters, specifically sun direction and turbidity based on the Hosek-Wilkie sky model. This lighting embedding is integrated into the U-Net's bottleneck. Training is performed using a composite loss function that combines $L_1$ distance with a VGG16-based perceptual loss, which is critical for preserving high-frequency details from normal maps.

Material visualization on a shaderball under varying lighting conditions. The dashed lines connect scene materials to their corresponding neural shaderball renders.

Dataset and Training

The authors developed a synthetic dataset of 50,000 material parameter maps. This dataset is split between uniform and spatially-varying materials, with the latter sourced from the Deschaintre et al. (2018) SVBRDF collection. Ground-truth images were rendered at 150 samples-per-pixel (spp) using Blender. The final model is compact, featuring only 11.7 million parameters, making it significantly more efficient than previous neural rendering architectures.

Results and Discussion

The proposed neural renderer achieves a rendering time of approximately 2.7 ms per frame, outperforming denoising pipelines (~72.5 ms) and prior direct renderers (~13.8 ms). Quantitatively, the model reaches a PSNR of 37.65 dB and an SSIM of 0.985.

A key finding of the research is the functional importance of interactive lighting. A user study demonstrated that artists' ability to identify materials improved from 17.9% under fixed lighting to 49.3% when given control over the light source direction. This underscores that real-time lighting feedback is not merely a visual enhancement but a necessity for accurate material perception.

A detailed render of a wood-textured spatially-varying material (SVBRDF) produced by the neural renderer.

Conclusion

This work provides a practical tool for 3D artists, enabling near-instantaneous feedback during the material design process. By combining a lightweight U-Net architecture with interactive lighting control, the renderer handles complex SVBRDFs with high fidelity. Future work aims to extend this capability to arbitrary, user-defined 3D geometries beyond the fixed shaderball.

Summary updated

BibTeX
@inproceedings{t2019flexible,
  author = {Aakash K. T. and Parikshit Sakurikar and Saurabh Saini and P. J. Narayanan},
  title = {{A Flexible Neural Renderer for Material Visualization}},
  booktitle = {SIGGRAPH Asia Technical Briefs},
  year = {2019},
  doi = {10.1145/3355088.3365160},
}
Figure