Abstract
We give an overview of the different rendering methods and we demonstrate that the use of a Generative Adversarial Networks (GAN) for Global Illumination (GI) gives a superior quality rendered image to that of a rasterisations image. We utilise the Pix2Pix architecture and specify the hyper-parameters and methodology used to mimic ray-traced images from a set of input features. We also demonstrate that the GANs quality is comparable to the quality of the ray-traced images, but is able to produce the image, at a fraction of the time.
Introduction
In 3D rendering, Global Illumination (GI) represents the pinnacle of realism, simulating the complex ways light bounces off surfaces to create natural effects like color bleeding and soft shadows. Traditionally, developers have faced a binary choice: high-speed rasterization, which lacks these natural light interactions, or ray-tracing, which offers stunning realism but requires immense computational power and modern hardware with dedicated "RT cores."
Millions of older devices lack this specialized hardware, leading to a widening gap in visual fidelity. This research addresses this challenge by treating global illumination as an image-to-image translation problem, aiming to bring ray-traced quality to "legacy" hardware by leveraging the predictive power of Generative Adversarial Networks (GANs).
Methodology
The researchers employed a modified Pix2Pix architecture to translate "cheap" rasterized buffers into high-quality GI images. The pipeline utilizes four primary inputs generated via Blender’s fast Eevee engine: Direct Illumination, a Depth Buffer, a Normal Map, and a Diffuse Image.
Architecture and Training
The network consists of a U-Net generator and a PatchGAN discriminator. A critical engineering challenge arose due to memory constraints: training with a batch size of one caused standard Batch Normalization to fail, resulting in visual artifacts. The authors resolved this by implementing Group Normalization, which significantly stabilized training and improved image clarity.
Dataset Generation
A custom dataset was created using Blender, consisting of 2,509 image sets (12,545 total images). These scenes featured randomized primitive shapes with varying positions, rotations, and colors. Ground truth images were rendered using Blender’s Cycles path-tracing engine at 1024 samples to provide a high-fidelity target for the GAN.
Key Results
The GAN-based approach achieved a remarkable balance between visual fidelity and computational efficiency:
- Visual Fidelity: The model achieved a Structural Similarity Index (SSIM) of 0.9849, where 1.0 is a perfect match to the ray-traced ground truth. This significantly outperformed standard rasterization (0.9376).
- Performance: In testing, a scene that required 33.91 seconds to ray-trace was processed by the GAN (including the initial rasterization pass) in just 0.275 seconds.
- Real-Time Potential: The authors estimate that with further optimization, this method could achieve approximately 45 FPS on older hardware like the GTX 1050, making it viable for real-time gaming applications.
Discussion and Future Work
By approximating light transport as a learned visual signature rather than a direct physical calculation, this method offers a path toward more sustainable computing. Extending the lifespan of older GPUs reduces e-waste and makes high-end graphics more accessible.
However, limitations remain. The current model is constrained by screen-space information; it cannot account for light sources or objects outside the camera's immediate view. Furthermore, highly reflective surfaces and temporal coherence (flicker in motion) require further investigation. Future work may involve moving toward 3D feature sets to provide the GAN with broader spatial awareness beyond the 2D image buffer.
Summary updated