Abstract
Real-time rendering at high resolutions remains a significant computational challenge for modern hardware. While video super-resolution (VSR) offers a potential solution, existing methods often struggle with the "trilemma" of visual quality, temporal stability, and inference latency. This paper introduces RDG, a novel VSR framework that leverages decoupled G-buffer guidance to achieve state-of-the-art performance. By strategically utilizing auxiliary engine data—such as normals, depth, and motion vectors—within an asymmetric UNet architecture, RDG achieves high-fidelity upscaling at over 120 FPS for 1080p content on modern GPUs.
Introduction
In the context of real-time graphics, rendering every pixel at native 4K resolution is often computationally prohibitive. The industry has shifted toward rendering at lower resolutions and upscaling using deep learning techniques. However, standard VSR methods are frequently unsuitable for real-time rendering because they rely on future frames (bidirectional propagation) or slow optical flow estimation. Existing rendering-specific SR methods, such as NSRR or NSRD, often fail to fully exploit the structural information within auxiliary G-buffers or introduce artifacts through techniques like radiance demodulation, which struggles with complex materials like glass or skin.
Methodology: Decoupled G-buffer Guidance
The core innovation of RDG is its Decoupled G-buffer Guidance strategy. Rather than concatenating all inputs, the architecture directs specific G-buffer components to modules where they are most effective, avoiding the pitfalls of mathematical demodulation.
Key Architectural Components
- Dynamic Feature Modulator (DFM): Located in the encoder, the DFM employs an asymmetric deformable sampling strategy. This allows the model to selectively encode spatial context and capture precise structural information without the high overhead of large-kernel convolutions or the blurring effects of standard pooling.
- High-frequency Feature Booster (HFB): Situated in the decoder, the HFB extracts sharp details from Normal and BRDF (Bidirectional Reflectance Distribution Function) maps. These maps provide "clean" geometric and material edges that guide the reconstruction of crisp textures.
- Cross-frame Temporal Refiner (CTR): To ensure temporal stability and eliminate "ghosting," the CTR uses Motion Vectors and Depth Maps for feature alignment. It incorporates a Partial Cross Attention (PCA) mechanism to effectively aggregate features, particularly in challenging occlusion areas, leading to significantly higher temporal consistency (measured by VMAF).
Experimental Results
The researchers evaluated RDG using a custom dataset generated with the Cycles engine (Blender), featuring 20 diverse scenes for training and 8 representative scenes for testing (e.g., Bistro, NYC, ZeroDay). The task involved upscaling from 270p to 1080p (4x SR).
Performance and Quality
RDG demonstrates a significant leap in both efficiency and quality:
- Efficiency: The "RDG-s" (small) variant achieves a blistering 126 FPS at 1080p on an RTX 3090, with a latency of just 7.93ms.
- Quality: RDG outperformed existing real-time methods like NSRR and FuseSR across all metrics. It achieved a 3.4dB PSNR gain over NSRD and a 1.95dB gain over FuseSR on average.
- Visual Fidelity: Qualitative analysis shows that RDG reconstructs much sharper text and geometric lines compared to competitors, which often produce blurry or artifact-heavy results in complex scenes.
Conclusion
RDG provides a robust solution for real-time rendering super-resolution by treating G-buffers as guidance rather than simple inputs. This approach proves more effective than traditional radiance demodulation, especially for anisotropic or translucent materials. While currently limited to rendered content where G-buffers are available, RDG sets a new benchmark for high-performance upscaling in interactive applications. Future work will focus on optimizing attention mechanisms to meet the strict 16ms frame budget required for 4K 60 FPS gaming.
Summary updated