Abstract
We present an efficient raycasting algorithm for rendering Volumetric Depth Images (VDIs), and we show how it can be used in a remote visualization setting with VDIs generated and streamed from a remote server. VDIs are compact view-dependent volume representations that enable interactive visualization of large volumes at high frame rates by decoupling viewpoint changes from expensive rendering calculations. However, current rendering approaches for VDIs struggle with achieving interactive frame rates at high image resolutions. Here, we exploit the properties of perspective projection to simplify intersections of rays with the view-dependent frustums in a VDI and leverage spatial smoothness in the volume data to minimize memory accesses. Benchmarks show that responsive frame rates can be achieved close to the viewpoint of generation for HD display resolutions, providing high-fidelity approximate renderings of Gigabyte-sized volumes. We also propose a method to subsample the VDI for preview rendering, maintaining high frame rates even for large viewpoint deviations. We provide our implementation as an extension of an established open-source visualization library.
Introduction
Direct Volume Rendering (DVR) is the gold standard for visualizing 3D scientific data, but its computational intensity often prevents interactive exploration of gigabyte-scale datasets. This challenge is exacerbated in remote visualization scenarios where network latency compounds rendering delays. Volumetric Depth Images (VDIs) address this by providing a compact, view-dependent representation. Instead of full voxel grids, VDIs store "supersegments"—composited chunks of color and opacity along rays from a specific viewpoint. While VDIs decouple expensive server-side rendering from client-side interaction, previous rendering methods failed to achieve the 60+ FPS required for modern high-definition displays.
Methods: Optimized VDI Raycasting
The proposed algorithm introduces several key optimizations to achieve high-performance rendering:
NDC Space Raycasting
The primary bottleneck in VDI rendering is calculating ray intersections with complex pyramidal frustums in world space. By transforming the entire volume into Normalized Device Coordinate (NDC) space, the VDI structure is simplified into a regular grid of cuboids. This transformation allows the use of efficient voxel traversal algorithms originally designed for simple grids, significantly reducing mathematical complexity.
Exploiting Spatial and Temporal Patterns
Real-world scientific data often exhibits spatial smoothness. The algorithm leverages this by using results from neighboring rays to predict data locations for the current ray, reducing memory lookups by up to 40%. Additionally, a grid-based acceleration structure enables empty-space skipping, and a PI (Proportional-Integral) controller dynamically adjusts rendering resolution (subsampling) during rapid camera movements to maintain responsiveness.
Results and Evaluation
The system was evaluated using diverse datasets, including the Kingsnake (1024x1024x795 CT scan), Rayleigh-Taylor (1024³ fluid simulation), and Richtmyer-Meshkov (massive 2048x2048x1920 entropy field).
Performance and Interactivity
On an NVIDIA RTX 3090, the proposed method achieved speedups of 4.5x to 24.5x compared to standard DVR for HD resolutions. For the Kingsnake dataset, the system maintained over 100 FPS at HD resolution.
Visual Fidelity
Using the Structural Similarity Index (SSIM), the VDI renderings were compared against "ground truth" DVR images. As shown in Figure 1, the method maintains high fidelity (SSIM > 0.93) even at 30° deviations from the original viewpoint, while providing significantly higher frame rates than the reference renderer.
Limitations and Future Work
While highly efficient, VDIs are view-dependent and do not store occluded data. Significant camera deviations can lead to quality degradation until a new VDI is streamed from the server. Furthermore, the current implementation uses pre-classified color and opacity, limiting dynamic lighting effects. Future research will explore foveated rendering and advanced controllers to further optimize the quality-performance trade-off.
Summary updated