Introduction
3D Gaussian Splatting (3DGS) has transformed 3D reconstruction, yet it remains largely isolated from traditional computer graphics workflows. Because 3DGS represents scenes as volumetric particles rather than solid surfaces, standard 3D meshes—like characters or vehicles—cannot naturally interact with these environments. Specifically, inserted objects lack the necessary lighting and reflection data from the surrounding "splatted" scene, making them appear flat and disconnected. GBake solves this by converting complex Gaussian data into a format traditional engines already understand: reflection probes (cubemaps).
The GBake Pipeline
GBake bridges the gap between 3DGS and traditional engines like Unity by following a structured workflow to bring Gaussians into the world of real-time mesh rendering:
- Bounding Box Definition: An artist defines a volume in the scene where meshes will be placed.
- Grid Generation: The system generates a 3D grid of sampling locations within that volume.
- Raytraced Sampling: At each grid point, the system renders six perspective views (forming a cube) with a 90-degree field of view.
- Integration: These renders are composited into cubemaps and exported. This data is then loaded into Unity as a system of reflection probes with "influence zones," allowing objects to smoothly interpolate between different probes as they move through the scene.
Overcoming the "Splatting Seam"
A key technical challenge in this process is the "splatting seam." Standard 3DGS rasterization (EWA Splatting) uses approximations that vary based on camera orientation. When these views are stitched into a cubemap, visible discontinuities appear at the edges. GBake solves this by using 3D Gaussian Raytracing (3DGRT). Unlike rasterization, raytracing ensures mathematical consistency; a ray in a specific direction always returns the same color, resulting in perfectly seamless environment maps.
Results and Performance
Evaluated on the Mip-Nerf 360 dataset (including scenes like Bonsai and Playroom), GBake demonstrates high visual fidelity for metallic and specular meshes. Despite the shift to raytracing, the baking process remains efficient. Generating a 125-probe grid (5x5x5) at 800px resolution takes only 21 to 36 seconds on an NVIDIA 3090, making it a viable tool for production environments.
Future Directions
While GBake successfully handles static environments, future work aims to integrate the tool more deeply with Unity’s Universal Render Pipeline (URP). This would allow for simultaneous baking of both splats and traditional meshes. Additionally, the team is exploring alternative projection formats, such as equirectangular maps, to further expand compatibility with various rendering pipelines.
Summary updated