TL;DR
DomeGS is a novel framework that fixes the "missing background" problem in 3D Gaussian Splatting (3DGS) scenes by integrating a generative skydome directly into the 3D representation. By turning the distant horizon and sky into actual 3D Gaussians rather than a static 2D backdrop, it provides seamless blending and realistic motion parallax, significantly boosting immersion and spatial presence for Virtual Reality (VR) users.
What Problem Does This Paper Address?
While 3D Gaussian Splatting (3DGS) has revolutionized high-fidelity 3D scene reconstruction, it has a major "unbounded" problem. When capturing outdoor environments, it’s often impossible to get enough multi-view data for the sky or distant horizons. This leaves reconstructions with "holes," jagged edges, or floating artifacts at the boundaries.
Traditional VR systems mask this with a skybox (an infinite cube) or a skydome (a large sphere) with a texture applied. However, these are usually post-processed layers that don't "talk" to the 3D geometry. This leads to two main issues:
- Radiometric Incoherence: The lighting and colors of the background often don't match the reconstructed foreground.
- Lack of Kinematic Parallax: In a standard skybox, the background stays static as you move your head. In the real world, even distant objects shift slightly (parallax), and losing this cue breaks immersion in VR.
Methods
The DomeGS pipeline consists of three main stages designed to turn a partial 3D capture into a complete, unified 3D environment:
- Panorama Extraction and Outpainting: The system extracts a partial 306° panorama from the existing 3DGS scene. It then uses a generative AI model (specifically FLUX.2 with a specialized LoRA) to "hallucinate" the missing sky and horizon regions while maintaining the original scene's colors and lighting.
- Scene-Adaptive Dataset Preparation: To move from 2D back to 3D, the system calculates geometric parameters like the optimal radius for a background shell. It creates a new "supervision dataset" by rendering views of the completed panorama from various angles.
- Unified Sky Gaussian Optimization: This is the "secret sauce." Instead of just displaying the panorama, DomeGS seeds new 3D Gaussians on a finite-radius spherical shell. It then fine-tunes the entire scene—original foreground plus new background Gaussians—together.
To keep the background Gaussians behaving like a sky, the authors introduced two new mathematical constraints: a radial shell loss (keeping them on the sphere) and a planarity loss (aligning them to prevent "spiky" artifacts).
Datasets
The researchers evaluated their method using four outdoor unbounded 3DGS reconstructions:
- Panoramic scenes: Buddha and Tower (open, aerial views dominated by distant backgrounds).
- Close-range scenes: Garden and Bear (cluttered environments with more nearby geometry).
They also conducted a comprehensive user study with N=36 participants to measure subjective immersion and presence.
Key Insights and Ideas
- The "Unified Asset" Philosophy: The core "aha moment" is that the background shouldn't be a separate layer. By representing the sky as Gaussians within the same 3D asset, the entire environment responds cohesively to user movement.
- Finite Geometry for Infinite Skies: Even though the sky is effectively at infinity, representing it on a finite sphere (a skydome) provides the subtle motion parallax cues that the human brain expects in VR.
- Generative AI as a Bridge: The paper demonstrates a practical way to use 2D generative models to solve 3D reconstruction gaps. It uses the AI to fill in what the camera missed, then "distills" that 2D knowledge back into the 3D Gaussian representation.
Key Results
- Increased Immersion: In the user study, DomeGS significantly outperformed the traditional skybox baseline in perceived immersion ($p < 0.001$) and background consistency under motion ($p < 0.001$).
- Spatial Presence: Participants reported a statistically significant increase in spatial presence ($p = 0.042$) and experienced realism ($p = 0.042$).
- User Preference: 75% of participants preferred DomeGS for a "prolonged stay" in the virtual environment compared to only 8.3% for the skybox.
- Automated Quality: Using a Vision-Language Model (Qwen3-VL) for objective inspection, DomeGS consistently scored higher in boundary continuity and lighting coherence across all test scenes.
Limitations and Future Work
While DomeGS creates a much more believable environment, it currently focuses on static backgrounds. The generated sky doesn't have moving clouds or dynamic day-night cycles.
The authors suggest that future work could involve integrating dynamic generative models to create "living" backgrounds. Additionally, while the method handles color matching well, more complex light-path integration between the generative background and the foreground objects (like reflections) remains an open challenge for future 3DGS research.
Summary updated