TL;DR
GeoTexDensifier is a new framework that significantly improves 3D Gaussian Splatting (3DGS) by making the "densification" process smarter. By using texture gradients and geometric priors (depth and normals), it places more splats where details are rich and filters out noisy "floaters" in flat areas, resulting in higher-quality renders with better memory efficiency.
What Problem Does This Paper Address?
3D Gaussian Splatting (3DGS) has revolutionized real-time 3D reconstruction, but it has a "splat distribution" problem. To capture a scene, 3DGS starts with a sparse point cloud and "densifies" it—splitting or cloning Gaussians to fill in details.
Current methods usually rely on simple positional gradients to decide where to add more splats. This often leads to two issues:
- Under-reconstruction: Missing fine details in highly textured areas (like the intricate patterns on a carpet).
- Over-reconstruction: Creating a "noisy" mess of floating splats in low-texture or unconstrained areas (like a flat white wall), which wastes memory and creates visual artifacts.
Existing solutions often over-densify the entire scene to fix the first problem, which tanks performance and bloats file sizes.
Methods
The authors introduce GeoTexDensifier, which combines two main strategies:
1. Texture-Aware Densification
Instead of just looking at positional gradients, the system looks at the texture richness of the training images. It uses a hyperbolic tangent function activated by image gradients to weight the importance of a region.
- High Texture: The system encourages more splitting to capture fine details.
- Low Texture: The system keeps splats sparse, preventing the creation of unnecessary "floaters" on flat surfaces.
2. Geometry-Aware Splitting
To ensure splats are placed accurately in 3D space, the framework uses monocular depth and normal priors (from ZoeDepth):
- Validation of Depth Ratio Change (VDRC): When a Gaussian splits, the system checks the new splats against a relative depth map. If a new splat's depth ratio deviates too much from its parent (e.g., a splat on a foreground door trying to "split" into the background wall), it is discarded.
- Normal-Guided Splitting: Rather than placing new splats randomly, the system uses surface normals to nudge them onto the estimated geometric surface. This is especially helpful in "weakly textured" areas where visual cues are scarce.
Datasets
The researchers evaluated their method on several standard and custom benchmarks:
- Mip-NeRF 360: 9 complex indoor and outdoor scenes.
- Tanks and Temples: 21 diverse scenes used to test reconstruction robustness.
- Self-Captured Outdoor Scenes: Two large-scale scenes ("Wufeng Tower" and "Office Park") captured via drones to test real-world applicability.
Key Insights and Ideas
- Relative Depth is Enough: You don't need perfectly scaled 3D depth maps to improve 3DGS. Because monocular depth models are great at relative depth, checking the ratio of depth change between a parent and child splat is a highly effective filter for noise.
- Texture as a Density Guide: Positional gradients can be misleading. By explicitly using image texture as a "budget" for splat density, the model achieves a "compact" representation—putting the complexity only where the eye can actually see it.
- Balancing Sparsity and Detail: The framework proves that you can actually get better visual quality with fewer Gaussians in certain areas, provided they are positioned correctly.
Key Results
GeoTexDensifier consistently outperformed state-of-the-art (SOTA) methods like Mini-Splatting and GeoGaussian across SSIM, PSNR, and LPIPS metrics.
- Visual Quality: Renders of complex textures (carpets, chairs, architectural ornaments) are noticeably sharper and free of the "blur" or "ghosting" seen in original 3DGS.
- Efficiency: In the "Bonsai" test case, GeoTexDensifier achieved a higher frame rate (177 FPS vs. 132 FPS for Mini-Splatting) and used significantly less GPU memory (3.75GB vs. 6.61GB).
- Quantitative Gains: On the Tanks and Temples dataset, the method showed a clear lead in PSNR (Peak Signal-to-Noise Ratio), indicating much more accurate reconstructions.
Limitations and Future Work
- Object-Centric Backgrounds: The VDRC (depth check) can sometimes be too aggressive. In scenes with a very distant background and a close foreground object, the system might fail to split splats into the far background because the depth ratio jump is too extreme, leading to "under-reconstruction" in the distance.
- Training Time: Processing depth and normal priors adds computational overhead during the training phase, making it slower to train than the original 3DGS.
- Future Directions: The authors plan to scale the system for urban-scale reconstructions using "chunk-based" processing and explore using generative diffusion models to fill in "invisible" regions that weren't captured by cameras.
Summary updated