TL;DR
PLS-GS introduces a probabilistic "scaffold" for 3D Gaussian Splatting that leverages LiDAR data to create more compact and geometrically accurate 3D models. By fitting a hierarchical mixture model to both position and color data, it produces representations that are 2.8x smaller and 1.7x faster to render than standard baselines while staying much more faithful to the actual scene geometry.
What Problem Does This Paper Address?
3D Gaussian Splatting (3DGS) has revolutionized real-time photorealistic rendering, but it has a "geometry problem." Because standard 3DGS is optimized primarily to match images, the resulting 3D "primitives" (the Gaussians) often end up in physically impossible places—floating in mid-air or overlapping redundantly—just to make the final render look right. This leads to bloated model sizes and poor geometric accuracy, which is a deal-breaker for robotics and mapping applications that need to know exactly where surfaces are.
Existing solutions either use discrete grids (like voxels) which are rigid and lose detail, or they try to "ground" Gaussians in LiDAR data but still allow them to drift away during the visual optimization process. There is a missing link: a continuous, adaptive structure that forces the model to respect real-world measurements without sacrificing visual quality.
Methods
The authors propose PLS-GS (Probabilistic LiDAR Scaffolds for Gaussian Splatting). Its core innovation is replacing discrete grids with a Hierarchical Gaussian Mixture Model (HGMM).
- Probabilistic Scaffolding: Instead of a voxel grid, they fit an HGMM to a 6D point cloud (3D position + 3D color). This creates a tree of "anchors" that adapt to the scene's density. Because they include color, the model can distinguish between surfaces that are close together but look different (like a white road marking on black asphalt).
- Surface-Aligned Decoding: Each anchor in the HGMM isn't just a point; it has a "covariance" that describes the local surface's orientation and extent. PLS-GS decodes neural Gaussians within this local surface frame.
- Mahalanobis Support Loss: To prevent Gaussians from drifting, they introduce a specialized loss function. It uses the Mahalanobis distance, which makes it "cheap" for a Gaussian to slide along a flat surface but "expensive" for it to move away from that surface.
- Probabilistic Growth: When the model needs more detail, it doesn't just clone points randomly. It fits a new local HGMM to areas with high rendering errors, adding anchors in a controlled, geometrically-grounded way.
Datasets
The researchers evaluated their method on GauU-Scene V2, a large-scale outdoor benchmark. This dataset is unique because it provides high-definition, survey-grade LiDAR data that is perfectly registered with the camera images, covering five massive scenes (Campus, Village, College, Modern Building, and Russian Building) spanning up to 1.56 $km^2$.
Key Insights and Ideas
- Color Matters for Geometry: By fitting the scaffold to both position and color, the HGMM naturally separates distinct physical objects even if they are touching, leading to better anchor placement.
- Continuous beats Discrete: Moving away from voxel grids to a continuous probabilistic model (HGMM) allows the scaffold to be much more compact while still representing complex, thin surfaces accurately.
- Geometric Inductive Bias: Forcing the model to express offsets in a "surface-aligned" frame acts as a powerful regularizer. It tells the optimization: "you can move things around to make the picture look better, but stay on the wall!"
- Controlled Growth: Standard 3DGS "densification" often explodes the number of primitives. The HGMM-based growth is self-limiting, ensuring that new anchors are only added where they are truly needed to explain visual residuals.
Key Results
PLS-GS demonstrates a superior trade-off between model size, rendering speed, and geometric fidelity:
- Compactness: It uses 2.8x fewer anchors and requires only 42% of the storage space compared to LiDAR-initialized baselines.
- Rendering Speed: It achieves a 1.7x faster frame rate, making it highly suitable for real-time robotics applications.
- Geometric Accuracy: Primitives are placed significantly closer to the ground-truth LiDAR measurements. The method improved surface-alignment precision by 1.47x over standard scaffolding methods.
- Visual Quality: Despite the massive reduction in model size, PLS-GS maintains comparable rendering quality (PSNR) to much larger, unconstrained models.
Limitations and Future Work
While PLS-GS is a major step forward for geometric grounding, the authors note that its peak rendering quality (PSNR) still trails slightly behind unconstrained, "bloated" models. They plan to address this by making the hierarchical refinement even more adaptive.
Future directions include investigating more aggressive compression strategies and extending the "geometrically faithful" approach to 4D Gaussian Splatting for dynamic scenes, where maintaining physical consistency over time is even more challenging.
Summary updated