PLS-GS: Probabilistic Lidar Scaffolds for Compact, Surface-Aligned Gaussian Splatting

Krishnan Harikumar, Alejandro Fontan, Thierry Peynot, Emili Hernández, Niko SünderhaufarXiv 2026 · arXiv

TL;DR

PLS-GS introduces a probabilistic "scaffold" for 3D Gaussian Splatting that leverages LiDAR data to create more compact and geometrically accurate 3D models. By fitting a hierarchical mixture model to both position and color data, it produces representations that are 2.8x smaller and 1.7x faster to render than standard baselines while staying much more faithful to the actual scene geometry.

What Problem Does This Paper Address?

3D Gaussian Splatting (3DGS) has revolutionized real-time photorealistic rendering, but it has a "geometry problem." Because standard 3DGS is optimized primarily to match images, the resulting 3D "primitives" (the Gaussians) often end up in physically impossible places—floating in mid-air or overlapping redundantly—just to make the final render look right. This leads to bloated model sizes and poor geometric accuracy, which is a deal-breaker for robotics and mapping applications that need to know exactly where surfaces are.

Existing solutions either use discrete grids (like voxels) which are rigid and lose detail, or they try to "ground" Gaussians in LiDAR data but still allow them to drift away during the visual optimization process. There is a missing link: a continuous, adaptive structure that forces the model to respect real-world measurements without sacrificing visual quality.

Methods

The authors propose PLS-GS (Probabilistic LiDAR Scaffolds for Gaussian Splatting). Its core innovation is replacing discrete grids with a Hierarchical Gaussian Mixture Model (HGMM).

  1. Probabilistic Scaffolding: Instead of a voxel grid, they fit an HGMM to a 6D point cloud (3D position + 3D color). This creates a tree of "anchors" that adapt to the scene's density. Because they include color, the model can distinguish between surfaces that are close together but look different (like a white road marking on black asphalt).
  2. Surface-Aligned Decoding: Each anchor in the HGMM isn't just a point; it has a "covariance" that describes the local surface's orientation and extent. PLS-GS decodes neural Gaussians within this local surface frame.
  3. Mahalanobis Support Loss: To prevent Gaussians from drifting, they introduce a specialized loss function. It uses the Mahalanobis distance, which makes it "cheap" for a Gaussian to slide along a flat surface but "expensive" for it to move away from that surface.
  4. Probabilistic Growth: When the model needs more detail, it doesn't just clone points randomly. It fits a new local HGMM to areas with high rendering errors, adding anchors in a controlled, geometrically-grounded way.
The PLS-GS anchor formulation (right) aligns Gaussians with the local surface frame, unlike the world-axis offsets used in standard Scaffold-GS (left).

Datasets

The researchers evaluated their method on GauU-Scene V2, a large-scale outdoor benchmark. This dataset is unique because it provides high-definition, survey-grade LiDAR data that is perfectly registered with the camera images, covering five massive scenes (Campus, Village, College, Modern Building, and Russian Building) spanning up to 1.56 $km^2$.

Key Insights and Ideas

  • Color Matters for Geometry: By fitting the scaffold to both position and color, the HGMM naturally separates distinct physical objects even if they are touching, leading to better anchor placement.
  • Continuous beats Discrete: Moving away from voxel grids to a continuous probabilistic model (HGMM) allows the scaffold to be much more compact while still representing complex, thin surfaces accurately.
  • Geometric Inductive Bias: Forcing the model to express offsets in a "surface-aligned" frame acts as a powerful regularizer. It tells the optimization: "you can move things around to make the picture look better, but stay on the wall!"
  • Controlled Growth: Standard 3DGS "densification" often explodes the number of primitives. The HGMM-based growth is self-limiting, ensuring that new anchors are only added where they are truly needed to explain visual residuals.
The growth stage fits local HGMMs to high-residual regions, progressively shrinking uncovered areas.

Key Results

PLS-GS demonstrates a superior trade-off between model size, rendering speed, and geometric fidelity:

  • Compactness: It uses 2.8x fewer anchors and requires only 42% of the storage space compared to LiDAR-initialized baselines.
  • Rendering Speed: It achieves a 1.7x faster frame rate, making it highly suitable for real-time robotics applications.
  • Geometric Accuracy: Primitives are placed significantly closer to the ground-truth LiDAR measurements. The method improved surface-alignment precision by 1.47x over standard scaffolding methods.
  • Visual Quality: Despite the massive reduction in model size, PLS-GS maintains comparable rendering quality (PSNR) to much larger, unconstrained models.
Figure 1. PLS-GS produces compact, surface-aligned scaffolds that place primitives closer to the measured surface. Two held-out views of the College scene (rows). From the LiDAR point cloud and SfM input we build a continuous 6D HGMM scaffold (6D Scaffold) whose anchors are surface-aligned, in contrast to the unstructured anchor set of Scaffold-GS. PLS-GS renders at 35% of the model size of the LiDAR-initialized baseline. Right: it attains higher precision (⋆) and lower Chamfer distance than that baseline (•) at a fraction of the size.

Limitations and Future Work

While PLS-GS is a major step forward for geometric grounding, the authors note that its peak rendering quality (PSNR) still trails slightly behind unconstrained, "bloated" models. They plan to address this by making the hierarchical refinement even more adaptive.

Future directions include investigating more aggressive compression strategies and extending the "geometrically faithful" approach to 4D Gaussian Splatting for dynamic scenes, where maintaining physical consistency over time is even more challenging.

Summary updated

Abstract

While 3D Gaussian Splatting (3DGS) is typically optimized from images only, many robotics and mapping applications provide both camera and high-definition LiDAR data, combining rich visual appearance with accurate geometric structure. We introduce PLS-GS, a probabilistic LiDAR scaffold for Gaussian Splatting that exploits this complementary sensing setting to construct a compact 3DGS representation aligned to the measured surface. PLS-GS fits a Hierarchical Gaussian Mixture Model jointly over position and color, yielding a continuous and adaptive scaffold that eliminates the need for discrete scene partitioning such as voxel grids or octrees. The fitted mixture then guides the renderable 3DGS Gaussians: each rendering Gaussian is associated with a GMM component and constrained in the local surface frame induced by that component's covariance. A Mahalanobis support loss bounds the deviation of the rendering Gaussians from the corresponding LiDAR-derived surface. Across five large outdoor scenes, PLS-GS places primitives markedly closer to the LiDAR measurements than both image- and LiDAR-initialized baselines, while employing substantially fewer primitives, yielding a considerably more compact model and faster rendering. At matched capacity, PLS-GS places primitives closer to the LiDAR measurements, with comparable rendering quality. Code is available at https://github.com/k-harikumar/PLS-GS.
BibTeX
@misc{harikumar2026pls,
  author = {Krishnan Harikumar and Alejandro Fontan and Thierry Peynot and Emili Hernández and Niko Sünderhauf},
  title = {{PLS-GS: Probabilistic Lidar Scaffolds for Compact, Surface-Aligned Gaussian Splatting}},
  year = {2026},
  eprint = {2611.10699},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10699},
}
Figure