Introduction
3D Gaussian Splatting (3DGS) has emerged as a transformative technique in computer vision, enabling high-fidelity, real-time rendering of complex scenes. However, its application in robotics has been hindered by a fundamental limitation: while 3DGS models are visually impressive, they often lack the metric accuracy and absolute scale required for precise navigation and localization. Most 3DGS frameworks rely on Structure-from-Motion (SfM) for initialization, which can lead to "geometric mush"—structures that appear correct from specific viewpoints but are spatially distorted or lack physical consistency.
To bridge this gap, we present GeomGS (Geometry-Aware Gaussian Splatting), a novel framework that integrates LiDAR data into the 3DGS optimization process. By introducing a "Geometric Confidence Score" (GCS), GeomGS balances photometric rendering with structural integrity, creating maps that are both photo-realistic and metrically precise.
The Challenge of Metric Accuracy
Integrating LiDAR into 3DGS is not as straightforward as it might seem. Simple distance constraints that force Gaussian primitives to align with LiDAR returns often backfire. LiDAR data can be noisy, especially on reflective surfaces or when capturing distant objects like the sky. If a model is forced to match these noisy points, rendering quality (PSNR) suffers. Conversely, prioritizing image quality often results in Gaussians "floating" in space to minimize pixel-wise error, rendering the map useless for metric localization.
Methodology: The Geometric Confidence Score (GCS)
The core innovation of GeomGS is the Geometric Confidence Score ($\gamma$), a per-Gaussian parameter learned during training. This score allows the model to differentiate between stable geometric structures and purely visual elements.
1. Probabilistic Mapping
Instead of treating LiDAR as a hard constraint, GeomGS employs a probabilistic distance loss. The GCS acts as a weight:
- High GCS: Assigned to Gaussians that consistently align with reliable LiDAR data (e.g., roads, building facades). The optimizer pulls these points tightly toward the LiDAR geometry.
- Low GCS: Assigned to Gaussians representing "visual fluff" (e.g., sky, reflections, thin foliage). For these points, the model prioritizes photometric (RGB) loss, allowing them to move freely to optimize image quality without corrupting the metric map.
2. Hybrid Localization Framework
GeomGS utilizes a dual-stage localization strategy to achieve robust pose estimation:
- Weighted ICP: A query LiDAR scan is aligned to the Gaussian map using GCS as weights. This ensures the alignment process ignores unreliable or transient Gaussians, focusing on the "trusted" geometry.
- Image-based Refinement: The pose is further fine-tuned by minimizing the photometric difference between the rendered view and the actual camera image.
- Iterative Optimization: These steps repeat, where the geometry handles large-scale corrections and the pixels provide sub-centimeter refinement.
Experimental Results
The framework was evaluated on the KITTI and KITTI-360 datasets, benchmarks for autonomous driving and outdoor SLAM.
Rendering and Geometric Fidelity
As shown in the comparison above, GeomGS significantly outperforms standard 3DGS and 2DGS. It achieved a PSNR of 24.30 on KITTI-360. More importantly, the geometric accuracy—measured by Chamfer Distance (CD)—improved dramatically, dropping from 261.18 in standard 3DGS to just 2.67 in GeomGS. This indicates that the "bones" of the scene are correctly placed in 3D space.
Localization Robustness
In scenarios with significant initial pose errors (up to 40° rotation and 4m translation), GeomGS maintained a translation error of only 2.70m. In contrast, traditional ICP and vision-only iNeRF methods frequently failed to converge or resulted in errors exceeding 9m.
Conclusion and Future Work
GeomGS demonstrates that geometric awareness does not have to come at the cost of visual quality. By intelligently weighting LiDAR constraints through the GCS, the framework produces maps that are "robot-ready"—suitable for both high-end visualization and high-precision localization. Future work will focus on extending this approach to dynamic environments, allowing the GCS to account for moving objects like vehicles and pedestrians.
Summary updated