New opportunities in SLAM-Gaussian splatting technology

Zhen Tan, Zhenmin Niu, Jingpu Zhang, Xieyuanli Chen, Dewen HuJournal of Image and Graphics 2025 · DOI

Abstract

Simultaneous Localization and Mapping (SLAM) is a cornerstone of autonomous robotics and driving. This paper reviews the transition from traditional feature-based SLAM to modern learning-based approaches. While Neural Radiance Fields (NeRF) significantly improved visual fidelity, they suffered from slow rendering speeds that hindered real-time application. 3D Gaussian Splatting (3DGS) has recently emerged as a breakthrough, offering real-time, photorealistic reconstruction. We categorize current GS-SLAM methods by data modality, discuss their performance gains, and outline remaining challenges like memory management and scalability.

The Evolution of SLAM

For decades, SLAM research has navigated a tension between computational efficiency and the richness of the generated map. Early methods relied on sparse point clouds, which were sufficient for navigation but lacked visual detail. The advent of deep learning introduced Neural Radiance Fields (NeRF), which provided stunningly dense and realistic maps but at the cost of real-time performance.

The evolution of SLAM technology from early 1980s feature-based methods to the recent rise of 3D Gaussian Splatting in 2023.

As shown in the timeline above, the field has rapidly shifted toward explicit representations like 3D Gaussian Splatting to overcome the "slow rendering" bottleneck of implicit neural networks. This transition represents a move toward "explicit" geometry that is easier to edit and faster to render.

Technical Foundations of GS-SLAM

The core innovation of 3D Gaussian Splatting is representing a 3D scene as a collection of millions of explicit ellipsoids (Gaussians). Unlike NeRF, which stores scene information in the weights of a neural network, 3DGS stores properties—position, rotation, scale, opacity, and color—directly on these ellipsoids.

The standard pipeline for GS-SLAM, including initialization, scene representation, tracking, and differentiable rendering.

The pipeline typically involves several key stages:

  • 3D Gaussian Initialization: Starting from a sparse point cloud or depth map.
  • Differentiable Rendering: Projecting 3D Gaussians onto a 2D image plane using a tile-based rasterizer, allowing for fast optimization via backpropagation.
  • Adaptive Density Control: Dynamically splitting or pruning Gaussians to match scene complexity and balance performance.
  • Tracking and Mapping: Optimizing camera poses and Gaussian parameters simultaneously to minimize the difference between rendered views and actual camera frames.

Current Research Landscape

Researchers are currently adapting GS-SLAM for various inputs and environments:

  • RGB/RGB-D Methods: Systems like SplaTAM and GS-SLAM focus on indoor environments using standard camera data, often employing "coarse-to-fine" tracking strategies.
  • Multimodal Integration: Newer approaches are pushing the field toward fusing RGB data with LiDAR and IMU (as seen in the MM3DGS SLAM dataset) for increased robustness in challenging conditions.
  • Semantic SLAM: By embedding semantic labels into Gaussian properties, robots can achieve a higher-level "understanding" of their environment, distinguishing between objects in 3D space.

Key Results and Performance

The shift to 3DGS has yielded dramatic improvements across the literature:

  • Rendering Speed: Specialized systems like EndoGSLAM have achieved speeds over 100 FPS, making them viable for high-stakes applications like medical endoscopy.
  • Large-Scale Mapping: RTG-SLAM has demonstrated the ability to reconstruct large environments at over 17 FPS, proving that GS-SLAM is scalable beyond small rooms.
  • Visual Fidelity: GS-SLAM consistently achieves higher PSNR (Peak Signal-to-Noise Ratio) and SSIM scores compared to traditional dense mapping techniques, producing maps that are virtually indistinguishable from photographs.

Challenges and Future Directions

Despite its promise, GS-SLAM faces several critical hurdles:

  1. Memory Efficiency: Storing millions of explicit Gaussians for city-scale environments requires significant VRAM, posing a challenge for mobile hardware.
  2. Catastrophic Forgetting: In incremental mapping, optimizing for new areas can sometimes inadvertently degrade the representation of previously visited locations.
  3. Standardization: The field lacks unified benchmarks to fairly compare tracking accuracy, rendering speed, and memory usage across different hardware and environments.

Future research is expected to focus on CUDA kernel optimization for even faster processing and the integration of Large Vision Models (LVMs) to provide SLAM systems with semantic "common sense" for better scene understanding.

Summary updated

Abstract

同步定位与建图(simultaneous localization and mapping, SLAM)是指在未知环境中同时实现自主移动机器人的定位和环境地图构建,其在机器人技术和自动驾驶等领域有着重要价值。本文首先回顾SLAM技术的发展历程,从早期的手工特征提取方法到现代的深度学习驱动的解决方案。其中,基于神经辐射场(neural radiance fields, NeRF)的SLAM方法利用神经网络进行场景表征,进一步提高了建图的可视化效果。然而,这类方法在渲染速度上仍然面临挑战,限制了其实时应用的可能性。相比之下,基于高斯溅射(Gaussian splatting, GS)的SLAM方法以其实时的渲染速度和照片级的场景渲染效果,为SLAM领域带来新的研究热点和机遇。接着,按照RGB/RGBD、多模态数据以及语义信息3种不同应用类型对基于高斯溅射的SLAM方法进行分类和总结,并针对每种情况讨论相应SLAM方法的优势和局限性。最后,针对当前基于高斯溅射的SLAM方法面临的实时性、基准一致化、大场景的扩展性以及灾难性遗忘等问题进行分析,并对未来研究方向进行展望。通过这些探讨和分析,旨在为SLAM领域的研究人员和工程师提供全面的视角和启发,帮助分析和理解当前SLAM系统面临的关键问题,推动该领域的技术进步和应用拓展。
BibTeX
@article{tan2025new,
  author = {Zhen Tan and Zhenmin Niu and Jingpu Zhang and Xieyuanli Chen and Dewen Hu},
  title = {{New opportunities in SLAM-Gaussian splatting technology}},
  journal = {Journal of Image and Graphics},
  year = {2025},
  doi = {10.11834/jig.240443},
}
Figure