Robotics: Science and Systems XI

Thomas J. Whelan, Stefan Leutenegger, Renato F. Salas-Moreno, Ben Glocker, Andrew J. Davison, J Chen, S Izadi, A Fitzgibbon, D Bautembach, N Molton, I Reid, O Stasse, F Endres, J Hess, N Engelhard, J Sturm, D Cremers, W Burgard, J Shotton, A Criminisi, A Handa, J Mcdonald, P Henry, D Fox, A Bhowmik, R Mongia, M Keller, D Lefloch, M Lambers, T Weyrich, A Kolb, C Kerl, G Klein, D Murray, K Konolige, M, M Kaess, C Cadena, J Neira, J Leonard, M Meilland, A Comport, R Newcombe, O Hilliges, D Molyneaux, D Kim, P Kohli, S Hodges, S Lovegrove, P Kelly, F Steinbrucker, H Strasdat, J Montiel, J Stckler, S Behnke, R Sumner, J Schmid, M Pauly, M Fallon, H JohannssonOpenalex Unknown 2015 · DOI · OpenAlex

Abstract

We present a novel approach to real-time dense visual SLAM. Our system is capable of capturing comprehensive dense globally consistent surfel-based maps of room scale environments explored using an RGB-D camera in an incremental online fashion, without pose graph optimisation or any postprocessing steps. This is accomplished by using dense frame-to-model camera tracking and windowed surfel-based fusion coupled with frequent model refinement through non-rigid surface deformations. Our approach applies local model-to-model surface loop closure optimisations as often as possible to stay close to the mode of the map distribution, while utilising global loop closure to recover from arbitrary drift and maintain global consistency.

Introduction

Simultaneous Localization and Mapping (SLAM) is a cornerstone of robotics, yet traditional methods often struggle with dense data. Most systems rely on a pose graph—a history of camera positions connected by constraints. While effective for sparse feature-based SLAM, pose graphs focus on optimizing the camera's trajectory rather than the map itself. This can lead to inconsistencies in the reconstructed 3D surface, especially during "loopy" motions where a user repeatedly traverses the same area.

ElasticFusion addresses this by introducing a map-centric approach that achieves global consistency without a traditional pose graph. By treating the map as an "elastic" surfel-based model and applying frequent non-rigid deformations, it captures room-scale environments with high accuracy in real-time.

A high-quality dense 3D reconstruction of a room environment produced by the ElasticFusion system.

Methodology

The core of ElasticFusion is its move away from rigid pose-to-pose constraints toward a flexible, surfel-based representation.

Surfel-based Mapping

The environment is represented as a collection of "surfels" (surface elements), each defined by a position, normal, color, and timestamp. This dense representation allows for significantly more detailed surface modeling compared to sparse point clouds.

Active vs. Inactive Map

To maintain real-time performance, the system partitions the map. The active area consists of surfels observed recently, while older, unobserved parts are moved to an inactive state. Tracking and fusion operations are restricted to the active area, preventing the computational cost from scaling linearly with the total map size as the environment grows.

The system distinguishes between active (green) and inactive (grey) map regions to maintain efficiency during tracking and fusion.

Joint Tracking and Deformation

The system tracks the camera by combining geometric alignment (Iterative Closest Point - ICP) with photometric alignment (RGB intensity). This dual approach provides robustness across various structural and textural environments.

The "elastic" nature of the system comes from a sparse deformation graph embedded directly into the surface.

  • Local Loop Closures: The system continuously attempts to align the active model with the inactive model. Overlaps trigger non-rigid deformations that "snap" the surfaces together.
  • Global Loop Closures: Using randomized ferns for appearance-based place recognition, the system detects when the camera returns to a previously mapped area after significant drift.
  • Non-Rigid Optimization: Upon closing a loop, the system optimizes the deformation graph to restore global consistency across the entire map without re-processing the full camera trajectory.

Evaluation and Results

ElasticFusion was evaluated using the TUM RGB-D and ICL-NUIM benchmarks, alongside new room-scale sequences like "Copy Room" and "Hotel Room."

Accuracy

The system achieved trajectory accuracy (ATE RMSE) comparable to or exceeding state-of-the-art pose-graph systems like DVO SLAM and Kintinuous. On the synthetic ICL-NUIM dataset, it produced surface reconstructions with mean errors as low as 0.7cm.

Performance

Running on a desktop PC with an NVIDIA GTX 780 Ti, the system maintains a frame rate of 22Hz to 31Hz. This real-time capability is essential for interactive mapping applications where immediate feedback is required.

Discussion

ElasticFusion demonstrates that dense data can overpower the need for traditional probabilistic filtering. By treating the map as the "source of truth," the system handles complex trajectories more elegantly than trajectory-centric methods. The use of temporal connectivity in the deformation graph is a key insight, preventing temporally distant map sections from incorrectly influencing each other until a loop closure is verified.

However, scalability remains a challenge. The surfel-based approach is memory-intensive, making multi-room or building-scale mapping difficult without more advanced memory management. Additionally, the current implementation assumes a static environment, which can be problematic in dynamic real-world settings. Future work will explore extending these elastic mapping principles to larger scales and dynamic scenes.

Summary updated

Abstract

We present a novel approach to real-time dense visual SLAM.Our system is capable of capturing comprehensive dense globally consistent surfel-based maps of room scale environments explored using an RGB-D camera in an incremental online fashion, without pose graph optimisation or any postprocessing steps.This is accomplished by using dense frame-tomodel camera tracking and windowed surfel-based fusion coupled with frequent model refinement through non-rigid surface deformations.Our approach applies local model-to-model surface loop closure optimisations as often as possible to stay close to the mode of the map distribution, while utilising global loop closure to recover from arbitrary drift and maintain global consistency.
BibTeX
@inproceedings{whelan2015robotics,
  author = {Thomas J. Whelan and Stefan Leutenegger and Renato F. Salas-Moreno and Ben Glocker and Andrew J. Davison and J Chen and S Izadi and A Fitzgibbon and J Chen and D Bautembach and S Izadi and A Davison and A Davison and N Molton and I Reid and O Stasse and F Endres and J Hess and N Engelhard and J Sturm and D Cremers and W Burgard and B Glocker and J Shotton and A Criminisi and S Izadi and A Handa and T Whelan and J Mcdonald and A Davison and P Henry and D Fox and A Bhowmik and R Mongia and M Keller and D Lefloch and M Lambers and S Izadi and T Weyrich and A Kolb and C Kerl and J Sturm and D Cremers and G Klein and D Murray and K Konolige and M and J Mcdonald and M Kaess and C Cadena and J Neira and J Leonard and M Meilland and A Comport and R Newcombe and S Izadi and O Hilliges and D Molyneaux and D Kim and A Davison and P Kohli and J Shotton and S Hodges and A Fitzgibbon and R Newcombe and S Lovegrove and A Davison and R Salas-Moreno and B Glocker and P Kelly and A Davison and F Steinbrucker and C Kerl and J Sturm and D Cremers and F Steinbrucker and J Sturm and D Cremers and H Strasdat and A Davison and J Montiel and K Konolige and J Stckler and S Behnke and J Sturm and N Engelhard and F Endres and W Burgard and D Cremers and R Sumner and J Schmid and M Pauly and T Whelan and J Mcdonald and M Kaess and M Fallon and H Johannsson and J Leonard and T Whelan and M Kaess},
  title = {{Robotics: Science and Systems XI}},
  booktitle = {Openalex Unknown},
  year = {2015},
  doi = {10.15607/rss.2015.xi},
}
Figure