Keyframe-based Dense Mapping with the Graph of View-Dependent Local Maps

Krzysztof Zielinski, Dominik BelterICRA 2020

TL;DR

This paper introduces a novel dense 3D mapping system that leverages view-dependent local maps stored within a keyframe graph. This approach allows for more precise representation of objects closer to the robot and enables efficient global map correction after loop closure detection, outperforming traditional voxel-based methods in reconstruction accuracy and efficiency.

What Problem Does This Paper Address?

Creating accurate and efficient 3D maps of environments is crucial for autonomous robots to navigate, plan motions, and interact with their surroundings. Traditional dense mapping methods like Octomap and NDT-Occupancy Map (NDT-OM) divide 3D space into fixed-size voxels. This fixed resolution presents a dilemma: a coarse resolution is memory-efficient but sacrifices detail, while a fine resolution captures more detail but leads to massive memory consumption and computational cost. Furthermore, these methods struggle to adapt the map's precision to the varying accuracy of sensor data (e.g., RGB-D cameras, which are more precise for nearby objects). Correcting global maps after loop closure detection is also challenging, as dense measurements are already integrated.

Methods

The proposed system builds a dense 3D map using a keyframe-based approach, inspired by sparse SLAM methods like ORB-SLAM2. Instead of a single global map, it maintains a collection of "local maps," each associated with a unique camera view (a keyframe).

  • View-Dependent Local Maps: Each local map is represented by a 2D grid where each cell stores an "ellipsoid" (a Normal Distribution Transform). These ellipsoids capture the position, covariance (uncertainty), and color of 3D points projected onto the image plane. This view-dependent representation naturally allows for higher precision for objects closer to the camera, as they occupy more pixels and thus contribute to more detailed ellipsoids.
  • Keyframe Graph: Local maps (keyframes) are organized in a graph structure, similar to a covisibility graph. Edges connect keyframes that share overlapping views, and their weights represent the degree of co-visibility. This graph structure is crucial for loop closure detection.
  • NDT Update: The core of the local map update uses a Normal Distribution Transform (NDT) methodology. When a new RGB-D frame is processed, its point cloud is transformed into the keyframe's coordinate system. Points are then projected onto the 2D grid, grouped by cell, and used to update the corresponding ellipsoid's mean position, covariance, and color using iterative NDT update equations.
  • Ellipsoid Filtering: To remove spurious or less precise ellipsoids, a filtering step is applied. This procedure rejects ellipsoids that are too elongated in 3D space relative to the sensor's uncertainty model, prioritizing those that represent objects with higher precision.
  • Merging Local Maps: To construct a global map, local maps are merged. Neighboring local maps in the graph are considered. Ellipsoids from overlapping regions are re-projected into a common coordinate frame and then clustered using mean-shift clustering. Within each cluster, a single representative ellipsoid is computed by averaging their positions, covariances, and colors.

Datasets

The paper evaluates its method on several datasets from the TUM RGB-D dataset, specifically mentioning:

  • freiburg1_room
  • freiburg2_desk
  • freiburg3_long_office
  • freiburg3_long

The ICL-NUIM living_room dataset is also used for comparing reconstruction accuracy against a reference mesh model.

Key Insights and Ideas

  • View-Dependent Precision: The core insight is that by projecting 3D information onto the 2D image plane and using ellipsoids, the map's precision naturally adapts to the sensor's capabilities. Objects closer to the camera are represented with higher detail, which is more efficient and informative than fixed-resolution voxel grids.
  • Leveraging Sparse SLAM Structures for Dense Mapping: The paper effectively bridges the gap between sparse feature-based SLAM (like ORB-SLAM2) and dense mapping by using the keyframe graph and covisibility concepts for organizing and correcting dense local maps.
  • Efficient Global Map Correction: The keyframe graph structure allows for loop closure detection, which in turn enables the correction of the global map. This is a significant advantage over methods where dense data is immediately integrated into a global structure, making it harder to revise.
  • Reduced Data Redundancy: The merging and filtering procedures significantly reduce the number of ellipsoids in the global map compared to raw point clouds or voxel-based representations, leading to more compact and manageable global maps.

Key Results

  • Improved Reconstruction Accuracy: The proposed view-dependent method achieves significantly lower Root Mean Square Error (RMSE) in reconstruction tasks compared to Octomap, even with a much smaller number of geometric primitives (ellipsoids vs. voxels). For instance, with a 15x15 px cell size, the RMSE was 12.5 mm with 27,081 ellipsoids, whereas Octomap with a 0.02 m voxel size had an RMSE of 48.1 mm and over a million voxels.
  • Reduced Number of Primitives: The global map generated by the proposed method contains a significantly smaller number of ellipsoids compared to the number of voxels in Octomap or NDT-OM for comparable environments. For the fr3_long sequence, the global map had 358,258 ellipsoids after clustering and filtering, compared to 892,232 initial ellipsoids.
  • Faster Global Map Generation: The time to generate the global map was reduced by more than two times compared to previous approaches, with maximal computation time around 14.2 seconds for a graph with 161 local maps.
  • Qualitative Improvements: Visual comparisons show that the proposed method provides a 3D map with varying resolution, where details of objects are much better visible, especially for objects closer to the camera.

Limitations and Future Work

  • Limitations: The paper acknowledges that the local map's resolution is tied to the number of pixels used for updates, and the accuracy of objects further away is naturally lower. The merging process relies on re-projection, which might introduce some inaccuracies.
  • Future Work: The authors plan to integrate their keyframe-based dense mapping method with ORB-SLAM2, allowing direct sharing of data structures between the localization and mapping systems. They also aim to work on simultaneous integration of local maps into the global map and camera pose estimation.

Summary updated

BibTeX
@inproceedings{zielinski2020keyframe,
  author = {Krzysztof Zielinski and Dominik Belter},
  title = {{Keyframe-based Dense Mapping with the Graph of View-Dependent Local Maps}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2020},
}
Figure