Dense 3D semantic mapping of indoor scenes from RGB-D images

Alexander Hermans, Georgios Floros, Bastian Leibe

Abstract

Dense semantic segmentation of 3D point clouds is a challenging task. Many approaches deal with 2D semantic segmentation and can obtain impressive results. With the availability of cheap RGB-D sensors the field of indoor semantic segmentation has seen a lot of progress. Still it remains unclear how to deal with 3D semantic segmentation in the best way. We propose a novel 2D-3D label transfer based on Bayesian updates and dense pairwise 3D Conditional Random Fields. This approach allows us to use 2D semantic segmentations to create a consistent 3D semantic reconstruction of indoor scenes. To this end, we also propose a fast 2D semantic segmentation approach based on Randomized Decision Forests. Furthermore, we show that it is not needed to obtain a semantic segmentation for every frame in a sequence in order to create accurate semantic 3D reconstructions. We evaluate our approach on both NYU Depth datasets and show that we can obtain a significant speed-up compared to other methods.
BibTeX
@inproceedings{hermans2014dense,
  author = {Alexander Hermans and Georgios Floros and Bastian Leibe},
  title = {{Dense 3D semantic mapping of indoor scenes from RGB-D images}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2014},
  doi = {10.1109/icra.2014.6907236},
}

Loading related content...

Figure