Tracking objects with point clouds from vision and touch

Gregory Izatt, Geronimo Mirano, Edward H. Adelson, Russ TedrakeICRA 2017 · OpenReview

Abstract

We present an object-tracking framework that fuses point cloud information from an RGB-D camera with tactile information from a GelSight contact sensor. GelSight can be treated as a source of dense local geometric information, which we incorporate directly into a conventional point-cloud-based articulated object tracker based on signed-distance functions. Our implementation runs at 12 Hz using an online depth reconstruction algorithm for GelSight and a modified second-order update for the tracking algorithm. We present data from hardware experiments demonstrating that the addition of contact-based geometric information significantly improves the pose accuracy during contact, and provides robustness to occlusions of small objects by the robot's end effector.
BibTeX
@inproceedings{izatt2017tracking,
  author = {Gregory Izatt and Geronimo Mirano and Edward H. Adelson and Russ Tedrake},
  title = {{Tracking objects with point clouds from vision and touch}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2017},
  url = {https://openreview.net/forum?id=eL6mxnUU9qR},
}

Loading related content...

Figure