Skill generalization of tubular object manipulation with tactile sensing and Sim2Real learning

Yongqiang Zhao, Xingshuo Jing, Kun Qian, Daniel Fernandes Gomes, Shan Luo

Abstract

Highlights•A novel tactile-motor policy framework is proposed for learning robotic skills of tubular object manipulation that are generalizable toward Sim2Real transfer and different tubular objects.•A tactile image-based Sim2Real in-hand pose estimation method for tubular objects is proposed, the key of which is CTF-CycleGAN, a novel pixel-level unsupervised domain adaptation network that cascades CNNs and Transformer.•The proposed approach is implemented for learning Sim2Real transferable robotic insert-and-pullout actions, which are further evaluated in a human-robot collaborative tube placing scenario and a robotic pipetting scenario.
BibTeX
@article{zhao2023skill,
  author = {Yongqiang Zhao and Xingshuo Jing and Kun Qian and Daniel Fernandes Gomes and Shan Luo},
  title = {{Skill generalization of tubular object manipulation with tactile sensing and Sim2Real learning}},
  journal = {Robotics Auton. Syst.},
  year = {2023},
  doi = {10.1016/j.robot.2022.104321},
}

Loading related content...

Figure