Sim-and-Human Co-training for Data-Efficient and Scene-Generalizable Bimanual Manipulation
Lists
Only you see this
Abstract
Real-robot demonstrations are prohibitively expensive, while simulation data and real-world human demonstrations are both scalable but each leaves a distinct gap—simulation suffers from a \textit{sim-to-real visual gap}, and human data suffers from a \textit{human-to-robot embodiment gap}. In this work, we identify a natural yet underexplored complementarity between these sources: simulation contributes robot-valid actions absent in human data, while human data provides real-world observations that simulation struggles to render. Building on this insight, we present \textbf{SimHum}, a co-training recipe that extracts kinematic priors from simulation and visual priors from human observations, then fine-tunes on a small real-robot dataset. \rev{SimHum exhibits strong scene-generalizable and data-efficient capabilities. With only $\mathbf{80}$ real-robot episodes per task, it achieves $\mathbf{62.5\%}$ success on held-out OOD scenes across four bimanual tabletop tasks, $\mathbf{53.7\%}$ higher than Real only in absolute success rate. Moreover, in a controlled data-collection study with matched collection time, SimHum improves over the best single-source pre-training baseline by $\mathbf{35.0\%}$ in absolute success rate.} More information at our project page\footnote{Project page: \url{https://kaipengfang.github.io/sim-and-human/}}.
BibTeX
@misc{fang2026sim,
author = {Kaipeng Fang and Weiqing Liang and Yuyang Li and Pengpeng Zeng and Heng Tao Shen and Jingkuan Song and Lianli Gao},
title = {{Sim-and-Human Co-training for Data-Efficient and Scene-Generalizable Bimanual Manipulation}},
year = {2026},
eprint = {2601.19406},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2601.19406},
}