Abstract
Collecting high-quality real-world demonstrations for robotic imitation learning is a significant bottleneck in scaling autonomous systems. This study investigates sim-and-real cotraining—the simultaneous training of models on both expensive real-world data and scalable simulated data—as a solution to data scarcity. Using a planar pushing task as a benchmark, we demonstrate that cotraining Diffusion Policies can improve success rates by up to 7x in low-data regimes. Our findings reveal a "discernability paradox": policies perform better when they can distinguish between simulation and reality, allowing them to adapt to differing physical dynamics. This suggests that physical fidelity in simulators is more critical than visual photorealism for contact-rich manipulation tasks.
Introduction
Imitation learning (IL) has enabled robots to perform complex tasks by mimicking human demonstrations. However, IL is notoriously data-hungry, often requiring thousands of trajectories to achieve robustness. While simulators offer a source of infinite data, the "sim-to-real gap"—discrepancies in physics and visual appearance—frequently prevents policies trained solely in simulation from transferring to the real world.
Sim-and-real cotraining has emerged as a practical technique to bridge this gap. By mixing real-world demonstrations ($D_R$) with simulated data ($D_S$), researchers aim to leverage the coverage of simulation while anchoring the policy in real-world physics. This paper provides an empirical analysis of this recipe, exploring the underlying mechanisms that drive its success.
Methodology
Task and Model Architecture
The study focuses on planar pushing from pixels, where a KUKA LBR iiwa 7 robot must push a T-shaped object to a target pose based on visual feedback. The researchers employed Diffusion Policy, a generative model that learns to map observations to action sequences by iteratively denoising a signal. A ResNet18 backbone was used for visual feature extraction.
Cotraining Framework
Cotraining was implemented by mixing real and simulated trajectories into a single training stream. The balance between these datasets was controlled by a mixing ratio, $\alpha$. To facilitate the massive scale of evaluation required for statistical significance (exceeding 50,000 trials), the authors developed a "target sim" surrogate. This environment served as a high-throughput proxy for the real world, featuring distinct lighting and physics parameters compared to the source simulation.
Key Insights: The Discernability Paradox
A central finding of this research is the discernability paradox. Conventional wisdom suggests that minimizing the visual gap between simulation and reality (domain randomization or photorealism) is the best path to transfer. However, this study found that a small visual gap is actually beneficial.
Because simulators rarely match real-world physics perfectly, a policy must identify which domain it is currently operating in to apply the appropriate physical model. When visual differences were artificially removed without correcting physical discrepancies, performance declined. Internal probes confirmed that the model’s embeddings could distinguish between sim and real domains with 100% accuracy, proving that the policy learns domain-specific behaviors rather than a single, averaged strategy.
Results and Scaling Laws
Performance Gains
Cotraining showed the most dramatic impact in low-data regimes. With only 10 real-world demonstrations, a policy trained only on real data achieved a 10% success rate. Adding simulated data through cotraining boosted this success rate to 70%, a 7x improvement. Overall, cotraining with simulation was found to be equivalent to increasing the real-world dataset size by 2-3x.
Physics vs. Pixels
The study compared the impact of visual photorealism versus physical accuracy. Improving the physical fidelity of the simulator (e.g., friction models and contact dynamics) resulted in a 15.5% boost in success rates. In contrast, "perfect" visual rendering provided negligible gains, reinforcing the idea that for contact-rich tasks, the simulator's "brain" matters more than its "eyes."
Data Coverage and Anchoring
Cotraining works by using simulation to provide "coverage"—exposing the model to a wide variety of states and recovery maneuvers that may not be present in limited real-world datasets. The real-world data then acts as an "anchor," defining the successful strategy for the specific physics of the deployment environment. While adding more sim data improves performance, it eventually hits a plateau; raising the performance ceiling further requires additional real-world data.
Conclusion
This empirical analysis confirms that sim-and-real cotraining is a powerful tool for robotic learning. By allowing the model to discern and adapt to the differences between domains, cotraining overcomes the limitations of the sim-to-real gap. Future work will investigate whether these scaling laws and the discernability paradox hold true for multi-task learning and more complex, long-horizon manipulation tasks.
Summary updated