Empirical Analysis of Sim-and-Real Cotraining of Diffusion Policies for Planar Pushing from Pixels

Adam Wei, Abhinav Agarwal, Boyuan Chen, Rohan Bosworth, Nicholas Ezra Pfaff, Russ Tedrake

Abstract

Collecting high-quality real-world demonstrations for robotic imitation learning is a significant bottleneck in scaling autonomous systems. This study investigates sim-and-real cotraining—the simultaneous training of models on both expensive real-world data and scalable simulated data—as a solution to data scarcity. Using a planar pushing task as a benchmark, we demonstrate that cotraining Diffusion Policies can improve success rates by up to 7x in low-data regimes. Our findings reveal a "discernability paradox": policies perform better when they can distinguish between simulation and reality, allowing them to adapt to differing physical dynamics. This suggests that physical fidelity in simulators is more critical than visual photorealism for contact-rich manipulation tasks.

Introduction

Imitation learning (IL) has enabled robots to perform complex tasks by mimicking human demonstrations. However, IL is notoriously data-hungry, often requiring thousands of trajectories to achieve robustness. While simulators offer a source of infinite data, the "sim-to-real gap"—discrepancies in physics and visual appearance—frequently prevents policies trained solely in simulation from transferring to the real world.

Sim-and-real cotraining has emerged as a practical technique to bridge this gap. By mixing real-world demonstrations ($D_R$) with simulated data ($D_S$), researchers aim to leverage the coverage of simulation while anchoring the policy in real-world physics. This paper provides an empirical analysis of this recipe, exploring the underlying mechanisms that drive its success.

Methodology

Task and Model Architecture

The study focuses on planar pushing from pixels, where a KUKA LBR iiwa 7 robot must push a T-shaped object to a target pose based on visual feedback. The researchers employed Diffusion Policy, a generative model that learns to map observations to action sequences by iteratively denoising a signal. A ResNet18 backbone was used for visual feature extraction.

Cotraining Framework

Cotraining was implemented by mixing real and simulated trajectories into a single training stream. The balance between these datasets was controlled by a mixing ratio, $\alpha$. To facilitate the massive scale of evaluation required for statistical significance (exceeding 50,000 trials), the authors developed a "target sim" surrogate. This environment served as a high-throughput proxy for the real world, featuring distinct lighting and physics parameters compared to the source simulation.

Key Insights: The Discernability Paradox

A central finding of this research is the discernability paradox. Conventional wisdom suggests that minimizing the visual gap between simulation and reality (domain randomization or photorealism) is the best path to transfer. However, this study found that a small visual gap is actually beneficial.

Because simulators rarely match real-world physics perfectly, a policy must identify which domain it is currently operating in to apply the appropriate physical model. When visual differences were artificially removed without correcting physical discrepancies, performance declined. Internal probes confirmed that the model’s embeddings could distinguish between sim and real domains with 100% accuracy, proving that the policy learns domain-specific behaviors rather than a single, averaged strategy.

Results and Scaling Laws

Performance Gains

Cotraining showed the most dramatic impact in low-data regimes. With only 10 real-world demonstrations, a policy trained only on real data achieved a 10% success rate. Adding simulated data through cotraining boosted this success rate to 70%, a 7x improvement. Overall, cotraining with simulation was found to be equivalent to increasing the real-world dataset size by 2-3x.

Physics vs. Pixels

The study compared the impact of visual photorealism versus physical accuracy. Improving the physical fidelity of the simulator (e.g., friction models and contact dynamics) resulted in a 15.5% boost in success rates. In contrast, "perfect" visual rendering provided negligible gains, reinforcing the idea that for contact-rich tasks, the simulator's "brain" matters more than its "eyes."

Data Coverage and Anchoring

Cotraining works by using simulation to provide "coverage"—exposing the model to a wide variety of states and recovery maneuvers that may not be present in limited real-world datasets. The real-world data then acts as an "anchor," defining the successful strategy for the specific physics of the deployment environment. While adding more sim data improves performance, it eventually hits a plateau; raising the performance ceiling further requires additional real-world data.

Conclusion

This empirical analysis confirms that sim-and-real cotraining is a powerful tool for robotic learning. By allowing the model to discern and adapt to the differences between domains, cotraining overcomes the limitations of the sim-to-real gap. Future work will investigate whether these scaling laws and the discernability paradox hold true for multi-task learning and more complex, long-horizon manipulation tasks.

Summary updated

Abstract

Cotraining with demonstrations generated both in simulation and on real hardware has emerged as a promising recipe for scaling imitation learning in robotics. This work seeks to elucidate basic principles of this sim-and-real cotraining to inform simulation design, sim-and-real dataset creation, and policy training. Our experiments confirm that cotraining with simulated data can dramatically improve performance, especially when real data is limited. We show that performance gains scale with additional simulated data until a plateau; adding more real data can increase this performance ceiling. The results also suggest that physical domain gaps may be more impactful than visual fidelity for contact-rich tasks. Perhaps surprisingly, some visual gap appears to help cotraining. We investigate this nuance and other mechanisms that facilitate positive transfer between sim-and-real. Lastly, we provide ablations for 2 alternative cotraining formulations. Videos can be found here: https://sim-and-real-cotraining.github.io/.
BibTeX
@inproceedings{wei2025empirical,
  author = {Adam Wei and Abhinav Agarwal and Boyuan Chen and Rohan Bosworth and Nicholas Ezra Pfaff and Russ Tedrake},
  title = {{Empirical Analysis of Sim-and-Real Cotraining of Diffusion Policies for Planar Pushing from Pixels}},
  booktitle = {Robot Data @ CoRL},
  year = {2025},
  url = {https://openreview.net/forum?id=kBzTJgYgol},
}
Figure