Position: Scaling Simulation is Neither Necessary Nor Sufficient for In-the-Wild Robot Manipulation

Homanga BharadhwajICML 2024 · OpenReview

Introduction

In fields like Computer Vision and Natural Language Processing, scaling data and compute has led to highly capable "zero-shot" foundation models like GPT-4. Robotics, however, has noticeably lagged behind. The dominant response within the research community has been to scale up physics simulations—creating increasingly complex, parallelized virtual environments to train robots before deploying them in the real world (the Sim2Real approach).

This position paper challenges that prevailing paradigm. It argues that massive physics simulations are neither necessary nor sufficient for achieving general-purpose, human-compliant robot manipulation. Instead, the author advocates for a fundamental shift toward learning from diverse real-world data and passive human videos.

The Fallacy of "Perfect" Simulation

The core argument against scaling simulation rests on what the author terms the "Harder Problem" fallacy. To simulate a task like pouring water perfectly, a system must solve complex fluid dynamics and particle interactions. Paradoxically, this is actually a harder computational problem than the physical manipulation task itself. Humans successfully pour water every day without possessing a perfect internal physics engine; we rely on coarse reasoning, intuition, and continuous sensory feedback.

Furthermore, high success rates in simulation (e.g., achieving 90% accuracy at opening virtual drawers) frequently fail to correlate with real-world success. This "sim-to-real gap" occurs because simulators often lack the right parameters, rather than just needing more of them. Even with techniques like Domain Randomization, a simulator that is fundamentally inaccurate cannot simply "randomize" its way into reflecting reality.

Deformable objects like cloth present massive challenges for accurate physics simulation.

Real-World Desiderata: Zero-Shot Deployment and Human Preferences

The paper evaluates the "Scaling Simulation" approach against two critical requirements for widely usable, "in-the-wild" robots:

  1. Zero-Shot Deployment: A robot must be able to operate out-of-the-box in a novel environment without requiring extensive fine-tuning. The chronic lack of generalization in robotics is highlighted by the "Green Background" problem—an anecdote noting that in early robotics papers, one could guess the research university simply by the background color of the lab. Simulation has yet to solve this deep generalization issue.
  2. Compliance with Human Preferences: Robots must perform tasks safely and in alignment with human expectations. For example, a robot should know to move a laptop out of the way before carrying a full cup of coffee over a desk. These subtle, task-dependent safety preferences are incredibly difficult to program into a simulator's reward function.
Human preferences, such as avoiding spilling liquids on expensive electronics, are difficult to encode in simulation reward functions.

An Alternative Path: Real-World and Passive Data

Instead of engineering perfect simulators, the paper proposes leveraging the abundance of human activity data. The author points to recent successes in video pre-training, where robots learn useful manipulation priors directly from passive web videos (like YouTube tutorials). Because human preferences and safety constraints are implicitly present in videos of humans acting in the real world, this data provides a richer, more accurate training signal.

Additionally, the paper champions democratized real-world data collection. Tools like ALOHA enable high-quality real-world data gathering via low-cost teleoperation, and initiatives like the Open-X Embodiment project demonstrate the viability of training across diverse, real-world robot datasets.

Limitations and the Road Ahead

The author acknowledges that simulation is not entirely obsolete; it remains an excellent tool for prototyping algorithms, such as reinforcement learning agents for games. However, transitioning fully to a "Real-World" learning path presents its own hurdles:

  • The Human-Robot Gap: Learning directly from human video is complicated by the "correspondence problem"—human hands and robot grippers have fundamentally different embodiments.
  • Data Scarcity: While passive web video is abundant, high-quality, robot-specific action data remains harder to acquire than simulated data.
  • Safety Risks: Training directly in the physical world carries inherent risks of hardware damage that simulation safely avoids.

Ultimately, the paper serves as a call to action. It urges the robotics community to redirect its collective resources away from building a "Matrix-like" perfect simulation of reality, and toward developing foundation models grounded in the vast, messy, and rich reality of human data.

Summary updated

Abstract

In this paper, we develop a structured critique of robotic simulations for real-world manipulation, by arguing that scaling simulators is neither necessary nor sufficient for making progress in general-purpose real-world robotic manipulation agents that are compliant with human preferences. With the ubiquity of robotic simulators, and recent efforts to scale them for diverse tasks, and at the same time the interest in generally capable real-world manipulation systems, we believe it is important to address the limitations of using simulation for real-world manipulation, so that as a community, we can focus our collective resources, energy, and time on approaches that have more principled odds of success. We further demonstrate the unique challenges that real-world manipulation presents, and show through examples and arguments why scaling simulation doesn't get us closer to solving these challenges required for diverse real-world deployment.
BibTeX
@inproceedings{bharadhwaj2024position,
  author = {Homanga Bharadhwaj},
  title = {{Position: Scaling Simulation is Neither Necessary Nor Sufficient for In-the-Wild Robot Manipulation}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=Jtjurj7oIJ},
}
Figure