Abstract
We approach the problem of learning by watching humans in the wild. While traditional approaches in Imitation and Reinforcement Learning are promising for learning in the real world, they are either sample inefficient or are constrained to lab settings. Meanwhile, there has been a lot of success in processing passive, unstructured human data. We propose tackling this problem via an efficient one-shot robot learning algorithm, centered around learning from a third-person perspective. We call our method WHIRL: In-the-Wild Human Imitating Robot Learning. WHIRL extracts a prior over the intent of the human demonstrator, using it to initialize our agent's policy. We introduce an efficient real-world policy learning scheme that improves using interactions. Our key contributions are a simple sampling-based policy optimization approach, a novel objective function for aligning human and robot videos as well as an exploration method to boost sample efficiency. We show one-shot generalization and success in real-world settings, including 20 different manipulation tasks in the wild.
Introduction
The dream of robotics has always been to create agents that can seamlessly integrate into human environments—kitchens, offices, and homes. However, the path to this reality is often hindered by the "lab gap." Most current robotic learning methods rely on massive amounts of data collected in controlled settings, often requiring expensive teleoperation or kinesthetic teaching. These methods fail to scale because they cannot leverage the vast repository of human activity already captured in passive video data.
In this work, we introduce WHIRL (In-the-Wild Human Imitating Robot Learning), a framework designed to bridge the gap between passive human observation and active robotic execution. By watching a single video of a human performing a task in a natural environment, WHIRL enables a robot to initialize a policy and refine it through targeted interaction.
The Challenge: Embodiment Mismatch
The primary obstacle in learning from human videos is the "embodiment mismatch." A robot’s end-effector (like a gripper) does not share the same geometry, degrees of freedom, or visual appearance as a human hand. Furthermore, human videos lack "action labels"—we see the hand move, but we do not know the specific motor torques applied to achieve that movement.
To overcome this, WHIRL shifts the focus from how the agent moves to what happens to the environment. By focusing on the change in the state of objects, the system can bypass the physical differences between humans and robots.
The WHIRL Framework: Watch, Improve, Repeat
WHIRL operates in three distinct phases to transform a human demonstration into a robust robotic policy.
1. Watch: Extracting Visual Priors
The process begins with a single video of a human performing a task. Using off-the-shelf computer vision models such as 100 Days of Hands (100DOH) and MANO, WHIRL tracks the human hand and its interaction with objects. It identifies key waypoints—the start point, the interaction point, and the end point—to create a "prior" trajectory. This trajectory serves as a "warm start" for the robot, providing a safe and sensible initialization for real-world exploration.
2. Improve: Agent-Agnostic Alignment
To handle the visual differences between humans and robots, WHIRL employs a technique called Inpainting. Using Copy-Paste Networks, the system digitally removes the human or the robot from the video frames. By filling in the background, the reward function is forced to focus entirely on the state of the environment (e.g., "Is the door open?"). This creates an "agent-agnostic" representation space where human and robot videos can be directly compared based on their environmental impact.
3. Repeat: Policy Optimization
The robot does not simply mimic the human prior. Instead, it uses a sampling-based optimization approach (similar to the Cross-Entropy Method). The robot attempts several variations of the task, evaluates them using the agent-agnostic reward, and identifies "elite" samples. These successful trajectories are then used to fit a Conditional Variational Auto-Encoder (CVAE), allowing the robot to learn a multi-modal policy that is resilient to noise and environmental changes.
Experimental Results
The researchers evaluated WHIRL on 20 diverse manipulation tasks in real-world settings, including opening dishwashers, pulling garbage bags, folding clothes, and cleaning whiteboards.
Performance Gains
The results demonstrate the power of iterative learning. In tasks like opening a drawer, the initial success rate based solely on the human prior was approximately 43%. However, after just two iterations of the WHIRL improvement cycle, the success rate rose to 83%. For door-opening tasks, the system achieved a 92% success rate. This highlights that while the human prior is a necessary starting point, real-world interaction is the key to mastery.
Comparison to Baselines
WHIRL significantly outperformed traditional methods. Standard Behavior Cloning (BC) and state-of-the-art offline Reinforcement Learning (such as CQL) struggled to learn from the limited, high-dimensional data available in unstructured environments. WHIRL’s ability to use the human video as a prior and then refine it through interaction proved to be the key differentiator.
Key Insights
- Environment Change > Agent Movement: The most effective way to bridge the embodiment gap is to prioritize the effect on the environment over the specific movements of the agent.
- Interaction is Essential: You cannot learn perfectly from a single video due to sensor noise and morphology differences. Interaction bridges the final gap to success.
- Residual Learning: Learning a "correction" to a human prior is far more efficient and safer for real-world deployment than learning a policy from scratch.
Limitations and Future Work
While WHIRL represents a significant step forward, challenges remain. The inpainting process is currently a computational bottleneck, taking several hours to process videos for a single task. Additionally, hardware constraints—such as the torque limits of the Hello Robot Stretch—prevented the robot from performing tasks involving heavy objects.
The ultimate goal is to move toward a "YouTube-to-Robot" pipeline, where robots can acquire complex skills directly from uncurated online video data without requiring any online interaction. WHIRL provides the foundation for this future by demonstrating that robots can indeed learn "in the wild."
Summary updated