Introduction
Training robots to perform robust manipulation tasks typically requires either massive amounts of real-world data or painstakingly hand-crafted simulation environments. PRISM (Projection-based Reward Integration for Scene-Aware Real-to-Sim-to-Real) introduces a novel pipeline that bridges this gap. By utilizing just a handful of real-world demonstrations, PRISM automatically constructs a simulation environment and leverages Vision-Language Models (VLMs) to generate reward signals from simple spatial prompts, eliminating the need for manual reward engineering.
The Challenge of Robust Manipulation
Robotic manipulation faces a significant hurdle: generalization. Traditional Imitation Learning (e.g., Behavior Cloning) is intuitive but brittle; a robot trained on limited data often fails when encountering novel object poses. Conversely, Reinforcement Learning (RL) excels at exploration and robustness but is unsafe to perform in the real world. While simulation offers a safe training ground, the "sim-to-real gap" and the complexity of coding mathematical reward functions for every new task remain major bottlenecks.
PRISM addresses these issues by asking: How can we take just five real-world demonstrations, automatically build a training sandbox, and learn a robust RL policy without hand-coding a reward function?
The PRISM Pipeline
PRISM employs a three-step Real-to-Sim-to-Real methodology designed for maximum data efficiency and robustness.
1. Automated Real-to-Sim Transfer
Starting with a single real-world image and 5–15 expert demonstrations, PRISM uses foundation models like Segment Anything (SAM) and 6D pose estimation to identify objects. It then retrieves matching 3D models from a library to reconstruct the scene in a simulator. Real-world demonstrations are mapped into this virtual environment, with object poses refined to ensure physical consistency (e.g., preventing floating or clipping).
2. Projection-Based Reward Modeling
Instead of manual coding, PRISM uses "projection relationships" to define success. Humans provide simple text prompts describing how objects should appear from specific camera angles (e.g., "the gripper should overlap the apple").
As shown in the pipeline overview, a VLM (such as QWen-72B or GPT-4V) evaluates multi-view images generated by the simulator. It provides binary labels based on the spatial prompts, which are then used to train a lightweight reward model. This multi-view approach is critical; VLMs often struggle with single-view real-world images due to occlusion, but the simulator allows for consistent, multi-angle verification.
3. Sim-to-Real Co-Training and Safety
To bridge the domain shift back to reality, the policy is fine-tuned by mixing real-world demonstrations into the RL replay buffer. Furthermore, the learned reward model is repurposed as an "action feasibility predictor." Using single-view RGB-D data, this predictor acts as a real-time safety check, preventing the robot from executing potentially catastrophic actions if they don't align with the learned success criteria.
Key Results and Insights
PRISM was evaluated on a UR5e robotic arm across six diverse tasks, including lifting objects, pressing buttons, and placing items in bowls.
- Superior Performance: PRISM achieved an 82% average success rate across tasks with randomized initial poses, using only 5 demonstrations. In contrast, standard Behavior Cloning (BC) achieved only 13% success under the same conditions.
- Data Efficiency: Even when the BC baseline was provided with three times more data (15 demos), its success rate only reached 30%, highlighting the superior robustness of PRISM’s RL-based exploration.
- VLM Accuracy: The study found that providing explicit geometric constraints ("does object A overlap object B?") rather than general success queries ("is the task done?") improved policy success rates by nearly 70%.
- Safety Gains: Integrating the action feasibility predictor during deployment boosted real-world success rates by up to 37%.
Conclusion
PRISM demonstrates that highly robust robotic policies can be learned from minimal real-world data by automating the simulation and reward-labeling process. While currently dependent on the availability of 3D assets, this framework paves the way for fully automated, scene-aware training pipelines that can rapidly adapt to new manipulation tasks in complex environments.
Summary updated