TL;DR
Objectives as Policies (OaP) bridges the gap between high-level semantic reasoning and low-level physical control by using Vision-Language Models (VLMs) to write editable "objective programs." By iteratively refining these programs in a simulation-based digital twin before real-world deployment, OaP enables robots to perform complex, contact-rich manipulation tasks—like pushing objects with tools or reorienting cups—without any task-specific training data.
What Problem Does This Paper Address?
Robotic manipulation is often a tale of two worlds. On one side, Vision-Language Models (VLMs) excel at semantic understanding (e.g., "put the eraser on the box"), but they struggle with the precise, non-differentiable physics of contact-rich tasks. On the other side, Model-Predictive Control (MPC) handles complex physics beautifully but requires painstakingly hand-engineered reward functions for every new task and environment.
Connecting these two is difficult because there is no common language that captures both the intent of a task and the physics required to execute it. Existing end-to-end "Vision-Language-Action" (VLA) models often fail at contact-rich tasks because they lack an explicit physics model, while previous attempts to use LLMs to script MPC often require frequent, slow updates during execution.
Methods
The authors introduce Objectives as Policies (OaP), a framework that uses a VLM as a "programmer" for a physics-based controller.
- Digital Twin via Real-to-Sim: The system first builds a simulation-based replica of the real-world scene using off-the-shelf perception tools (like SAM 3D and FoundationPose).
- Objective Programs: Instead of predicting raw actions, the VLM generates a structured JSON program. This program decomposes a task into a sequence of stages. Each stage defines:
- Running Costs: Objectives that guide the robot's motion (e.g., "move toward the cup").
- Terminal Costs: Conditions that must be met to finish the stage (e.g., "the cup is tilted 20 degrees").
- Cost Library: To keep the search space manageable, the VLM selects from a predefined library of parameterized functions (distance, alignment, contact indicators).
- Iterative Refinement: This is the "secret sauce." The VLM doesn't just guess once. It performs a self-review of its draft, then executes the program in the digital twin. If the simulation fails, the VLM analyzes the rollout statistics (like residual errors and contact states) to update and fix the program.
- MPC Execution: A sampling-based MPC (specifically MPPI) executes the final program. Because it runs in a GPU-parallel simulator (MuJoCo-warp), it can evaluate thousands of potential trajectories in real-time to find the optimal path.
Datasets
The framework was evaluated on four tabletop manipulation tasks designed to test contact-rich interactions:
- Pick-and-place: Moving an eraser onto a box.
- Cup reorientation: Grasping and tilting a cup.
- Direct push: Pushing a box to a target.
- Push with Tool: Using an eraser to push a box.
The authors compared OaP against several strong baselines, including Language to Rewards (L2R), ReKep, CoRAL, and the VLA model OpenVLA.
Key Insights and Ideas
- The Program as an Interface: By forcing the VLM to output a structured program rather than raw rewards or actions, the authors create a human-readable and editable bridge. This allows for "debugging" the robot's behavior in simulation.
- Refinement is Crucial: The paper shows that "zero-shot" generation is often insufficient. The ability of the VLM to "see" why a plan failed in simulation and adjust the cost weights or add a "settling" stage is what drives high success rates.
- Physics Matters: Unlike VLA models that try to learn everything from pixels, OaP leverages the fact that we already have excellent physics simulators. The VLM just needs to tell the simulator what to care about.
Key Results
- Simulation Success: In simulation, OaP achieved a 90% success rate with just two rounds of feedback, rising to 97.5% with five rounds.
- Real-World Robustness: When deployed on a Flexiv Rizon 4s arm, the system maintained an 80% success rate across all tasks, even when object locations were perturbed.
- Baseline Comparison: OaP significantly outperformed VLAs (which achieved 0-25% success) and outperformed CoRAL (77.5%), which requires much more frequent and computationally expensive LLM updates during execution.
Limitations and Future Work
- Library Dependency: The VLM is limited by the predefined cost library. If a task requires a physical concept not in the library, the VLM cannot express it.
- Linear Planning: The current objective programs follow a linear, forward-only sequence. If a stage fails halfway through real-world execution, the controller doesn't have an explicit mechanism to "backtrack" to a previous stage.
- Sim-to-Real Gap: While the digital twin is effective, errors in estimating friction or object stiffness (e.g., a 3D-printed cup vs. a real one) can still cause failures.
- Future Directions: The authors aim to incorporate online real-to-sim updates and develop more agentic frameworks that allow for explicit stage recovery during execution.
Summary updated