Learning to Ground Objects for Robot Task and Motion Planning

Yan Ding, Xiaohan Zhang, Xingyue Zhan, Shiqi Zhang

TL;DR

This paper introduces TMOC (Task-Motion Object-Centric planning), a framework that allows robots to perform complex manipulation tasks without prior knowledge of object properties like size or weight. By interleaving real-world execution with physics-based simulations, the robot "grounds" its symbolic task plans into physical reality, learning to predict how its actions will affect the world through trial and error.

What Problem Does This Paper Address?

Traditional Task and Motion Planning (TAMP) algorithms are great at solving "what to do" (task level) and "how to move" (motion level). However, they usually assume the robot has a perfect "world model"—meaning it already knows the exact dimensions, weights, and friction of every object it encounters.

In the real world (like a messy kitchen), this is rarely true. A robot might not know if a box is empty or full of lead until it tries to lift it. If the robot doesn't know these physical properties, its high-level plan (e.g., "stack these three blocks") might fail at the execution level (e.g., the stack falls over because the bottom block was too light). This paper tackles the challenge of planning when object properties, state mappings, and action transition functions are all unknown.

Methods

The authors propose a new framework called Planning with Physics-based Simulation (PPS) and an algorithm called TMOC.

  1. The PPS Framework: Unlike standard TAMP, PPS includes a high-fidelity physics engine. This engine acts as a bridge to "ground" symbolic states (like is_holding(block)) into continuous physical parameters.
  2. TMOC Algorithm:
    • Optimistic Initialization: The robot starts by assuming every action will succeed (inspired by the R-max algorithm).
  3. Particle-based Simulation: The robot maintains $N$ "simulated worlds" (particles), each with different hypothesized object properties (e.g., different weights for a block).
  4. Dual-Loop Learning: For every action taken in the real world, the robot runs the same action in all $N$ simulated worlds.
  5. Grounding & Update: By comparing the real-world outcome with the simulated outcomes, the robot updates the weights of its particles (similar to a particle filter). Over time, the simulations that best match reality "win," effectively teaching the robot the true physical properties of the objects.
  6. Integrated Planning: The robot uses these learned properties to refine both its motion trajectories (where to grasp) and its task strategies (how many blocks to stack safely).

Datasets

The researchers evaluated their approach on a custom "Stack-and-Push" domain:

  • Simulation: Used Pybox2D, a 2D physics simulator. The task involved a gripper using a container to move eight blocks of unknown sizes to a goal area.
  • Real-World: Conducted experiments using a UR5e robotic arm. To keep the problem tractable for a 2D-focused algorithm, they attached a rail to the table, constraining the motion to a 2D plane while maintaining 3D physical interactions.

Key Insights and Ideas

  • The Interdependency of TAMP: A robot’s high-level strategy depends on its low-level performance. For example, if a robot learns it is bad at grasping heavy blocks, its task planner should stop choosing plans that involve moving heavy blocks. TMOC captures this feedback loop.
  • Simulation as a Teacher: Instead of needing thousands of expensive real-world trials, the robot uses a physics engine to "hallucinate" outcomes. This allows it to learn much faster than pure reinforcement learning approaches that don't use a structured world model.
  • Grounding is the Key: The "aha moment" is that "grounding"—mapping a symbol like "Block A" to a physical reality like "a 5cm, 200g cube"—is a learning problem that can be solved by treating simulation parameters as latent variables to be optimized.

Key Results

  • Superior Utility: TMOC outperformed three competitive baselines (TMP-RL, GP, and TOEP) in cumulative utility. While other methods struggled to adapt to unknown properties, TMOC's utility curve rose sharply as it learned the object properties.
  • Convergence: In simulation, TMOC reached near 100% optimal plans and success rates after approximately 5000 episodes.
  • Efficiency: In real-world trials with the UR5e arm, the robot learned a reliable state mapping and transition function within just 32 to 44 episodes (roughly 12 hours of operation), demonstrating that the simulation-to-real transfer is highly effective.
  • Robustness to Variation: The algorithm successfully adapted even when blocks had different sizes, though it required more training episodes to reach peak performance compared to a "uniform size" scenario.

Limitations and Future Work

  • Dimensionality: The current implementation focuses on 2D interactions (or 3D interactions constrained to a 2D plane). Extending this to fully unconstrained 3D environments with complex rotations remains a challenge.
  • Computational Load: Running $N$ parallel simulations (the authors used $N=200$) can be computationally expensive. The authors suggest that distributed computing could help scale this.
  • Object Relevance: The current model assumes all objects are relevant. Future work could involve learning which objects in a scene actually need to be grounded to complete a specific task, saving computation on irrelevant background items.

Summary updated

Abstract

Task and motion planning (TAMP) algorithms have been developed to help robots plan behaviors in discrete and continuous spaces. Robots face complex real-world scenarios, where it is hardly possible to model all objects or their physical properties for robot planning (e.g., in kitchens or shopping centers). In this letter, we define a new object-centric TAMP problem, where the TAMP robot does not know object properties (e.g., size and weight of blocks). We then introduce Task-Motion Object-Centric planning ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TMOC</b> ), a grounded TAMP algorithm that learns to ground objects and their physical properties with a physics engine. TMOC is particularly useful for those tasks that involve dynamic complex robot-multi-object interactions that can hardly be modeled beforehand. We have demonstrated and evaluated TMOC in simulation and using a real robot. Results show that TMOC outperforms competitive baselines from the literature in cumulative utility.
BibTeX
@article{ding2022learning,
  author = {Yan Ding and Xiaohan Zhang and Xingyue Zhan and Shiqi Zhang},
  title = {{Learning to Ground Objects for Robot Task and Motion Planning}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2022},
  url = {https://openreview.net/forum?id=LwGpppjOGuu},
}

Loading related content...

Figure