TL;DR
BiPOP is a new framework that combines the semantic reasoning of Large Language Models (LLMs) with the rigorous logic of temporal partial-order planners to master complex, two-armed robotic tasks. By delegating tricky spatial and temporal coordination to a symbolic planner, BiPOP achieves higher success rates and more efficient parallel execution than even the most advanced "reasoning" LLMs.
What Problem Does This Paper Address?
Bimanual manipulation—using two arms to perform a task—is a cornerstone of human dexterity, yet it remains a "grand challenge" in robotics. The difficulty isn't just in moving each hand; it’s in the coordination.
Imagine setting a table: your left hand might hold a tray while your right hand places a glass. This requires understanding reachability (which hand can reach what?), dependencies (I can't pour the water until the glass is on the table), and concurrency (can I reach for the fork while I'm setting down the plate?).
While LLMs are great at high-level task decomposition (e.g., "to serve water, I need a bottle and a glass"), they are notoriously bad at "embodiment-aware" planning. They frequently hallucinate physically impossible actions, ignore spatial constraints, or fail to parallelize tasks effectively. Conversely, classical symbolic planners are mathematically rigorous but require expert-level "hand-coding" to translate a human's natural language request into a formal logic problem.
Methods
The authors introduce BiPOP (Bimanual Partial-Order Planning), which bridges this gap by splitting the labor between an LLM and a symbolic planner.
-
Spatio-temporal Modeling: The workspace is partitioned into three zones: two "uni-manual" regions reachable by only one arm, and one "coordinated" overlap region. This allows the system to automatically identify when arms can work independently (parallel execution) versus when they must synchronize.
Fig. 3: Spatio-temporal modelling between the two hands. -
LLM as a PDDL Translator: Instead of asking an LLM to write a step-by-step plan, BiPOP uses it to translate a natural language instruction and a visual scene description into a formal PDDL (Planning Domain Definition Language) problem. It uses Python code generation to interface with the
unified-planninglibrary, making the process more robust. - Temporal Partial-Order Planning: A specialized planner called Temporal Fast Downward (TFD) takes the PDDL problem and finds the most efficient sequence of actions. Crucially, it produces "partial-order" plans—meaning it schedules independent actions to run at the same time, minimizing the total "makespan" (execution time).
-
The Pipeline: As shown in the overview below, the system includes a validation loop. If the planner can't find a solution, the error is fed back to the LLM to fix the problem specification.
Fig. 1: Overview of BiPOP, illustrated on the ServeWater task. From the visual scene description, the bimanual domain knowledge, and the natural-language instruction, BiPOP generates a PDDL problem solved by a temporal planner. The validated partial-order plan with the best symbolic makespan found (cf. Fig. 2) is executed by the bimanual robot NICOL [15].
Datasets
The researchers evaluated BiPOP across three simulated bimanual domains in CoppeliaSim, each featuring 100 randomized instances:
- ServeFruit: Placing fruit in a bowl and serving it (requires a two-handed grasp of the bowl).
- ServeWater: Serving a cup of water while clearing a bottle to a storage area.
- StackBlock: A long-horizon task of stacking six colored blocks in specific orders.
The framework was also validated in the real world on the NICOL robot, a semi-humanoid platform designed for social interaction and manipulation.
Key Insights and Ideas
- The "Bottleneck" is Coordination, Not Skills: Most bimanual tasks are just a series of single-arm skills with a few "handshakes" in between. The real trick is managing the schedule, not just learning the motions.
- Delegation is Key: LLMs are best at semantic reasoning ("A bottle is a container"), while symbolic planners are best at resource management ("Arm A cannot be in two places at once"). BiPOP's success comes from not forcing the LLM to do the planner's job.
- LLM Failures are Predictable: A failure analysis showed that LLM-only planners (like GPT-4o or DeepSeek-R1) mostly fail by violating spatial or temporal constraints—trying to use both hands on one object simultaneously or releasing an object before it's placed. BiPOP avoids these "by construction" because the PDDL domain forbids them.
Key Results
BiPOP was compared against "I/O" (open-loop LLM planning) and "ReAct" (closed-loop LLM planning) using state-of-the-art models like OpenAI o3 and DeepSeek-R1.
- Success Rate: BiPOP achieved a $\ge 95\%$ success rate across all domains, significantly outperforming the baselines. On the difficult StackBlock task, GPT-4o alone had a 0% success rate, while BiPOP reached 95%.
- Planning Speed: Despite the overhead of symbolic planning, BiPOP was much faster than the reasoning models. For StackBlock, BiPOP took ~35 seconds, while DeepSeek-R1 took over 6 minutes (~367s).
- Efficiency: BiPOP consistently found plans with the shortest "symbolic makespan," meaning it maximized the time both arms were working simultaneously.
Limitations and Future Work
- Atomic Skills: BiPOP currently relies on a pre-defined library of motion primitives (e.g.,
grasp,pour). It cannot "invent" new physical skills on the fly. - Uniform Durations: The planner currently assumes every action takes the same amount of time. In reality, moving across the table takes longer than a simple grasp. The authors suggest integrating motion-layer duration estimates in the future.
- Domain Knowledge: While the LLM writes the problem, the domain (the rules of physics for the robot) is still hand-authored. Future work could involve using LLMs to induce these domain rules from demonstrations.
Summary updated