Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents

Wenlong Huang, Pieter Abbeel, Deepak Pathak, Igor Mordatch

TL;DR

Large Language Models (LLMs) like GPT-3 and Codex contain vast amounts of latent world knowledge that can be repurposed for robotic planning without any additional training. This paper introduces a "translation" framework that grounds free-form LLM suggestions into valid, executable actions for embodied agents, significantly boosting the success rate of complex household tasks.

The Challenge of Grounding

While LLMs have shown a remarkable ability to generate human-like text, using them to control robots (embodied agents) is difficult. A robot requires precise, syntactically correct commands that match its capabilities—for example, [Walk] <kitchen>(1). In contrast, an LLM might suggest "Go to the kitchen and find some eggs," which is semantically sensible but technically un-executable for a simple agent.

The authors tackle the challenge of grounding: how do we take the high-level "common sense" found in LLMs and map it to a specific, constrained set of actions in a virtual environment? Crucially, they aim to achieve this "zero-shot," meaning the model isn't fine-tuned on robotic data; it simply uses the knowledge it already possesses.

The "Translated [LM]" Pipeline

To bridge the gap between natural language and robotic commands, the authors propose a three-component pipeline:

  1. Planning LM: A massive pre-trained model (like GPT-3 175B or Codex) acts as the "brain," decomposing a high-level task (e.g., "make breakfast") into a sequence of natural language steps.
  2. Semantic Translation: To translate free-form text into robot commands, a separate "Translation LM" (Sentence-RoBERTa) is used. This model calculates the cosine similarity between the LLM’s suggested step and a bank of all possible admissible actions in the environment, picking the closest match.
  3. Autoregressive Trajectory Correction: Instead of generating a whole plan and then translating it, the system interleaves the two. After a step is translated into a valid action, that valid action is fed back into the LLM's prompt. This ensures the model stays "anchored" to reality as it plans subsequent steps.
The proposed pipeline: Zero-shot planning is followed by semantic translation and autoregressive feedback to ensure executability.

Experimental Setup

The research utilizes VirtualHome, a platform that simulates household activities through programmatic instructions. The dataset includes a knowledge base of 292 distinct high-level tasks with over 2,800 human-annotated programs. The evaluation set consists of 88 randomly sampled high-level tasks, while a separate demonstration set of 204 tasks is used only to provide "weak supervision" (examples) in the prompts, ensuring no overlap with the evaluation tasks.

Key Insights and Results

  • Scale is Mandatory: The ability to decompose tasks into logical plans is an emergent property of scale. Smaller models (like GPT-2) fail miserably, often just repeating the prompt or giving nonsensical one-step answers.
  • Codex is a Superior Planner: Models trained on code (Codex) outperform those trained only on natural language (GPT-3). The authors hypothesize that exposure to structured data helps the model better understand how to break a goal into a sequence of discrete operations.
  • Massive Executability Gains: For Codex-12B, the "Vanilla" model produced executable plans only 18% of the time. With the authors' translation and correction tools, this jumped to 78.5%.
  • Surpassing Human Labels: In a surprising twist, human evaluators rated the plans generated by the largest GPT-3 model (175B) as more correct than the ground-truth programs in the VirtualHome dataset (65.19% vs 54.80%). This suggests that LLMs can sometimes be more thorough than the humans who originally labeled the data.
  • The Executability-Correctness Trade-off: There is a fascinating tension between making a plan "runnable" and making it "right." While the translation layer makes plans executable, it can sometimes map a nuanced LLM suggestion to a slightly incorrect valid action, leading to a drop in human-rated correctness.
The trade-off between executability and correctness across different model sizes and methods.

Limitations and Future Work

The current approach relies on "blind" planning. The LLM generates a plan based on imagination and general knowledge without seeing the actual state of the environment (e.g., it might suggest "open fridge" even if the fridge is already open).

The authors identify three major areas for future research:

  1. Closing the Loop: Incorporating visual feedback and environment observations so the planner can react to failures or changes in the world.
  2. Low-level Grounding: Moving beyond "mid-level" tasks (like "grab cup") to low-level sensorimotor control.
  3. Improving Translation: Using even larger models for the translation layer to reduce the "correctness drop" seen when mapping natural language to robot APIs.

Summary updated

Abstract

Can world knowledge learned by large language models (LLMs) be used to act in interactive environments? In this paper, we investigate the possibility of grounding high-level tasks, expressed in nat...
BibTeX
@inproceedings{huang2022language,
  author = {Wenlong Huang and Pieter Abbeel and Deepak Pathak and Igor Mordatch},
  title = {{Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=OZXbcywVsT},
}
Figure