Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents

Wenlong Huang, Pieter Abbeel, Deepak Pathak, Igor Mordatch

Abstract

Can world knowledge learned by large language models (LLMs) be used to act in interactive environments? In this paper, we investigate the possibility of grounding high-level tasks, expressed in natural language (e.g. "make breakfast"), to a chosen set of actionable steps (e.g. "open fridge"). While prior work focused on learning from explicit step-by-step examples of how to act, we surprisingly find that if pre-trained LMs are large enough and prompted appropriately, they can effectively decompose high-level tasks into mid-level plans without any further training. However, the plans produced naively by LLMs often cannot map precisely to admissible actions. We propose a procedure that conditions on existing demonstrations and semantically translates the plans to admissible actions. Our evaluation in the recent VirtualHome environment shows that the resulting method substantially improves executability over the LLM baseline. The conducted human evaluation reveals a trade-off between executability and correctness but shows a promising sign towards extracting actionable knowledge from language models.

Introduction

For an embodied agent to be effective in a domestic setting, it must bridge the gap between high-level human intent (e.g., "clean the living room") and low-level executable actions. Traditionally, this has required massive datasets of expert demonstrations or complex symbolic logic. While Large Language Models (LLMs) possess vast amounts of world knowledge, they suffer from a "grounding" problem: their generated text often lacks physical context or fails to match the specific API of a robotic system. This research explores how to extract and ground this latent knowledge into executable sequences of actions without additional fine-tuning.

Methodology

The proposed framework treats the LLM as a zero-shot planner, augmented by a semantic translation layer to ensure environmental compatibility.

  1. Zero-Shot Planning: Massive models like GPT-3 (175B) and Codex are prompted with a single example of task decomposition. The model then attempts to break down a novel task into natural language steps.
  2. Semantic Translation (Admissible Action Parsing): To resolve the grounding issue, a BERT-style model (Sentence-RoBERTa) maps free-form text to the nearest "admissible" action within the agent's capabilities. For instance, "go to the kitchen" is translated into the command [WALK] kitchen.
  3. Autoregressive Trajectory Correction: The system operates step-by-step. After a step is generated and translated, the valid action is fed back into the LLM's prompt. This closed-loop feedback prevents the model from generating plans that diverge from physical reality.
  4. Dynamic Example Selection: The system selects the most semantically relevant human demonstration from a library to serve as the in-context example for the current task, improving the relevance of the generated plan.

Experimental Results

The framework was evaluated using the VirtualHome simulator and the ActivityPrograms knowledge base, which contains nearly 3,000 human-annotated household tasks.

Performance comparison of various LLMs on correctness and executability, alongside qualitative plan examples and execution frames.

Quantitative Performance

  • Executability: Naive planning with Codex resulted in an executability rate of only 18.1%. By applying the proposed translation and correction techniques, executability increased dramatically to 78.6%.
  • Correctness: Human evaluations showed that "Translated GPT-3" plans achieved a correctness score of 66.1%, nearing the human baseline of 70.1%.
  • Comparison to Supervised Learning: The zero-shot LLM approach performed comparably to supervised LSTM models specifically trained on the dataset, demonstrating the power of pre-trained world knowledge.

Key Insights

  • Emergent Planning at Scale: Planning capability is an emergent property of model size. Smaller models (e.g., GPT-2) fail to sequence tasks logically, whereas models with over 100B parameters exhibit significant "common sense" in task decomposition.
  • The Thoroughness of LLMs: Interestingly, human evaluators occasionally rated LLM plans as more correct than human-written ones. Humans often omit "obvious" steps (like opening a door), while LLMs tend to be more exhaustive in their step-by-step reasoning.

Discussion and Future Work

While the results are promising, the current system is "blind" to the immediate environment, planning based solely on task names. Future iterations will incorporate visual feedback to allow the agent to adapt to dynamic changes (e.g., if a required item is not in its expected location). Additionally, there remains a trade-off between executability and correctness; forcing a plan to be executable can sometimes simplify it to the point of losing semantic nuance. Bridging the gap between these mid-level plans and low-level physical control remains a critical challenge for future research.

Summary updated

Abstract

Can world knowledge learned by large language models (LLMs) be used to act in interactive environments? In this paper, we investigate the possibility of grounding high-level tasks, expressed in natural language (e.g. "make breakfast"), to a chosen set of actionable steps (e.g. "open fridge"). While prior work focused on learning from explicit step-by-step examples of how to act, we surprisingly find that if pre-trained LMs are large enough and prompted appropriately, they can effectively decompose high-level tasks into mid-level plans without any further training. However, the plans produced naively by LLMs often cannot map precisely to admissible actions. We propose a procedure that conditions on existing demonstrations and semantically translates the plans to admissible actions. Our evaluation in the recent VirtualHome environment shows that the resulting method substantially improves executability over the LLM baseline. The conducted human evaluation reveals a trade-off between executability and correctness but shows a promising sign towards extracting actionable knowledge from language models. Website at https://huangwl18.github.io/language-planner
BibTeX
@inproceedings{huang2022language,
  author = {Wenlong Huang and Pieter Abbeel and Deepak Pathak and Igor Mordatch},
  title = {{Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=OZXbcywVsT},
}
Figure