Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue

So Yeon Min, Hao Zhu, Ruslan Salakhutdinov, Yonatan Bisk

Abstract

Embodied dialogue instruction following requires an agent to complete a complex sequence of tasks from a natural language exchange. The recent introduction of benchmarks (Padmakumar et al., 2022) raises the question of how best to train and evaluate models for this multi-turn, multi-agent, long-horizon task. This paper contributes to that conversation by arguing that imitation learning (IL) and related low-level metrics are actually misleading and do not align with the goals of embodied dialogue research and may hinder progress. We provide empirical comparisons of metrics, analysis of three models, and make suggestions for how the field might best progress. First, we observe that models trained with IL take spurious actions during evaluation. Second, we find that existing models fail to ground query utterances, which are essential for task completion. Third, we argue evaluation should focus on higher-level semantic goals.

Introduction

In the pursuit of creating agents capable of assisting humans in physical environments, the field of Embodied Dialogue has emerged as a critical frontier. Unlike simple instruction following, embodied dialogue requires agents to engage in multi-turn exchanges to resolve ambiguities and execute long-horizon tasks. The TEACh (Task-driven Embodied Agents that Chat) benchmark was designed to facilitate research in this area, providing over 3,000 human-human dialogues within the AI2-THOR simulation environment.

However, a significant challenge has emerged: the standard approach of training these agents via Imitation Learning (IL), specifically behavior cloning, appears to be a "trap." This paper argues that by forcing models to mimic the exact action traces of human "teachers," we are inadvertently training them to replicate human noise and inefficiency rather than understanding the underlying task logic.

The Problem: Ecologically Valid Noise

Human demonstrations in the TEACh dataset are "ecologically valid," meaning they reflect real-world human behavior, which is often messy. Humans get distracted, make mistakes, or perform "No-Op" actions—such as picking up and immediately setting down an object or spinning in circles while thinking.

When models are trained using behavior cloning, they are rewarded for matching these exact low-level actions. Our analysis reveals two major issues:

  • Spurious Actions: Approximately 25% of human demonstrations include redundant actions.
  • The Copycat Effect: Models like the Episodic Transformer (ET), which use end-to-end IL, replicate these "No-Op" behaviors in up to 46.2% of evaluation tasks. This often leads to infinite loops where the agent repeats a useless action instead of progressing toward the goal.

Methodology and Model Comparison

We evaluated three distinct model paradigms to understand how different architectures handle the complexities of TEACh:

  1. Episodic Transformer (ET): An end-to-end transformer model that maps visual and linguistic inputs directly to low-level actions via behavior cloning.
  2. FILM (Adapted): A modular approach that utilizes semantic mapping and a deterministic policy, focusing on abstract planning.
  3. Symbiote: A modular baseline combining a T5 language model with a DETR-based object detector.

These models were tested on two primary tasks: Execution from Dialogue History (EDH), which involves predicting the next set of actions given a partial history, and Trajectory from Dialogue (TfD), which requires completing the entire task from scratch based on the full dialogue.

Key Findings

1. The Misleading Nature of EDH

While EDH is a common metric, it can be highly misleading. Success in EDH often relies on predicting nearly deterministic sub-sequences (e.g., the final step of placing an object). A model can achieve high scores on EDH while remaining incapable of completing a full task (TfD), as it lacks the high-level planning necessary to bridge the gaps between dialogue turns.

2. Failure in Query Grounding

A cornerstone of dialogue is the ability to ask for and use information (e.g., "Where is the mug?"). Our study found a catastrophic failure in this area: zero of the tested models successfully utilized information provided by a "commander" in response to a query. Instead of using the provided location to find an object, models reverted to random searching, indicating a total lack of grounding in the dialogue exchange.

3. Modular Planning vs. Raw Imitation

Modular models like FILM, which plan at a semantic level (e.g., "Go to the fridge") rather than a low-level action level ("MoveAhead"), proved far more robust. By abstracting away the "how" of movement, these models avoid being bogged down by the noise inherent in human action traces.

Results

The performance gap between end-to-end imitation and modular planning is stark. On the full TfD task—the true measure of an embodied agent—the end-to-end ET model achieved a success rate of only 1.0% in seen environments. In contrast, the modular FILM model achieved 5.5%. While both numbers are low, the five-fold increase for FILM suggests that moving away from raw imitation toward goal-oriented planning is the correct path forward.

Conclusion and Future Directions

To advance the field of embodied dialogue, we must move beyond simple behavior cloning. We propose three shifts in research focus:

  1. Beyond Behavior Cloning: Develop training methods that prioritize verifiable semantic goals over exact action replication.
  2. Explicit Query Grounding: Design architectures that can store and retrieve information gained through dialogue to inform future actions.
  3. Evaluation Reform: Shift the primary metric of success from low-level action overlap (like EDH) to high-level goal completion (Success Rate on TfD).

By stopping the mindless "copying of the teacher," we can begin to build agents that truly understand the intent behind the conversation.

Summary updated

Abstract

Embodied dialogue instruction following requires an agent to complete a complex sequence of tasks from a natural language exchange. The recent introduction of benchmarks (Padmakumar et al., 2022) raises the question of how best to train and evaluate models for this multi-turn, multi-agent, long-horizon task. This paper contributes to that conversation, by arguing that imitation learning (IL) and related low-level metrics are actually misleading and do not align with the goals of embodied dialogue research and may hinder progress. We provide empirical comparisons of metrics, analysis of three models, and make suggestions for how the field might best progress. First, we observe that models trained with IL take spurious actions during evaluation. Second, we find that existing models fail to ground query utterances, which are essential for task completion. Third, we argue evaluation should focus on higher-level semantic goals.
BibTeX
@inproceedings{min2022don,
  author = {So Yeon Min and Hao Zhu and Ruslan Salakhutdinov and Yonatan Bisk},
  title = {{Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  url = {https://openreview.net/forum?id=oA8UjCsk85},
}
Figure