Multimodal Sequential Generative Models for Semi-Supervised Language Instruction Following

Kei Akuzawa, Yusuke Iwasawa, Yutaka MatsuoarXiv 2023 · DOI · arXiv

Abstract

Training agents to follow natural language instructions in complex environments typically requires vast amounts of paired data. This paper introduces the Multimodal Sequential Variational Autoencoder (MS-VAE), a generative framework that enables semi-supervised learning by utilizing both paired instruction-trajectory data and unlabeled trajectories. Central to this approach is the Bottleneck Attention module, which maps variable-length sequences into a fixed-length sequence of latent variables. Our results on the BabyAI and Room-to-Room (R2R) benchmarks show that MS-VAE significantly improves instruction-following performance, particularly in unseen environments, by effectively leveraging unpaired data.

Introduction

The task of instruction following—where an agent must execute a sequence of actions based on natural language commands—is a cornerstone of human-robot interaction. However, deep learning models for this task are notoriously data-hungry, often requiring hundreds of thousands of expert demonstrations. While "unpaired" data, such as raw videos of robot movement or standalone text instructions, is abundant, integrating it into a unified training framework remains a significant challenge.

The prevailing "speaker-follower" paradigm uses a supervised speaker model to generate synthetic instructions for unlabeled trajectories. However, this method's efficacy is capped by the quality of the initial speaker, which is itself trained on limited paired data. To overcome this, we propose a generative approach that learns a shared multimodal representation, allowing the model to improve its understanding of both language and action through self-reconstruction and cross-modal alignment.

Methodology: The MS-VAE Framework

The core of our contribution is the Multimodal Sequential Variational Autoencoder (MS-VAE). Unlike standard VAEs that compress inputs into a single vector, MS-VAE maintains the temporal structure necessary for sequential tasks.

Bottleneck Attention

A primary hurdle in multimodal generative modeling is the discrepancy in sequence lengths between text (instructions) and actions (trajectories). Traditional attention mechanisms are difficult to integrate into VAEs because they typically require direct access to the input sequence during decoding.

We introduce Bottleneck Attention to bridge this gap. This module uses a fixed number of $K$ learnable "tokens" to attend to the input sequence (either text or trajectory). These tokens act as a "bottleneck," absorbing the essential information into a fixed-length sequence of latent variables $z_1, z_2, \dots, z_K$. This consistent latent interface allows the decoder to apply standard attention mechanisms over the latent tokens, regardless of the original input's length.

Figure 1: Architecture of the MS-VAE showing the Bottleneck Attention module bridging the Encoder and Decoder.

Training and Regularization

The MS-VAE is trained using a combination of reconstruction losses. For paired data, the model performs cross-modal reconstruction (e.g., reconstructing a trajectory from an instruction). For unpaired data, it performs self-reconstruction.

To prevent the model from overfitting to the reconstruction of unlabeled trajectories at the expense of language alignment, we employ Domain Distance Regularization using Sliced Wasserstein Distance. This ensures the distribution of latent variables for unpaired trajectories remains close to those derived from paired data, maintaining a coherent shared latent space.

Experimental Setup

We evaluated the MS-VAE on two distinct benchmarks:

  1. BabyAI: A gridworld environment with synthetic instructions. We focused on "BossLevel," the most complex task, and two miniaturized variants, "GoToSeqLocal" and "BossLocal," to isolate specific challenges.
  2. Room-to-Room (R2R): A large-scale benchmark featuring 7,000+ trajectories in realistic 3D indoor environments with human-annotated instructions. We supplemented the standard training set with 178,000 unlabeled trajectories for semi-supervised learning.

Results and Discussion

Our experiments demonstrate that the MS-VAE framework provides a robust performance boost across both synthetic and realistic environments.

Performance on R2R

In the R2R benchmark, the MS-VAE improved the success rate (SR) of the state-of-the-art speaker-follower model by approximately 2% to 4%. Notably, when using pragmatic inference—where the agent evaluates multiple candidate paths and selects the one most likely to be described by the internal "speaker" model—the MS-VAE achieved a success rate of 56.1% on unseen validation environments, outperforming previous non-RL baselines.

Figure 6: Quantitative results on the R2R benchmark comparing MS-VAE with existing state-of-the-art methods under greedy and pragmatic decoding.

The Importance of Bottleneck Attention

Ablation studies on the BabyAI "BossLevel" task highlighted the criticality of the bottleneck mechanism. Models using a single-vector latent representation saw success rates plummet by nearly 40%. In contrast, the MS-VAE's ability to retain sequential information through its $K$ latent tokens allowed it to maintain high performance even as task complexity increased.

Conclusion

The MS-VAE offers a powerful framework for semi-supervised instruction following by treating the task as a multimodal generative problem. By utilizing Bottleneck Attention, the model successfully bridges the gap between variable-length sequences and latent generative models. Future work will explore incorporating unpaired language data and developing dynamic methods for determining the optimal number of bottleneck tokens.

Summary updated

Abstract

Agents that can follow language instructions are expected to be useful in a variety of situations such as navigation. However, training neural network-based agents requires numerous paired trajectories and languages. This paper proposes using multimodal generative models for semi-supervised learning in the instruction following tasks. The models learn a shared representation of the paired data, and enable semi-supervised learning by reconstructing unpaired data through the representation. Key challenges in applying the models to sequence-to-sequence tasks including instruction following are learning a shared representation of variable-length mulitimodal data and incorporating attention mechanisms. To address the problems, this paper proposes a novel network architecture to absorb the difference in the sequence lengths of the multimodal data. In addition, to further improve the performance, this paper shows how to incorporate the generative model-based approach with an existing semi-supervised method called a speaker-follower model, and proposes a regularization term that improves inference using unpaired trajectories. Experiments on BabyAI and Room-to-Room (R2R) environments show that the proposed method improves the performance of instruction following by leveraging unpaired data, and improves the performance of the speaker-follower model by 2\% to 4\% in R2R.
BibTeX
@misc{akuzawa2023multimodal,
  author = {Kei Akuzawa and Yusuke Iwasawa and Yutaka Matsuo},
  title = {{Multimodal Sequential Generative Models for Semi-Supervised Language Instruction Following}},
  year = {2023},
  eprint = {2301.00676},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2301.00676},
}
Figure