Robust Navigation with Language Pretraining and Stochastic Sampling

Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah A. Smith, Yejin Choi

Abstract

Core to the vision-and-language navigation (VLN) challenge is building robust instruction representations and action decoding schemes, which can generalize well to previously unseen instructions and environments. In this paper, we report two simple but highly effective methods to address these challenges and lead to a new state-of-the-art performance. First, we adapt large-scale pretrained language models to learn text representations that generalize better to previously unseen instructions. Second, we propose a stochastic sampling scheme to reduce the considerable gap between the expert actions in training and sampled actions in test, so that the agent can learn to correct its own mistakes during long sequential action decoding. Combining the two techniques, we achieve a new state of the art on the Room-to-Room benchmark with 6% absolute gain over the previous best result (47% -> 53%) on the Success Rate weighted by Path Length metric.

Introduction

Vision-and-Language Navigation (VLN) requires an AI agent to navigate a 3D environment based on natural language instructions, such as "Walk up the stairs, turn left, and stop near the mannequins." Success in this task depends on two critical components: an instruction encoder for text understanding and an action decoder for decision-making. However, current agents face two significant hurdles:

  1. Generalization (The Encoder Problem): Agents often overfit to the specific vocabulary of training environments. With only a 69% bigram overlap between training and evaluation instructions in standard datasets, agents frequently fail when encountering rare or unseen words in new settings.
  2. Exposure Bias (The Decoder Problem): Training typically relies on "teacher-forcing," where the agent is provided with ground-truth expert actions at every step. During inference, however, the agent must rely on its own previous decisions ("student-forcing"). A single error can lead the agent into an unfamiliar state from which it cannot recover, as it never learned to correct mistakes during training.
The two primary challenges in VLN: Generalization and Exposure Bias.

Methods: The PRESS Framework

To address these challenges, we introduce PRESS (Pretrained language models and Stochastic Sampling).

Instruction Understanding with Pretrained LMs

Instead of training embeddings from scratch, PRESS leverages large-scale contextualized language models like BERT or GPT. To ensure robust integration, we employ a two-stage learning scheme:

  1. Frozen Stage: The language model is frozen while only the navigation components are trained.
  2. Fine-tuning Stage: The entire network is fine-tuned end-to-end with a reduced learning rate to prevent overfitting.

Stochastic Action Sampling

To mitigate exposure bias, we introduce a training scheme that alternates between teacher-forcing and student-forcing. At each training step, the agent decides via a fixed probability whether to follow the expert's ground-truth action or an action sampled from its own current policy. This stochastic approach forces the agent to experience and learn to recover from its own mistakes without making the training process inefficiently random.

The PRESS architecture featuring Pre-trained Language Models and Stochastic Action Sampling.

Experimental Setup

We evaluate PRESS on the Room-to-Room (R2R) benchmark, built on the Matterport3D dataset. R2R contains 10,800 panoramic views and 7,189 trajectories, each with three natural language instructions. The dataset is split to ensure no overlap between "seen" (training) and "unseen" (validation/test) environments, strictly testing generalization.

Results and Discussion

PRESS achieves a new state-of-the-art on the R2R test unseen split, reaching 53% SPL (Success weighted by Path Length). This represents a 6% absolute improvement over the previous best result of 47%. The raw Success Rate (SR) reached 57%.

Key insights from our experiments include:

  • Transferability of Text Pretraining: Large-scale text-only pretraining significantly improves visual grounding. For instance, the model successfully maps the unseen word "manikins" to the visual concept of "mannequins," a task where standard LSTM agents fail.
  • Balancing the Forcing Tradeoff: Stochastic sampling provides the necessary balance between the speed of teacher-forcing and the robustness of student-forcing, drastically reducing the performance gap between seen and unseen environments.
  • Model Ablations: BERT proved superior to GPT and standard LSTMs for instruction encoding in unseen environments. Similarly, Stochastic Sampling (SS) outperformed both pure teacher-forcing and pure student-forcing.

Conclusion

The PRESS framework demonstrates that robust navigation can be achieved through simple yet effective improvements to language understanding and action decoding. By utilizing pretrained language models and stochastic sampling, we significantly reduce exposure bias and improve generalization. As these components are modular, they serve as a strong foundation for future, more complex VLN architectures.

Summary updated

Abstract

Core to the vision-and-language navigation (VLN) challenge is building robust instruction representations and action decoding schemes, which can generalize well to previously unseen instructions and environments. In this paper, we report two simple but highly effective methods to address these challenges and lead to a new state-of-the-art performance. First, we adapt large-scale pretrained language models to learn text representations that generalize better to previously unseen instructions. Second, we propose a stochastic sampling scheme to reduce the considerable gap between the expert actions in training and sampled actions in test, so that the agent can learn to correct its own mistakes during long sequential action decoding. Combining the two techniques, we achieve a new state of the art on the Room-to-Room benchmark with 6% absolute gain over the previous best result (47% -> 53%) on the Success Rate weighted by Path Length metric.
BibTeX
@inproceedings{li2019robust,
  author = {Xiujun Li and Chunyuan Li and Qiaolin Xia and Yonatan Bisk and Asli Celikyilmaz and Jianfeng Gao and Noah A. Smith and Yejin Choi},
  title = {{Robust Navigation with Language Pretraining and Stochastic Sampling}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=Gno0GaqF8Qz},
}
Figure