Abstract: Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah A. Smith, Yejin Choi. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).
2019.
Introduction
Vision-and-Language Navigation (VLN) is a complex task requiring an agent to navigate 3D environments based on natural language instructions. This paper introduces PRESS (PREtrained LMS AND STOCHASTIC SAMPLING), a framework that significantly improves an agent's ability to follow instructions in unfamiliar indoor environments by addressing two fundamental bottlenecks: generalization and exposure bias.
Problem Formulation: Generalization and Exposure Bias
The authors identify two primary challenges that hinder current VLN systems:
- The Generalization Gap: Agents often overfit to the specific instructions and environments seen during training. When faced with "unseen" environments, performance typically drops. Standard models use static embeddings like GloVe, which fail to capture the contextual nuances needed for complex grounding.
- Exposure Bias: Most agents are trained using "teacher-forcing," where the model is always given the ground-truth next action. However, at test time, the agent must rely on its own previous (potentially mistaken) actions. If the agent makes one wrong turn, it enters a state it never saw during training and has no mechanism to recover.
The PRESS Framework
The authors propose two main interventions to build the PRESS agent:
- Instruction Encoding with Pretrained LMs: Instead of training word representations from scratch, they adapt large-scale pretrained language models—specifically BERT and GPT. To prevent overfitting during fine-tuning, they use a two-stage learning scheme: first freezing the pretrained LM to train the decoder, then fine-tuning the entire network with a lower learning rate.
- Stochastic Action Sampling (SS): To bridge the gap between training and inference, they use a sampling strategy where the agent occasionally takes its own predicted action instead of the teacher's action. This forces the agent to learn "self-correction" behavior to get back on track after mistakes.
Key Results
The PRESS agent set a new state-of-the-art on the Room-to-Room (R2R) benchmark at the time of publication. The richer linguistic representations from BERT/GPT helped the agent understand rare words and complex syntactic structures that are often missing from smaller VLN training sets.
Performance Highlights
- Primary Metric (SPL): Achieved 53% Success weighted by Path Length (SPL) on the test unseen set, a significant jump from the previous best of 47%.
- Generalization: The gap between "Seen" and "Unseen" environments was notably smaller. In the "Unseen" validation set, PRESS improved the Success Rate (SR) by 23% over the standard seq2seq baseline.
- Ablation: The authors found that BERT generally outperformed GPT and LSTM encoders, and that the combination of LM pretraining and stochastic sampling was synergistic—each technique boosted the other.
Conclusion
The PRESS framework demonstrates that "language-only" knowledge from large-scale pretraining transfers effectively to multimodal tasks. By combining these representations with stochastic sampling, the agent becomes more robust to its own errors and better equipped to navigate novel environments.
Summary updated