Interactive Fiction Games: A Colossal Adventure

Matthew J. Hausknecht, Prithviraj Ammanabrolu, Marc-Alexandre Côté, Xingdi Yuan

Abstract

A hallmark of human intelligence is the ability to understand and communicate with language. Interactive Fiction games are fully text-based simulation environments where a player issues text commands to effect change in the environment and progress through the story. We argue that IF games are an excellent testbed for studying language-based autonomous agents. In particular, IF games combine challenges of combinatorial action spaces, language understanding, and commonsense reasoning. To facilitate rapid development of language-based agents, we introduce Jericho, a learning environment for man-made IF games and conduct a comprehensive study of text-agents across a rich set of games, highlighting directions in which agents can improve.

Introduction

While reinforcement learning (RL) has achieved remarkable success in visual domains like Atari and Go, language-based RL remains a "colossal" challenge. Interactive Fiction (IF) games, such as the classic Zork, require agents to process rich text descriptions and respond with meaningful text commands. This paper introduces Jericho, a new learning environment that enables RL agents to play human-made text adventure games, serving as a formidable testbed for natural language understanding and commonsense reasoning.

The Challenges of Text-Based RL

The authors identify three primary hurdles that make IF games difficult for traditional AI:

  1. Combinatorial Action Space: A simple four-word command drawn from a modest vocabulary can result in billions of possible combinations, most of which are nonsensical to the game's parser.
  2. Commonsense Reasoning: Humans intuitively know that a "locked chest" requires a "key." Agents lack this embodied experience and struggle to prioritize meaningful interactions over random ones.
  3. Knowledge Representation: Navigating complex, non-Euclidean maps and tracking nested objects (e.g., a key inside a box inside a room) requires sophisticated mental mapping.

The Jericho Environment

Jericho is an open-source Python environment, compatible with OpenAI Gym, that interfaces with the Z-Machine (the engine behind classic Infocom games). To make these games tractable, Jericho offers several innovations:

  • World Object Tree: Extracts the game's internal state as a tree of objects and locations, providing a semi-interpretable representation.
  • Valid Action Identification: A "handicap" mode identifies which actions will actually change the world state, helping agents bypass "parser frustration."
  • Template-Based Action Generation: Instead of generating raw strings, agents select a "template" (e.g., put OBJ in OBJ) and fill in the blanks with vocabulary words.

Agent Architectures

The study evaluates three types of agents: the Deep Reinforcement Relevance Network (DRRN), a choice-based agent; the Template-DQN (TDQN), a parser-based agent; and NAIL, a heuristic-based baseline. As shown in Figure 2, the TDQN architecture specifically addresses the action space challenge by learning to predict both templates and their parameters.

The Template-DQN (TDQN) architecture generates actions by predicting a template and its associated objects.

Experimental Results

The authors conducted a comprehensive study across 32 human-made IF games, ranging from classic dungeon crawls to complex narrative horror. This represents a significant increase in complexity over previous synthetic benchmarks.

Training curves across four different games (acorncourt, zork1, detective, and omniquest) demonstrate the learning progress of the agents.

Key insights from the results include:

  • RL Viability: Both DRRN and TDQN made progress in many games, though overall completion rates remain low (average scores: NAIL 4.9%, TDQN 6.1%, DRRN 1.7%).
  • Specificity vs. Generality: DRRN performed exceptionally well on games with focused action spaces, like Detective, by concentrating on high-value navigational actions.
  • The "Moonshot" Games: Several titles, such as Anchorhead, remained completely unsolved (0% score), highlighting challenges with long-term dependencies and sparse rewards.

Conclusion and Future Work

Jericho demonstrates that human-made IF games are a rich environment for developing language-aware AI. While current agents are "single-game" experts, future research will focus on generalization—training agents across hundreds of games to acquire "common sense." The authors suggest that Transformer-based architectures and unsupervised learning will be essential for capturing the nuanced prose and creative action generation required to master these digital worlds.

Summary updated

Abstract

A hallmark of human intelligence is the ability to understand and communicate with language. Interactive Fiction games are fully text-based simulation environments where a player issues text commands to effect change in the environment and progress through the story. We argue that IF games are an excellent testbed for studying language-based autonomous agents. In particular, IF games combine challenges of combinatorial action spaces, language understanding, and commonsense reasoning. To facilitate rapid development of language-based agents, we introduce Jericho, a learning environment for man-made IF games and conduct a comprehensive study of text-agents across a rich set of games, highlighting directions in which agents can improve.
BibTeX
@inproceedings{hausknecht2020interactive,
  author = {Matthew J. Hausknecht and Prithviraj Ammanabrolu and Marc-Alexandre Côté and Xingdi Yuan},
  title = {{Interactive Fiction Games: A Colossal Adventure}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  url = {https://openreview.net/forum?id=_pZoiF2aaqn},
}
Figure