Interactive Fiction Games: A Colossal Adventure

Matthew J. Hausknecht, Prithviraj Ammanabrolu, Marc-Alexandre Côté, Xingdi Yuan

Abstract

A hallmark of human intelligence is the ability to understand and communicate with language. Interactive Fiction (IF) games are fully text-based simulation environments where a player issues text commands to effect change in the environment and progress through the story. We argue that IF games are an excellent testbed for studying language-based autonomous agents. In particular, IF games combine challenges of combinatorial action spaces, language understanding, and commonsense reasoning. To facilitate rapid development of language-based agents, we introduce Jericho, a learning environment for man-made IF games and conduct a comprehensive study of text-agents across a rich set of games, highlighting directions in which agents can improve.

Introduction

Interactive Fiction games—such as Zork or The Hitchhiker's Guide to the Galaxy—represent a "perfect storm" of research challenges for artificial intelligence. Unlike traditional video games with discrete button inputs, IF games require agents to navigate a world purely through ungrounded text descriptions and respond with natural language commands.

The primary hurdles include:

  1. Combinatorial Action Space: Natural language allows for billions of possible commands. A simple four-word sentence from a 700-word vocabulary creates a search space of approximately 240 billion actions.
  2. Commonsense Reasoning: Humans use prior knowledge to solve puzzles (e.g., knowing a "locked chest" requires a "key"). Reinforcement learning (RL) agents lack this intuition and often get stuck in unproductive loops.
  3. Narrative Complexity: Unlike procedurally generated environments, human-authored games contain intricate puzzles and linguistic nuances that test the limits of current NLP models.

The Jericho Environment

To address these challenges, we introduce Jericho, a Python-based interface for the Z-Machine. Jericho provides several "handicaps" to facilitate agent training:

  • State Management: Ability to save and load game states.
  • World-Object Tree: Access to a latent representation of items and locations.
  • Valid Action Identification: Detection of commands that actually alter the game state, significantly narrowing the search space.

Agent Architectures

We evaluate two primary neural architectures designed for text-based environments:

  • Deep Reinforcement Relevance Network (DRRN): A choice-based agent that learns to rank a provided list of valid actions by computing the relevance between the observation and each action.
  • Template-DQN (TDQN): A parser-based agent that generates actions by first selecting a template (e.g., put _ in _) and then filling the slots with vocabulary words.
Architectures for the DRRN and TDQN agents used in the study.

Experimental Results

We benchmarked these agents across 32 human-made IF games. The results highlight the significant gap between current AI capabilities and human-level performance.

Performance comparison of RAND, NAIL, TDQN, and DRRN agents across 32 IF games.

Key findings include:

  • Overall Performance: The best-performing agent, DRRN, achieved only 10.7% completion on average. TDQN followed at 6.1%, while the heuristic-based NAIL baseline reached 4.9%.
  • The "Valid Action" Bottleneck: Learning is often hindered not by choosing the best action, but by finding any action that the game's parser accepts.
  • Choice vs. Generation: Choice-based agents (DRRN) significantly outperform generation-based agents (TDQN) because they do not need to guess the correct vocabulary; they only rank known valid actions.

Conclusion and Future Work

Human-level Interactive Fiction remains an open challenge. Future research should focus on conditional action generation to avoid nonsensical commands, unsupervised exploration to handle sparse rewards, and the integration of large-scale Transformer models to better capture narrative context and commonsense reasoning.

Summary updated

Abstract

A hallmark of human intelligence is the ability to understand and communicate with language. Interactive Fiction games are fully text-based simulation environments where a player issues text commands to effect change in the environment and progress through the story. We argue that IF games are an excellent testbed for studying language-based autonomous agents. In particular, IF games combine challenges of combinatorial action spaces, language understanding, and commonsense reasoning. To facilitate rapid development of language-based agents, we introduce Jericho, a learning environment for man-made IF games and conduct a comprehensive study of text-agents across a rich set of games, highlighting directions in which agents can improve.
BibTeX
@inproceedings{hausknecht2020interactive,
  author = {Matthew J. Hausknecht and Prithviraj Ammanabrolu and Marc-Alexandre Côté and Xingdi Yuan},
  title = {{Interactive Fiction Games: A Colossal Adventure}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  url = {https://openreview.net/forum?id=_pZoiF2aaqn},
}
Figure