Abstract
A hallmark of human intelligence is the ability to understand and communicate with language. Interactive Fiction (IF) games are fully text-based simulation environments where a player issues text commands to effect change in the environment and progress through the story. We argue that IF games are an excellent testbed for studying language-based autonomous agents. In particular, IF games combine challenges of combinatorial action spaces, language understanding, and commonsense reasoning. To facilitate rapid development of language-based agents, we introduce Jericho, a learning environment for man-made IF games and conduct a comprehensive study of text-agents across a rich set of games, highlighting directions in which agents can improve.
Introduction
Interactive Fiction games—such as Zork or The Hitchhiker's Guide to the Galaxy—represent a "perfect storm" of research challenges for artificial intelligence. Unlike traditional video games with discrete button inputs, IF games require agents to navigate a world purely through ungrounded text descriptions and respond with natural language commands.
The primary hurdles include:
- Combinatorial Action Space: Natural language allows for billions of possible commands. A simple four-word sentence from a 700-word vocabulary creates a search space of approximately 240 billion actions.
- Commonsense Reasoning: Humans use prior knowledge to solve puzzles (e.g., knowing a "locked chest" requires a "key"). Reinforcement learning (RL) agents lack this intuition and often get stuck in unproductive loops.
- Narrative Complexity: Unlike procedurally generated environments, human-authored games contain intricate puzzles and linguistic nuances that test the limits of current NLP models.
The Jericho Environment
To address these challenges, we introduce Jericho, a Python-based interface for the Z-Machine. Jericho provides several "handicaps" to facilitate agent training:
- State Management: Ability to save and load game states.
- World-Object Tree: Access to a latent representation of items and locations.
- Valid Action Identification: Detection of commands that actually alter the game state, significantly narrowing the search space.
Agent Architectures
We evaluate two primary neural architectures designed for text-based environments:
- Deep Reinforcement Relevance Network (DRRN): A choice-based agent that learns to rank a provided list of valid actions by computing the relevance between the observation and each action.
- Template-DQN (TDQN): A parser-based agent that generates actions by first selecting a template (e.g.,
put _ in _) and then filling the slots with vocabulary words.
Experimental Results
We benchmarked these agents across 32 human-made IF games. The results highlight the significant gap between current AI capabilities and human-level performance.
Key findings include:
- Overall Performance: The best-performing agent, DRRN, achieved only 10.7% completion on average. TDQN followed at 6.1%, while the heuristic-based NAIL baseline reached 4.9%.
- The "Valid Action" Bottleneck: Learning is often hindered not by choosing the best action, but by finding any action that the game's parser accepts.
- Choice vs. Generation: Choice-based agents (DRRN) significantly outperform generation-based agents (TDQN) because they do not need to guess the correct vocabulary; they only rank known valid actions.
Conclusion and Future Work
Human-level Interactive Fiction remains an open challenge. Future research should focus on conditional action generation to avoid nonsensical commands, unsupervised exploration to handle sparse rewards, and the integration of large-scale Transformer models to better capture narrative context and commonsense reasoning.
Summary updated