Terra Nova: A Comprehensive Challenge Environment for Intelligent Agents

Trevor McInroearXiv 2025 · DOI · arXiv

Abstract

Terra Nova is a high-fidelity reinforcement learning (RL) benchmark inspired by Civilization V, designed to shift the research focus from narrow tasks toward "Comprehensive Challenge Environments" (CCEs). It presents a 6-player, partially observable world characterized by an astronomical action space ($\sim 10^{745}$), multi-timescale credit assignment, and four mutually exclusive victory conditions. This paper introduces the environment's design, technical architecture, and its positioning as a new frontier for general intelligence.

Introduction

Most contemporary RL research utilizes "clean" environments to isolate specific hurdles, such as sparse rewards in Montezuma’s Revenge or complex control in MuJoCo. However, these benchmarks often fail to capture the interconnected variables found in real-world intelligence. Terra Nova is proposed as a Comprehensive Challenge Environment (CCE) where multiple canonical RL difficulties—including partial observability, representation learning, and enormous action spaces—must be addressed simultaneously.

Comparison of Terra Nova against existing RL benchmarks.

As shown in the comparison above, Terra Nova distinguishes itself by combining a "1vMany" opponent structure with large action spaces and multiple ways to win, surpassing the complexity of benchmarks like StarCraft II or NetHack.

The Terra Nova Environment

Terra Nova is formalized as a Turn-based Partially Observable Stochastic Game (POSG). A single game turn comprises six individual agent turns, creating a dynamic and evolving state space.

Strategic Dynamics and Victory Conditions

Unlike the 1v1 or team-based structures common in competitive RL, Terra Nova employs a "1vMany" free-for-all. This design makes cooperation strategically dominant; agents who fail to engage in diplomacy or trade are likely to be outpaced by those who do. Agents can pursue victory through four distinct paths: Science, Domination, Culture, or Diplomacy. Because progress in one area often consumes resources required for another, agents must demonstrate high-level strategic commitment and "Theory of Mind" to anticipate and counter opponents' strategies.

A procedurally generated map in Terra Nova.

Action and Observation Complexity

The environment features a factorized action space of approximately $10^{745}$, requiring agents to reason across hierarchical control surfaces ranging from city management to tactical unit movement. Observations are equally complex, containing over 100 elements, including continuous values, categorical data, and spatial arrays representing the "Fog of War."

Technical Implementation

Terra Nova is built using JAX to support high-throughput research. It utilizes shard_map utilities to automatically distribute parallel games across XLA devices (GPUs/TPUs). The release includes 10,000 procedurally generated maps, balanced to ensure each player begins with a "regional resource" monopoly, thereby incentivizing trade.

To lower the barrier to entry, a baseline neural network architecture is provided. This model uses specialized encoders for diverse observation types and a cross-attention bottleneck to integrate information streams before passing them to factorized action heads.

Conclusion and Future Work

Terra Nova represents a significant step toward testing general intelligence in complex, interconnected systems. Key challenges remain, particularly regarding the "sparse reward" wall—where winning is difficult enough that agents may require dense auxiliary rewards that do not necessarily align with victory conditions. Future work will focus on developing agents capable of long-horizon reasoning and sophisticated social modeling within this high-dimensional space.

Summary updated

Abstract

We introduce Terra Nova, a new comprehensive challenge environment (CCE) for reinforcement learning (RL) research inspired by Civilization V. A CCE is a single environment in which multiple canonical RL challenges (e.g., partial observability, credit assignment, representation learning, enormous action spaces, etc.) arise simultaneously. Mastery therefore demands integrated, long-horizon understanding across many interacting variables. We emphasize that this definition excludes challenges that only aggregate unrelated tasks in independent, parallel streams (e.g., learning to play all Atari games at once). These aggregated multitask benchmarks primarily asses whether an agent can catalog and switch among unrelated policies rather than test an agent's ability to perform deep reasoning across many interacting challenges.
BibTeX
@misc{mcinroe2025terra,
  author = {Trevor McInroe},
  title = {{Terra Nova: A Comprehensive Challenge Environment for Intelligent Agents}},
  year = {2025},
  eprint = {2511.15378},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.15378},
}
Figure