Abstract
Terra Nova is a high-fidelity reinforcement learning (RL) benchmark inspired by Civilization V, designed to shift the research focus from narrow tasks toward "Comprehensive Challenge Environments" (CCEs). It presents a 6-player, partially observable world characterized by an astronomical action space ($\sim 10^{745}$), multi-timescale credit assignment, and four mutually exclusive victory conditions. This paper introduces the environment's design, technical architecture, and its positioning as a new frontier for general intelligence.
Introduction
Most contemporary RL research utilizes "clean" environments to isolate specific hurdles, such as sparse rewards in Montezuma’s Revenge or complex control in MuJoCo. However, these benchmarks often fail to capture the interconnected variables found in real-world intelligence. Terra Nova is proposed as a Comprehensive Challenge Environment (CCE) where multiple canonical RL difficulties—including partial observability, representation learning, and enormous action spaces—must be addressed simultaneously.
As shown in the comparison above, Terra Nova distinguishes itself by combining a "1vMany" opponent structure with large action spaces and multiple ways to win, surpassing the complexity of benchmarks like StarCraft II or NetHack.
The Terra Nova Environment
Terra Nova is formalized as a Turn-based Partially Observable Stochastic Game (POSG). A single game turn comprises six individual agent turns, creating a dynamic and evolving state space.
Strategic Dynamics and Victory Conditions
Unlike the 1v1 or team-based structures common in competitive RL, Terra Nova employs a "1vMany" free-for-all. This design makes cooperation strategically dominant; agents who fail to engage in diplomacy or trade are likely to be outpaced by those who do. Agents can pursue victory through four distinct paths: Science, Domination, Culture, or Diplomacy. Because progress in one area often consumes resources required for another, agents must demonstrate high-level strategic commitment and "Theory of Mind" to anticipate and counter opponents' strategies.
Action and Observation Complexity
The environment features a factorized action space of approximately $10^{745}$, requiring agents to reason across hierarchical control surfaces ranging from city management to tactical unit movement. Observations are equally complex, containing over 100 elements, including continuous values, categorical data, and spatial arrays representing the "Fog of War."
Technical Implementation
Terra Nova is built using JAX to support high-throughput research. It utilizes shard_map utilities to automatically distribute parallel games across XLA devices (GPUs/TPUs). The release includes 10,000 procedurally generated maps, balanced to ensure each player begins with a "regional resource" monopoly, thereby incentivizing trade.
To lower the barrier to entry, a baseline neural network architecture is provided. This model uses specialized encoders for diverse observation types and a cross-attention bottleneck to integrate information streams before passing them to factorized action heads.
Conclusion and Future Work
Terra Nova represents a significant step toward testing general intelligence in complex, interconnected systems. Key challenges remain, particularly regarding the "sparse reward" wall—where winning is difficult enough that agents may require dense auxiliary rewards that do not necessarily align with victory conditions. Future work will focus on developing agents capable of long-horizon reasoning and sophisticated social modeling within this high-dimensional space.
Summary updated