A Gamified Framework for Conflict-Driven Design: Discovering Strategies for Collaborative Resilience with Multi-Agent Learning

Jin GaoarXiv 2026 · DOI · arXiv

TL;DR

This paper introduces a gamified framework that uses Multi-Agent Reinforcement Learning (MARL) to simulate the complex negotiations of urban design. By modeling stakeholders as competing agents in a non-zero-sum digital board game, the researchers demonstrate how AI can discover collaborative strategies that prioritize long-term urban resilience over short-term individual gains.

What Problem Does This Paper Address?

Urban planning is notoriously difficult because it involves a "clash of interests." Developers want density and profit, residents want greenery and quiet, and city officials want flood resistance and sustainability. Reaching a consensus often feels like a zero-sum game where one party’s gain is another’s loss.

Traditional computational tools like Agent-Based Models (ABMs) often rely on overly simplistic, rule-based behaviors that can't capture the strategic depth of real-world negotiations. This paper tackles the challenge of modeling these "conflict-driven" scenarios to see if AI can find "win-win" configurations that improve collective resilience—the ability of a city to withstand and adapt to climate-related shocks.

Methods

The authors developed a customized digital board game where agents represent different stakeholder personas (e.g., "Altruistic," "Balanced," or "Interest-driven").

The Game Mechanics: The environment is a configurable $M \times N$ grid. In each turn, agents choose to build structures (houses, shops, parks, flood barriers) or skip. Every action has a spatial and temporal impact: building a park might increase local greenery but decrease economic "vitality."

The Learning Architecture: To train these agents, the researchers employed Multi-Agent Reinforcement Learning (MARL). Specifically, they used MAPPO (Multi-Agent Proximal Policy Optimization).

Figure 3: The sequential game loop for stakeholder actions and state updates.

Unlike standard reinforcement learning where an agent only cares about its own score, this framework uses Centralized Training and Decentralized Execution (CTDE). During training, a "critic" sees the global state of the board to help agents learn how their individual actions contribute to the collective "Environment Score." At execution time, agents act independently based on their own observations.

Figure 4: Training architecture adapted by the author from the MARLlib proximal policy optimization family (Hu et al., 2023). The source is licensed under CC BY 4.0.

Datasets

The framework was tested across three increasingly complex environments:

  1. Proof-of-Concept: A minimal $4 \times 4$ grid used to benchmark MARL against traditional optimization (simulated annealing).
  2. Scaled-up Planning: An $8 \times 8$ grid with added terrain constraints and pre-built infrastructure.
  3. Tatsumi, Tokyo Case Study: A $12 \times 12$ simulation initialized using historical map data from the 1980s provided by the Geospatial Information Authority of Japan. This site was chosen specifically for its vulnerability to typhoons and its history of low-lying wetland development.

Key Insights and Ideas

  • Non-Zero-Sum Coordination: The agents learned that by coordinating actions—such as placing resilience-oriented projects like solar grids or community hubs—they could raise the overall environment score, which in turn boosted their own long-term rewards.
  • The Resilience Trade-off: The simulations revealed a stark reality: gains in flood resistance, sustainability, and greenery almost always come at the cost of urban "vitality" (economic activity) and density.
  • Long-term vs. Short-term: MARL agents successfully internalized that high-cost resilience projects (like flood barriers) yield greater cumulative rewards over a 100-200 turn horizon, even if they seem "expensive" in the short term.
  • Human-in-the-Loop: The authors developed an interactive interface that allows human planners to step into the game, either to observe AI strategies or to take over an agent's role to test how human intuition clashes or aligns with AI-optimized paths.
Figure 5: Human–agent interface for observing simulations and taking over an agent’s decisions.

Key Results

  • Performance: In the $4 \times 4$ test, MARL agents achieved ~84% of the total return found by a simulated annealing benchmark, proving the efficiency of the learning approach.
  • Tatsumi Case Study: In 10,000 simulation runs, the trained AI agents achieved a mean total score of 452.1, significantly outperforming a random-policy baseline of 357.3.
  • Resilience Gains: Compared to the baseline, the AI-driven designs showed massive improvements in resilience metrics: +98.3% in flood resistance and +45.2% in adaptability, though this resulted in a 17.1% drop in density.

Limitations and Future Work

The authors acknowledge several hurdles:

  • Geometric Constraints: The use of an orthogonal grid is a "minimal abstraction" that doesn't capture the irregular blocks of real cities.
  • Scoring Subjectivity: The "Environment Score" is based on predefined indices which might oversimplify the nuanced, subjective values of real human stakeholders.
  • Training Stability: Like many MARL applications, the training process can be volatile and prone to converging on local optima.

Future Directions: The team plans to integrate Large Language Models (LLMs) to provide "human-interpretable reasoning" for the agents' decisions, making the AI's "thought process" transparent to human planners during negotiations.

Summary updated

Abstract

Conflicting stakeholder objectives make it difficult to coordinate investments in urban resilience. We model conflict-driven urban design as a non-zero-sum digital board game in which stakeholder agents allocate scarce resources and balance individual payoffs against shared urban development and resilience objectives. We use multi-agent reinforcement learning (MARL) to train policies and examine their behavior across three settings. On a 4 x 4 board, learned policies reach about 84% of the total return found by a simulated annealing benchmark. On an 8 x 8 board with spatial and resource constraints, trained agents favor projects that improve resilience in self-play. In a 12 x 12 simulation initialized from historical maps of Tatsumi, Tokyo, trained agents achieve a mean total score of 452.1, compared with 357.3 for a random-policy baseline across 10,000 simulation runs. Gains in adaptability, sustainability, flood resistance, and greenery come with reductions in vitality and density. A human-in-the-loop interface supports replay and lets participants take over stakeholder decisions, providing a testbed for exploring these trade-offs in participatory urban planning.
BibTeX
@misc{gao2026gamified,
  author = {Jin Gao},
  title = {{A Gamified Framework for Conflict-Driven Design: Discovering Strategies for Collaborative Resilience with Multi-Agent Learning}},
  year = {2026},
  eprint = {2611.10841},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10841},
}

Loading related content...

Figure