Strategic Tradeoffs Between Humans and AI in Multi-Agent Bargaining

Crystal Qian, Kehang Zhu, John Horton, Benjamin S. Manning, Vivian Tsai, James Wexler, Nithum Thain

TL;DR

When evaluated on aggregate outcomes like total profit, frontier large language models (LLMs) appear to match human performance in multi-agent economic negotiations. However, this study reveals that this outcome parity masks a massive procedural divide: while humans negotiate based on deep-seated fairness norms, LLMs adopt an overly concessionary, friction-minimizing posture that leaves them highly vulnerable to exploitation by strategic actors.

What Problem Does This Paper Address?

As LLMs transition into autonomous economic actors—handling contract negotiations, e-commerce, and supply chain coordination—evaluating their strategic behavior becomes critical. Traditional LLM evaluations rely heavily on static, question-answering benchmarks that fail to capture the dynamic, multi-agent complexities of real-world interactions where incentives are mixed and alignment must be negotiated in real-time.

Furthermore, standard multi-agent evaluations often focus purely on outcomes (e.g., total surplus or win rate). The authors tackle a foundational question in AI evaluation: Does achieving the same final score as a human mean an AI agent is actually reasoning and behaving like a human, or does it mask fundamental strategic differences?

Methods

The authors designed a novel, incentivized multi-player bargaining game implemented on the open-source Deliberate Lab platform.

Figure 1: Overview of the bargaining game. Left: Game setup showing chip endowments and valuation structure. Center: Gameplay mechanics for proposing and accepting offers. Right: The three agent types evaluated in this study.
  • Game Setup: Three players are endowed with bundles of colored chips. One color (green) is a publicly known numeraire ($0.50/chip), while other colors have private, randomly assigned valuations unique to each player. This setup induces a strong incentive to trade to capture valuation gaps.
  • Gameplay: Players take turns making structured trade proposals (e.g., "I'll give 2 red chips for 3 blue chips") over 9 cyclic rounds. The other two players simultaneously choose to accept or reject the offer.
  • Populations Evaluated: The authors compared three distinct, homogeneous groups under identical randomized initial conditions:
  • Humans: 216 participants recruited via Prolific.
  • LLM Agents: Frontier models (GPT-4o and Gemini 1.5 Pro) tested with both simple baseline prompts and a refined "Type 2" deliberate reasoning prompt architecture.
  • Bayesian Agents: A task-specific statistical benchmark explicitly programmed to optimize expected utility using Bayesian updates on opponents' acceptance behaviors.
  • Analysis Framework: To benchmark performance, they formulated a centralized Pareto-optimal upper bound via linear programming. They also introduced a retrospective regret framework to classify actions into No regret, Forced regret (early over-commitments blocking later, better deals), and Unforced regret (irrational passes).

Datasets

The paper does not use pre-existing training datasets. Instead, it introduces an empirical behavioral dataset generated from 1,296 trade proposals across 144 fully tracked games played by humans, LLMs, and Bayesian models under identical, controlled experimental conditions.

Key Insights and Ideas

  • The Illusion of Performance Parity: GPT-4o and humans achieved nearly identical aggregate surplus. However, their underlying strategies were completely different, proving that outcome-based benchmarking alone is insufficient for evaluating strategic AI safety and alignment.
  • LLMs as People-Pleasers: LLMs suffer from an inherent "concessionary posture." They consistently propose trades that give away more value than they request, maximizing immediate acceptance rates at the expense of long-term optimal value. This likely stems from instruction-tuning that rewards helpfulness and friction reduction.
  • Humans Default to Fairness: Even in anonymous, one-shot digital games with zero reputational risk, humans strongly adhere to unwritten fairness norms, typically proposing balanced 1:1 chip exchanges.
  • Bayesian Aggression Wins: The bespoke Bayesian agents aggressively maximized value by proposing highly self-interested, asymmetric trades. Though frequently rejected, this value-extractive strategy secured the highest overall surplus.
  • The "Lazy Generation" of Smaller Models: While frontier models like GPT-4o show complex (if flawed) strategic distributions, smaller models (like GPT-4o-mini and Gemini-2.5-Flash) collapse into rigid, low-variance template play, proposing near-zero surplus trades.

Key Results

  • Surplus Extraction: Bayesian agents dominated, capturing 74–80% of the theoretical Pareto-optimal bound. GPT-4o achieved human-level outcomes, capturing ~62% of the bound compared to the humans' 59% (in the 3-chip setting). Gemini 1.5 Pro lagged significantly behind at 42–45%.
  • Regret Analysis: The LLMs' concessionary strategy led to high rates of forced regret—by giving away chips too generously early on, they frequently ran out of inventory to capitalize on highly lucrative opportunities in later rounds.
  • Prompt Refinement Failure: Prompting LLMs to generate multiple candidate ideas and select the best one ("Type 2" reasoning) failed to yield statistically significant improvements in final surplus.

Limitations and Future Work

  • Stylized & Structured Environment: The game relies on static valuations and structured, text-free proposals, which naturally favors the mathematical precision of the Bayesian benchmark. Real-world bargaining involves free-form persuasion, bluffing, and relationship-building, which were excluded here.
  • Homogeneous Groups Only: The study only evaluated groups composed entirely of one agent type (e.g., all LLMs or all humans).
  • Future Directions: The authors call for evaluating mixed-agent environments (humans negotiating directly with AIs) to see if humans will exploit concessionary LLMs or if adversarial AIs will erode human fairness norms. They emphasize the urgent need for process-sensitive evaluation metrics rather than simple outcome scores.

Summary updated

Abstract

Markets increasingly accommodate large language models (LLMs) as autonomous decision-making agents. As this transition occurs, it becomes critical to evaluate how these agents behave relative to their human and task-specific statistical predecessors. In this work, we present results from an empirical study comparing humans (N=216), multiple frontier LLMs, and customized Bayesian agents in dynamic multi-player bargaining games under identical conditions. Bayesian agents extract the highest surplus with aggressive trade proposals that are frequently rejected. Humans and LLMs achieve comparable aggregate surplus within their groups, but exhibit different trading strategies. LLMs favor conservative, concessionary proposals that are usually accepted by other LLMs, while humans propose trades that are consistent with fairness norms but are more likely to be rejected. These findings highlight that performance parity -- a common benchmark in agent evaluation -- can mask substantive procedural differences in how LLMs behave in complex multi-agent interactions.
BibTeX
@inproceedings{qian2026strategic,
  author = {Crystal Qian and Kehang Zhu and John Horton and Benjamin S. Manning and Vivian Tsai and James Wexler and Nithum Thain},
  title = {{Strategic Tradeoffs Between Humans and AI in Multi-Agent Bargaining}},
  booktitle = {IUI},
  year = {2026},
  doi = {10.1145/3742413.3789078},
}

Loading related content...

Figure