Beyond the Tragedy of the Commons: Building A Reputation System for Generative Multi-agent Systems

Siyue Ren, Wanli Fu, Xinkun Zou, Chen Shen, Yi Cai, Chen Chu, Zhen Wang, Shuyue Hu

Abstract

Large Language Model (LLM)-based multi-agent systems (MASs) are prone to "cooperation collapse," where individual incentives lead to the degradation of collective welfare. This paper introduces RepuNet, a decentralized reputation framework designed to foster sustained cooperation. RepuNet enables agents to form nuanced evaluations through direct encounters and indirect gossip, subsequently using these reputations to drive dynamic network rewiring. Our results across three social dilemmas—Iterated Prisoner’s Dilemma, Voluntary Participation, and Trading Investment—demonstrate that RepuNet effectively prevents collapse, maintaining cooperation rates as high as 98% by allowing cooperative agents to self-organize and isolate defectors.

Introduction

As autonomous agents powered by LLMs become increasingly integrated into social and economic systems, they encounter the classic "Tragedy of the Commons." Recent studies highlight a critical vulnerability: LLM agents often prioritize short-term gains, leading to cascading mutual defection in resource-sharing scenarios. Unlike human societies, which rely on complex social structures like reputation and ostracism to enforce norms, standard LLM deployments lack inherent mechanisms to manage social trust. RepuNet addresses this gap by operationalizing a reputation system that leverages the natural language reasoning capabilities of LLMs to build and maintain social order.

Methods

RepuNet operates through a dual-level architecture that bridges individual cognition with system-wide structure.

  1. Agent-Level Reputation Dynamics: Agents update their internal memory and reputation databases through two channels. In Direct Encounters, agents use LLM prompting to evaluate a partner's behavior, updating both "peer-reputation" and "self-reputation." In Indirect Gossip, agents broadcast evaluations to the community. Listeners weigh this information based on the gossiper's own reputation, ensuring that only credible information propagates.
  2. System-Level Network Evolution: Reputation serves as the primary driver for "Network Rewiring." Agents autonomously decide to build, maintain, or sever connections (edges) based on the reputations stored in their database. This allows the network topology to evolve dynamically in response to agent behavior.
The RepuNet framework integrates direct encounters, indirect gossip, and dynamic network rewiring to manage agent reputations and social structure.

Experimental Evaluation

The framework was tested using 20 LLM-based agents (utilizing backends like GPT-4o mini and Gemini 2.5 Flash) across three distinct scenarios:

  • Scenario 1 (Iterated Prisoner’s Dilemma): A fundamental test of cooperation vs. defection.
  • Scenario 2 (Voluntary Participation): A public goods dilemma involving energy consumption and community stability.
  • Scenario 3 (Trading Investment): A trust game measuring honesty and long-term investment strategies.

Results and Discussion

RepuNet significantly outperformed baseline systems lacking reputation mechanisms. In the absence of RepuNet, cooperation rates plummeted to between 9% and 19% across all scenarios. With the framework active, agents maintained high stability: 93% in the Prisoner’s Dilemma, 85% in Voluntary Participation, and 98% in Trading Investment.

Quantitative results across three scenarios show that RepuNet sustains high cooperation rates (left column) and establishes a strong correlation between an agent's behavior and their reputation (right column).

Key insights from the simulations include:

  • Emergent Clustering: Cooperative agents naturally formed dense clusters, while defectors were systematically isolated through edge disconnection.
  • The Virtuous Cycle: High reputation incentivizes continued cooperation to maintain access to high-quality partners.
  • Positive Gossip Bias: Interestingly, agents shared positive evaluations approximately 90% of the time, suggesting a "pro-social" bias in LLM reasoning that differs from human gossip patterns.

Conclusion

RepuNet demonstrates that decentralized reputation systems can effectively mitigate cooperation collapse in LLM-based societies. While future work must address scalability to larger agent populations and potential strategic dishonesty, this framework provides a scalable, "lightweight" alternative to centralized enforcement for maintaining social order in autonomous systems.

Summary updated

BibTeX
@misc{ren2025beyond,
  author = {Siyue Ren and Wanli Fu and Xinkun Zou and Chen Shen and Yi Cai and Chen Chu and Zhen Wang and Shuyue Hu},
  title = {{Beyond the Tragedy of the Commons: Building A Reputation System for Generative Multi-agent Systems}},
  year = {2025},
  eprint = {2505.05029},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.05029},
}
Figure