Spontaneous Giving and Calculated Greed in Language Models

Yuxuan Li, Hirokazu Shirado

Abstract

Large language models demonstrate advanced problem-solving capabilities by incorporating reasoning techniques such as chain of thought and reflection. However, how these reasoning capabilities extend to social intelligence remains unclear. In this study, we investigate this question using economic games that model social dilemmas, where social intelligence plays a crucial role. First, we examine the effects of chain-of-thought and reflection techniques in a public goods game. We then extend our analysis to six economic games on cooperation and punishment, comparing off-the-shelf non-reasoning and reasoning models. We find that reasoning models significantly reduce cooperation and norm enforcement, prioritizing individual rationality. Consequently, groups with more reasoning models exhibit less cooperation and lower gains through repeated interactions. These behaviors parallel human tendencies of "spontaneous giving and calculated greed." Our results suggest the need for AI architectures that incorporate social intelligence alongside reasoning capabilities to ensure that AI supports, rather than disrupts, human cooperative intuition.

Introduction

As large language models (LLMs) are increasingly deployed as agents in social and economic environments—negotiating deals, managing resources, or advising human decision-makers—the question of their "social intelligence" becomes paramount. While these models are becoming better at "individual rationality" (e.g., solving complex logic or coding tasks), their ability to cooperate effectively when individual and group interests clash remains under-explored.

In human psychology, the dual-process framework suggests that cooperation often stems from "System 1" (fast, intuitive impulses), whereas "System 2" (slow, deliberative reasoning) can sometimes lead individuals to favor selfish outcomes. This study investigates whether reasoning-enhanced LLMs suffer from a similar "calculated greed," where increased deliberation makes them worse social partners.

Dual-process framework: System 1 intuition promotes cooperation, while System 2 reasoning can inhibit it.

Experimental Framework

To test this hypothesis, the researchers utilized a framework of six canonical economic games designed to simulate social dilemmas and norm enforcement. These games were adapted for LLM prompting to measure two primary behaviors: cooperation and punishment.

Overview of the six economic games used to measure cooperation and punishment behaviors.

The experiments were divided into three main phases:

  1. Prompt-based Reasoning: Forcing a standard model (GPT-4o) to use "Chain-of-Thought" (CoT) or "Reflection" before deciding in a Public Goods Game.
  2. Cross-Model Comparison: Comparing native "reasoning" models (e.g., OpenAI o1, Gemini-2.0-Flash-Thinking, DeepSeek V3/R1) against their non-reasoning counterparts.
  3. Evolutionary Dynamics: Simulating repeated interactions in groups with varying ratios of reasoning and non-reasoning agents to observe long-term collective welfare.

Key Results and Insights

The Reasoning-Cooperation Trade-off

Across almost every model family tested, the "smarter" reasoning version was significantly less likely to cooperate. In the Public Goods Game (PGG), GPT-4o's cooperation rate plummeted from 96% (direct response) to 33% when forced to use a 15-step Chain-of-Thought. This trend was consistent across providers: Gemini-2.0-Flash-Thinking and Qwen3-30B-Extended-Thinking showed nearly identical drops in cooperation compared to their base versions.

Spontaneous Giving vs. Calculated Greed

The results mirror human behavioral data. When models act "intuitively" (direct response), they exhibit "spontaneous giving." However, when forced to deliberate, they identify that defection is the "rationally" superior strategy for the individual, leading to "calculated greed."

The Tragedy of the Reasoning Commons

In mixed-group simulations, reasoning agents (like o1) often "win" individually by free-riding on the cooperation of non-reasoning agents (like GPT-4o). However, as the proportion of reasoning agents in a group increases, the total group earnings collapse. In a 10-round iterated PGG, groups of four GPT-4o agents earned an average of 3932 points, while groups of four o1 agents earned only 740 points—an 81% decrease in collective welfare.

Norm Erosion

Reasoning models were also less likely to punish "bad actors" in punishment games (e.g., Ultimatum Game, Third-Party Punishment). They calculated that the personal cost of enforcing a norm outweighed the individual benefit, even though such punishment is essential for maintaining long-term societal cooperation.

Discussion and Conclusion

The findings suggest that current reasoning models may be over-optimized for "winning" in zero-sum contexts (like exams or games), creating a mindset toxic to cooperation. This study highlights a critical limitation: as we optimize for "cleverness," we may inadvertently build agents that prioritize individual gains at the expense of collective stability.

Future work must address the cultural biases inherent in these models, as cooperation norms vary globally. Most importantly, there is an urgent need for AI architectures that integrate social intelligence alongside logical reasoning. We must develop models that don't just ask "What is best for me?" but "What is the best sustainable outcome for the group?" to ensure AI supports human cooperative intuition rather than disrupting it.

Summary updated

Abstract

Large language models demonstrate advanced problem-solving capabilities by incorporating reasoning techniques such as chain of thought and reflection. However, how these reasoning capabilities extend to social intelligence remains unclear. In this study, we investigate this question using economic games that model social dilemmas, where social intelligence plays a crucial role. First, we examine the effects of chain-of-thought and reflection techniques in a public goods game. We then extend our analysis to six economic games on cooperation and punishment, comparing off-the-shelf non-reasoning and reasoning models. We find that reasoning models significantly reduce cooperation and norm enforcement, prioritizing individual rationality. Consequently, groups with more reasoning models exhibit less cooperation and lower gains through repeated interactions. These behaviors parallel human tendencies of "spontaneous giving and calculated greed." Our results suggest the need for AI architectures that incorporate social intelligence alongside reasoning capabilities to ensure that AI supports, rather than disrupts, human cooperative intuition.
BibTeX
@inproceedings{li2025spontaneous,
  author = {Yuxuan Li and Hirokazu Shirado},
  title = {{Spontaneous Giving and Calculated Greed in Language Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure