Abstract
The threat of algorithmic collusion, and whether it merits regulatory intervention, remains debated, as existing evaluations of its emergence often rely on long learning horizons, assumptions about counterparty rationality in adopting collusive strategies, and symmetry in hyperparameters and economic settings among players. To study collusion risk, we introduce a meta-game design for analyzing algorithmic behavior under test-time constraints. We model agents as possessing pretrained policies with distinct strategic characteristics (e.g., competitive, naively cooperative, or robustly collusive), and formulate the problem as selecting a meta-strategy that combines a pretrained, initial policy with an in-game adaptation rule. We seek to examine whether collusion can emerge under rational choices and how agents co-adapt toward cooperation or competition. To this end, we sample normal-form empirical games over meta-strategy profiles, compute relevant game statistics (e.g., payoffs against individuals and regret against an equilibrium mixture of opponents), and construct empirical best-response graphs to uncover strategic relationships. We evaluate reinforcement-learning, UCB, and LLM-based strategies in repeated pricing games under symmetric and asymmetric cost settings, and present findings on the feasibility of algorithmic collusion and the effectiveness of pricing strategies in practical "test-time" environments.
Introduction
The potential for algorithmic collusion—where autonomous pricing agents learn to maintain supra-competitive prices without explicit communication—poses a significant challenge for regulatory oversight. While previous research has demonstrated the emergence of such behavior, these studies often rely on idealized conditions: long learning horizons, assumptions of counterparty rationality, and symmetric economic settings. In real-world applications, agents are typically pretrained and then deployed into "test-time" environments where they must adapt rapidly to unknown opponents over limited rounds. This paper introduces a meta-game framework to evaluate the feasibility and stability of algorithmic collusion under these more realistic, constrained conditions.
Methods
The authors propose a meta-game design to analyze strategic behavior by treating the selection of an agent's initial policy and its adaptation rule as a meta-strategy.
- Strategic Categorization: Thousands of policies (Q-learning, UCB, LLMs) are pretrained and categorized using two metrics:
- Paired Cooperativeness (PC): Measures cooperation with the original training partner.
- Cooperative Robustness (CR): Measures performance against a best-response exploitative opponent.
- Meta-Strategies and EGTA: A meta-strategy combines an initial policy category with an in-game adaptation rule (e.g., a specific learning rate). Empirical Game-Theoretic Analysis (EGTA) is used to construct payoff matrices from simulated interactions, allowing for the identification of Nash Equilibria (NE).
- Experimental Setup: The framework is tested in a Repeated Pricing Game based on the Logit Demand Model. The evaluation includes test-time interactions of varying lengths (50 to 10,000 rounds) across symmetric and asymmetric cost settings.
Results and Insights
The study reveals several critical findings regarding the emergence and fragility of collusion:
- Rationality of Collusion: In symmetric settings, Nash Equilibria often involve collusive meta-strategies, achieving approximately 70% of the maximum possible collusive profit. This suggests collusion can be a strategically sound choice even in short-term test-time scenarios.
- Impact of Asymmetry: Economic asymmetries, such as unequal marginal costs, act as a "collusion-killer." Low-cost agents prioritize market share, forcing high-cost agents into defensive, competitive stances.
- Algorithm-Specific Behaviors: UCB-based agents are "glass cannons"—highly collusive with each other but easily exploited by targeted best-responses. In contrast, LLM-based agents demonstrate resilience, capable of re-establishing collusion after periods of exploitation by leveraging historical context.
- Pessimism as a Defense: Initializing agents with "pessimistic" beliefs (expecting competitive opponents) significantly hinders the formation of collusion, highlighting the importance of initial agent configurations.
Conclusion
While algorithmic collusion can emerge as a rational equilibrium in symmetric markets at test-time, it remains fragile and sensitive to economic asymmetries and initial beliefs. Future work will extend this framework to Bayesian Nash Equilibrium to model uncertainty regarding opponent costs and explore cross-algorithmic interactions to further understand market stability in diverse agent populations.
Summary updated