Abstract: We propose a multi-agent learning approach for designing crowdsourcing contests and All-Pay auctions. Prizes in contests incentivise contestants to expend effort on their entries, with different prize allocations resulting in different incentives and bidding behaviors. In contrast to auctions designed manually by economists, our method searches the possible design space using a simulation of the multi-agent learning process, and can thus handle settings where a game-theoretic equilibrium analysis is not tractable. Our method simulates agent learning in contests and evaluates the utility of the resulting outcome for the auctioneer. Given a large contest design space, we assess through simulation many possible contest designs within the space, and fit a neural network to predict outcomes for previously untested contest designs. Finally, we apply mirror ascent to optimize the design so as to achieve more desirable outcomes. Our empirical analysis shows our approach closely matches the optimal outcomes in settings where the equilibrium is known, and can produce high quality designs in settings where the equilibrium strategies are not solvable analytically.
Introduction
In an all-pay auction, such as a crowdsourcing contest or a political campaign, every participant "pays" by exerting effort, but only the top performers receive prizes. For an auctioneer—such as a company offering a prize for a better algorithm—the goal is to design a prize structure that maximizes their own utility.
Traditionally, economists use Nash Equilibrium analysis to predict participant behavior. However, this approach faces significant bottlenecks in the real world. Once noise is introduced (where effort doesn't perfectly translate to quality) or complex utility functions are used, the mathematical analysis becomes analytically intractable. Furthermore, real participants often exhibit bounded rationality, using simple learning behaviors rather than perfect Nash strategies.
Methodology
To address these challenges, researchers at DeepMind developed a "differentiable economics" pipeline consisting of three main phases: simulation, surrogate modeling, and gradient-based optimization.
- Multi-Agent Simulation: The researchers simulate contests using learning agents, primarily employing Fictitious Play (FP) and Reinforcement Learning (REINFORCE). This generates a dataset mapping specific prize designs to the resulting auctioneer utility.
- Neural Surrogate Modeling: A feedforward neural network is trained to act as a "proxy" for the simulation. This network learns to predict the auctioneer’s utility for any given prize distribution. Because the neural network is differentiable, the gradient of the utility can be calculated with respect to the prize design.
- Design Optimization: Using Entropic Mirror Ascent, the system optimizes the prize distribution. This allows for an efficient search of the design space while adhering to constraints, such as a fixed total budget and non-increasing prize ranks.
Key Insights and Results
A central finding of the study is the "insurance" effect of secondary prizes. In noiseless auctions, the optimal strategy is often "winner-takes-all." However, as performance noise increases, the optimal design shifts toward awarding larger 2nd and 3rd prizes. This acts as a form of insurance, incentivizing participants to exert effort even if they might lose the top spot due to random factors.
Key results from the empirical analysis include:
- Theoretical Validation: In simple 3-bidder noiseless cases, the ML framework identified the same optimal design as traditional analytical Nash analysis, validating simulation as a reliable proxy.
- Scalability: In a 10-bidder auction with uniform noise, the optimizer successfully moved away from a 1st-prize-only model to a more distributed model (~0.44, ~0.32, ~0.15).
- Robustness and Generalization: The auctioneer’s utility remained relatively stable around the optimal design, and the neural network accurately predicted outcomes for auction designs it had never encountered during the simulation phase.
Limitations and Future Work
While powerful, the framework relies on the assumption that the chosen simulation models (FP or RL) accurately reflect human learning and bidding. If human behavior deviates significantly, the "optimal" design may be sub-optimal in practice. Future research may explore continuous bid spaces, more complex strategic scenarios like collusion, and settings where participants have varying levels of bidding granularity.
Summary updated