Performative Prediction on Games and Mechanism Design

António Góis, Mehrnaz Mofakhami, Fernando P. Santos, Gauthier Gidel, Simon Lacoste-Julien

TL;DR

This paper extends the framework of performative prediction—where forecasts influence the outcomes they predict—to settings with interdependent agents, such as pandemic modeling or climate action. The authors demonstrate that simply maximizing prediction accuracy can inadvertently trap societies in low-welfare states (like total defection), and they propose using predictions as a mechanism to nudge populations toward better collective outcomes.

What Problem Does This Paper Address?

Traditional machine learning assumes that the data distribution is static. Performative prediction challenges this, acknowledging that a model's deployment can shift the distribution (e.g., a traffic forecast changes which routes drivers take).

However, existing research has largely treated agents as independent data points. In the real world, our decisions are often coupled: my choice to vaccinate or reduce carbon emissions depends on what I think everyone else will do. This paper addresses the gap of interdependency. When agents are linked via a social or geographic graph, a prediction of collective behavior becomes a powerful signal. The core challenge is that a "perfectly accurate" prediction might describe a social disaster, while a "wrong" prediction might nudge people toward a better reality. This creates a fundamental tension between accuracy and social welfare.

Methods

The authors model this interaction using the Collective Risk Dilemma (CRD). In this game, agents on a graph must decide whether to cooperate (at a cost) or defect. If the number of cooperators falls below a threshold $T$, everyone risks a major loss.

Key components of their approach include:

  • Trust Dynamics: Agents aren't blind followers. They use a Bayesian update to track the predictor's past accuracy. If the predictor "lies" to nudge cooperation and fails, agents lose trust and revert to their internal (often pessimistic) beliefs.
  • Repeated Risk Minimization (RRM): A standard algorithm that retrains models to maximize accuracy. The authors analyze its convergence in this multi-agent setting.
  • Learned Predictors: They employ Neural Networks (MLPs and GNNs) to optimize for welfare. They decompose the gradient of the welfare objective into two parts: an accuracy component (to maintain trust) and a steering component (to nudge agents toward cooperation).

Datasets

The study relies on synthetic populations structured by different graph topologies to simulate social connections:

  • Fully-connected graphs: Where every agent influences every other agent.
  • Scale-free networks: Representing realistic social structures with "hubs."
  • Lattices: Representing geographic proximity.

These simulations allow the authors to precisely control game parameters like risk, cost, and cooperation thresholds.

Key Insights and Ideas

  • The Accuracy Trap: The most striking finding is that maximizing accuracy can be socially harmful. In many game-theoretic settings, "everyone defects" is a stable, predictable state. An accuracy-maximizing model will learn to predict this failure perfectly, thereby reinforcing it.
  • Predictions as Mechanism Design: Instead of just observing reality, predictors act as "mechanisms" that can coordinate agents. By strategically distributing prediction errors, a model can nudge a population toward a Pareto-optimal state that they wouldn't reach on their own.
  • Trust as a Finite Resource: Trust is the "currency" the predictor spends to steer the population. If the predictor asks for too much cooperation (low accuracy), trust collapses.
  • Oscillatory Cycles: To maintain high welfare over time, the optimal strategy often involves "trust cycles." The predictor might be highly accurate for a while to build trust, then "spend" that trust to nudge a difficult collective action, as seen in the oscillations in Figure 6.
Figure 6: Trust and welfare oscillations. The predictor must balance building trust through accuracy and spending it to steer the population toward success.

Key Results

  • Theoretical Failure of RRM: Theorem 3.4 proves that for large populations in a fully connected graph, standard accuracy-maximizing algorithms (RRM) converge to a state of zero welfare (total defection) with high probability.
  • The Welfare-Accuracy Trade-off: The authors empirically compute a Pareto front showing that significant gains in social welfare (cooperation) can be achieved by accepting a relatively small drop in prediction accuracy.
Figure 1: Dark nodes have achieved success, and thick arrows are self-fulfilling prophecies. Both a) and c) are self-fulfilling prophecies where accuracy is maximized, therefore an accuracy maximizer is indifferent between them. However, in a) full success is achieved, but in c) all fail. b) also maximizes group success but at the expense of 0% accuracy.
  • Architecture Matters: Interestingly, standard Graph Neural Networks (GNNs) struggled to promote cooperation because they cannot easily "break symmetry"—sometimes coordination requires identical nodes to take different actions (e.g., one cooperates while the other defects to hit a threshold). MLPs or GNNs with additional linear layers performed better at this centralized coordination task.

Limitations and Future Work

  • Knowledge of Agent Models: The current framework assumes the predictor knows the agents' internal Bayesian trust models. In practice, these would need to be estimated from behavior.
  • Deterministic Actions: The model assumes agents act deterministically based on their expectations; introducing stochasticity could change the stability of the equilibria.
  • Ethics of Manipulation: The paper opens a significant ethical can of worms. If a pandemic modeler knows that a slightly "pessimistic" forecast will save lives by increasing social distancing, should they deploy it? The authors argue that since any prediction is performative and non-neutral, we must explicitly study these trade-offs rather than ignoring them.

Summary updated

Abstract

Agents often have individual goals which depend on a group's actions. If agents trust a forecast of collective action and adapt strategically, such prediction can influence outcomes non-trivially, resulting in a form of performative prediction. This effect is ubiquitous in scenarios ranging from pandemic predictions to election polls, but existing work has ignored interdependencies among predicted agents. As a first step in this direction, we study a collective risk dilemma where agents dynamically decide whether to trust predictions based on past accuracy. As predictions shape collective outcomes, social welfare arises naturally as a metric of concern. We explore the resulting interplay between accuracy and welfare, and demonstrate that searching for stable accurate predictions can minimize social welfare with high probability in our setting. By assuming knowledge of a Bayesian agent behavior model, we then show how to achieve better trade-offs and use them for mechanism design.
BibTeX
@inproceedings{gois2025performative,
  author = {António Góis and Mehrnaz Mofakhami and Fernando P. Santos and Gauthier Gidel and Simon Lacoste-Julien},
  title = {{Performative Prediction on Games and Mechanism Design}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2025},
  url = {https://openreview.net/forum?id=ebYWRXD5NK},
}
Figure