Long-term causal effects via behavioral game theory

Panagiotis, Toulis, David C. Parkes

Abstract

Planned experiments are the gold standard in reliably comparing the causal effect of switching from a baseline policy to a new policy. One critical shortcoming of classical experimental methods, however, is that they typically do not take into account the dynamic nature of response to policy changes. For instance, in an experiment where we seek to understand the effects of a new ad pricing policy on auction revenue, agents may adapt their bidding in response to the experimental pricing changes. Thus, causal effects of the new pricing policy after such adaptation period, the long-term causal effects, are not captured by the classical methodology even though they clearly are more indicative of the value of the new policy. Here, we formalize a framework to define and estimate long-term causal effects of policy changes in multiagent economies. Central to our approach is behavioral game theory, which we leverage to formulate the ignorability assumptions that are necessary for causal inference. Under such assumptions we estimate long-term causal effects through a latent space approach, where a behavioral model of how agents act conditional on their latent behaviors is combined with a temporal model of how behaviors evolve over time.

The Challenge of Long-Term Inference

In multiagent economies—such as online ad auctions, financial markets, or peer-to-peer marketplaces—participants are not static. When a platform operator changes a rule or policy, agents (the users or bidders) eventually adapt their strategies to maximize their own utility under the new regime.

Standard A/B testing often fails in these dynamic environments because it captures short-term "transient" reactions rather than long-term equilibrium. This creates two major hurdles:

  1. The Fundamental Problem of Causal Inference: One cannot observe the same population under two different policies at the exact same time.
  2. The Temporal Gap: Short-term data from an experiment often reflects confusion or immediate reaction, not the stable, long-term behavior that determines a policy's true value. For example, raising a reserve price might spike revenue for a week, but if it causes bidders to leave the platform a month later, the long-term effect is negative.
Conceptual framework for estimating long-term causal effects from short-term observed data.

As illustrated above, the "observed data" collected during the initial phase of an experiment is often a poor proxy for the "target of inference" at a future time $T$. The goal is to bridge this gap by creating a model that can predict where the system will eventually stabilize.

Methodology: A Latent Space Approach

The proposed framework decomposes the problem into two distinct but interconnected parts using a latent space approach:

1. Behavioral Model

Inspired by behavioral game theory, this model predicts how agents act based on their "latent behavior" (their internal strategy). The researchers specifically utilize a Quantal Response (QLp) model. This accounts for varying levels of agent sophistication; for instance, "Level-0" agents might play randomly, while "Level-1" agents play the best response to Level-0 strategies.

2. Temporal Model

This component describes how the distribution of these latent behaviors evolves over time. A Vector Autoregressive (VAR(1)) model is used to track the shifts in population strategy, allowing the researchers to model the trajectory of adaptation.

Algorithm 1: Bayesian Estimation

The core of the paper is a Bayesian procedure that takes short-term experimental data and fits the behavioral and temporal parameters. It then performs a "pivot" to counterfactual scenarios—simulating what would happen if the entire population were in the treatment group versus the control group. By simulating these scenarios forward in time, the model estimates the long-term Average Causal Effect (CE).

Empirical Validation

The framework was evaluated using repurposed experimental data from a study by Rapoport and Boebel (1992). The data involved a zero-sum, two-agent game with five discrete actions. Forty agents were randomized into two different game designs (different payoff matrices), with each agent playing 240 rounds.

In this setup, the different payoff matrices were treated as "policies." The researchers used the final rounds of the experiment as the "ground truth" long-term outcome to test the model's predictive accuracy against short-term observations.

Key Results

The authors compared their method, LACE (Long-term Average Causal Effect), against a "naive" approach (using the last observed data point) and the standard econometric Difference-in-Differences (DID) method. LACE demonstrated significantly higher accuracy in predicting the long-term outcome:

  • LACE MSE: 0.045
  • Naive MSE: 0.185
  • DID MSE: 0.361

LACE was roughly 4x more accurate than the naive approach and 8x more accurate than DID. This suggests that ignoring the strategic, dynamic nature of agents leads to substantial errors in policy evaluation.

Discussion and Future Work

By using game theory as a structural prior, the authors provide a more robust method for extrapolation than purely statistical trends. This "structural information" dictates what behaviors are likely to be successful and how they should evolve.

However, the model currently relies on several assumptions, such as a fixed set of possible behaviors and a pre-defined time horizon $T$. Future work may involve non-parametric models that learn behaviors "on-the-fly" or incorporating more complex learning mechanisms like reinforcement learning to better represent human psychology in multiagent systems.

Summary updated

Abstract

Planned experiments are the gold standard in reliably comparing the causal effect of switching from a baseline policy to a new policy. One critical shortcoming of classical experimental methods, however, is that they typically do not take into account the dynamic nature of response to policy changes. For instance, in an experiment where we seek to understand the effects of a new ad pricing policy on auction revenue, agents may adapt their bidding in response to the experimental pricing changes. Thus, causal effects of the new pricing policy after such adaptation period, the {\em long-term causal effects}, are not captured by the classical methodology even though they clearly are more indicative of the value of the new policy. Here, we formalize a framework to define and estimate long-term causal effects of policy changes in multiagent economies. Central to our approach is behavioral game theory, which we leverage to formulate the ignorability assumptions that are necessary for causal inference. Under such assumptions we estimate long-term causal effects through a latent space approach, where a behavioral model of how agents act conditional on their latent behaviors is combined with a temporal model of how behaviors evolve over time.
BibTeX
@inproceedings{panagiotis2016long,
  author = {Panagiotis and Toulis and David C. Parkes},
  title = {{Long-term causal effects via behavioral game theory}},
  booktitle = {Digital},
  year = {2016},
}
Figure