Budget Partitioning for Interference Free Marketplace Experiments

This direction is merged into another direction: network interference. Its last version stays readable and citable here.

Version 1 of 112 papers21 conceptsfrozen 2026-10-04

In a standard A/B test, we assume that the treatment applied to one person has no effect on another. This is known as the Stable Unit Treatment Value Assumption (SUTVA). However, in modern two-sided marketplaces like Uber, Airbnb, or Google Ads, this assumption is almost always violated. If a platform gives a discount to one group of riders, they may snap up all the available drivers, leaving the control group with longer wait times and higher prices.

This phenomenon, often called marketplace interference or cannibalization bias, makes it impossible to measure the true "incremental" effect of a change. If the treatment group wins at the expense of the control group, the experiment will overestimate the benefit of the intervention.

The Resource Competition Problem

Interference in marketplaces is fundamentally driven by competition for finite resources. As Guillaume W. Basse, Hossein Azari Soufiani, and Diane Lambert demonstrate in Randomization and The Pernicious Effects of Limited Budgets on Auction Experiments, limited advertiser budgets create a zero-sum game. When an advertiser spends their budget in a "Treatment" auction, they have less money left for the "Control" auction.

To solve this, researchers have moved toward budget-split designs. Instead of just randomizing users, the platform partitions the underlying resources. Min Liu, Jialiang Mao, and Kang Kang describe this in Trustworthy Online Marketplace Experimentation with Budget-split Design as creating two independent, parallel "mini-marketplaces." By splitting both the supply (e.g., ad slots) and the demand-side budgets, actions in one group cannot consume resources allocated to the other.

From Heuristics to Market Equilibria

While budget splitting is intuitive, it was long treated as a heuristic. Recent work has sought to provide a rigorous statistical foundation for these designs by modeling markets as systems in equilibrium.

Christian Kroer and Luofeng Liao have pioneered the use of First-Price Pacing Equilibria (FPPE) to analyze these experiments. In Statistical Inference and A/B Testing for First-Price Pacing Equilibria, they show that budget splitting is not just a practical fix but is actually "asymptotically local minimax optimal." By treating the market equilibrium as a mathematical estimator, they derived the first Central Limit Theorems that allow researchers to calculate valid confidence intervals even when agents are strategic and budgets are constrained.

Managing Interference at Scale

In large-scale systems, interference isn't just a statistical problem; it's an engineering one. Frameworks like PlanOut, developed by Eytan Bakshy, Dean Eckles, and Michael S. Bernstein, manage this by partitioning populations into discrete segments within a shared namespace. This prevents concurrent experiments from contaminating each other's results.

However, even with partitioning, some "leakage" often remains. In Interference Among First-Price Pacing Equilibria: A Bias and Variance Analysis, Luofeng Liao and colleagues introduce debiased estimators. These mathematical tools "undo" the bias caused by cross-market interference, allowing platforms to run many parallel tests simultaneously without sacrificing the accuracy of their treatment effect estimates.

Optimal Clustering in Bipartite Markets

When a market is structured as a bipartite graph (e.g., hosts and guests on Airbnb), the interference patterns can be complex. Jennifer Brennan and Jean Pouget-Abadie suggest that the best way to minimize bias is through cluster-randomized designs. By grouping units that compete for the same resources into the same treatment or control cluster, researchers can ensure that the "interference" happens within the group rather than across the experimental boundary, preserving the integrity of the comparison.

By combining resource partitioning with equilibrium theory, researchers can finally move beyond simple A/B tests to conduct rigorous experiments in the world's most competitive digital environments.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_budget_partitioning_for_interference_fre_af244396,
  title = {Budget Partitioning for Interference Free Marketplace Experiments},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/budget-partitioning-for-interference-free-marketplace-experiments/rdr_af244396c1f644bfab4abd9264a3e0ba},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_budget_partitioning_for_interference_fre_af244396_v1,
  title = {Budget Partitioning for Interference Free Marketplace Experiments},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/budget-partitioning-for-interference-free-marketplace-experiments/rdr_af244396c1f644bfab4abd9264a3e0ba/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}