Clustered Experimental Design under Interference

This direction is merged into another direction: network interference. Its last version stays readable and citable here.

Version 1 of 116 papers28 conceptsfrozen 2026-10-04

In standard A/B testing, we rely on the Stable Unit Treatment Value Assumption (SUTVA): the idea that the treatment assigned to one user has no effect on the outcome of another. In the real world, this assumption is frequently violated. On social networks, a new feature given to one person might change how they interact with their friends. In marketplaces like Uber or Airbnb, a promotion for one driver might reduce the available rides for another.

These "spillovers" or "network interference" create a significant problem for causal inference. If the treatment "leaks" from the treatment group to the control group, the difference between the two groups no longer represents the true impact of the feature. To solve this, researchers use clustered experimental design, a strategy that groups interconnected units together to "contain" the interference.

The Mechanism of Containment

The core intuition behind clustered design is to align the boundaries of the experiment with the boundaries of human interaction. If we can identify a group of users who interact heavily with each other but rarely with anyone outside that group, we can treat that entire group as a single experimental unit.

As Brian Karrer, Dean Eckles, and Johan Ugander demonstrate in Design and analysis of experiments in networks: Reducing bias from interference, randomizing at the cluster level rather than the individual level ensures that most "interference" happens within the cluster. Since everyone in a treatment cluster receives the treatment, the internal spillover doesn't contaminate a control group.

The definition of a "cluster" depends on the source of interference:

  • Social Networks: Clusters are communities of friends or followers.
  • Geography: David Rolnick, Vahab Mirrokni, and colleagues developed GeoCUTS, which uses travel statistics to cluster regions. By minimizing the number of people traveling between treatment and control zones, they reduce geographic interference.
  • Bipartite Markets: In platforms where buyers and sellers interact, Jennifer Brennan and Jean Pouget-Abadie show in Cluster Randomized Designs for One-Sided Bipartite Experiments that clustering one side of the market (e.g., items or sellers) can mitigate the bias caused by competition for shared resources.

The Fundamental Bias-Variance Trade-off

While clustering reduces bias, it introduces a new problem: it increases variance. This creates a fundamental tension that governs all network experimental design.

  1. The Bias Problem: Small clusters (or individual randomization) lead to many "edges" between treatment and control units. This results in high interference bias because the treatment effect leaks across these boundaries.
  2. The Variance Problem: Large clusters reduce the number of independent "dice rolls" in the experiment. If you have 1,000 users but group them into 2 massive clusters, you effectively have a sample size of $N=2$. This leads to high variance, making it difficult to determine if the observed effect is statistically significant.

Davide Viviano and Lihua Lei formalize this in Causal clustering: design of cluster experiments under network interference, arguing that the optimal design must balance "cluster purity" (minimizing cross-boundary edges) against the sum of squared cluster sizes (minimizing variance).

This trade-off often results in a V-shaped error curve. As Daniel Guzman Olivares and colleagues observe in Clustering Context in Off-Policy Evaluation, having too few clusters oversimplifies the data (high bias), while having too many clusters approaches point-wise estimation with insufficient data support (high variance).

Beyond Hard Partitions

Modern research is moving toward more flexible ways to manage this trade-off. Instead of just picking "hard" clusters, researchers are optimizing the structure of the randomization itself.

For example, Qianyi Chen and Yong Wang propose Optimized Covariance Design (OCD). Rather than assigning treatment to fixed clusters, they optimize the correlation between treatment assignments of connected units. This allows for a "soft" clustering that can more precisely balance the need for isolation (to reduce bias) with the need for a large effective sample size (to reduce variance).

Similarly, in settings with high-dimensional data, researchers like Jonas Schweisthal and Niki Kilbertus are learning representations to partition complex spaces. Whether the "network" is a social graph, a geographic map, or a latent space of user behaviors, the goal remains the same: find the partitions that maximize our ability to see the true effect through the noise of human interconnectedness.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_clustered_experimental_design_under_inte_76bbaf57,
  title = {Clustered Experimental Design under Interference},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/clustered-experimental-design-under-interference/rdr_76bbaf578b784813b0d2269d57310f97},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_clustered_experimental_design_under_inte_76bbaf57_v1,
  title = {Clustered Experimental Design under Interference},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/clustered-experimental-design-under-interference/rdr_76bbaf578b784813b0d2269d57310f97/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}