Optimizing Randomized and Deterministic Saturation Designs under Interference

Chencheng Cai, Jean Pouget-Abadie, Edoardo M. Airoldi

Abstract

Randomized saturation designs are a family of designs which assign a possibly different treatment proportion to each cluster of a population at random. As a result, they generalize the well-known (stratified) completely randomized designs and the cluster-based randomized designs, which are included as special cases. We show that, under the stable unit treatment value assumption, either the cluster-based or the stratified completely randomized design are in fact optimal for the bias and variance of the difference-in-means estimator among randomized saturation designs. However, this is no longer the case when interference is present. We provide the closed form of the bias and variance of the difference-in-means estimator under a linear model of interference and investigate the optimization of each of these objectives. In addition to the randomized saturation designs, we propose a deterministic saturation design, where the treatment proportion for clusters are fixed, rather than randomized, in order to further improve the estimator under correct model specification. Through simulations, we illustrate the merits of optimizing randomized saturation designs to the graph and potential outcome structure, as well as showcasing the additional improvements yielded by well-chosen deterministic saturation designs.

Introduction

In classical causal inference, researchers typically rely on the Stable Unit Treatment Value Assumption (SUTVA), which posits that the treatment status of one individual does not influence the outcome of another. However, in modern experimental settings—such as social networks, infectious disease studies, or digital marketplaces—this assumption is frequently violated by interference, or spillover effects. When interference is present, the standard difference-in-means estimator becomes biased, and traditional experimental designs may fail to provide precise estimates of the average treatment effect (ATE).

To address this, researchers often employ Randomized Saturation Designs (RSDs). In an RSD, a population is partitioned into clusters (e.g., schools, villages, or geographic regions), and each cluster is assigned a treatment proportion. While RSDs are flexible, there has historically been little guidance on how to optimize the distribution of these proportions. This paper provides a rigorous framework for optimizing these designs to minimize the bias and variance of the ATE estimator under interference.

Theoretical Framework

The Linear Interference Model

The authors analyze the performance of the difference-in-means estimator using a linear interference model. In this model, an individual's outcome is determined by their own treatment status and the proportion of their neighbors who are treated. This allows for a tractable mathematical analysis of how different treatment allocations affect the estimator's error.

Analytical Expressions for Bias and Variance

A primary contribution of this work is the derivation of closed-form formulas for the bias and variance of the estimator. These formulas are expressed as functions of the moments of the treatment proportions—specifically variance, skewness, and kurtosis.

Under SUTVA, the optimization is straightforward: the math dictates that either a stratified design (equal proportions across all clusters) or a cluster-based design (clusters are either 100% treated or 100% control) is always optimal. However, the introduction of interference changes the objective function. The variance is no longer a simple linear function; it becomes dependent on the fourth central moment (kurtosis) of the treatment proportions. This mathematical shift explains why "middle-ground" designs—where clusters receive intermediate treatment proportions—can outperform traditional extreme designs.

Deterministic Saturation Designs (DSD)

Beyond optimizing randomized designs, the paper introduces Deterministic Saturation Designs (DSDs). While RSDs assign proportions to clusters randomly based on a chosen distribution, DSDs fix specific proportions for specific clusters. By leveraging known cluster characteristics, such as internal connectivity or "vulnerability" to influence, researchers can tailor the design to the specific graph structure. This approach further reduces Mean Squared Error (MSE) compared to randomized approaches, provided the researcher has accurate prior data on the cluster statistics.

Simulation Results

The theoretical findings were validated through two primary synthetic simulations:

  1. Stochastic Block Model (SBM): Using a population of 2,000 units across 40 clusters, the authors simulated a graph where edge probability decays with distance. The optimized RSD reduced the estimator's variance by approximately 17% compared to a standard stratified design.
  2. Perfect Clustering: In a scenario where interference is strictly contained within clusters, the authors compared DSDs against RSDs. The results showed that DSDs consistently outperformed randomized designs because they could adapt to the specific influence levels of each cluster.

Key Insights

  • The SUTVA Threshold: The authors identify a threshold (denoted as $\gamma'$) based on clustering quality. If clustering is "better than random," cluster-based designs minimize bias. If it is "worse than random," stratified designs are preferable.
  • The Plug-in Advantage: Optimization does not require knowledge of every individual's potential outcome. Knowing cluster-level statistics (intra-cluster vs. inter-cluster variance) is sufficient to significantly improve experimental precision.
  • Complexity of Interference: Interference necessitates more sophisticated designs. The optimal treatment distribution is often "curved," favoring intermediate proportions that balance the trade-offs between bias and variance.

Limitations and Future Work

The current optimization framework relies on a linear model of interference. If the true spillover effects are highly non-linear (e.g., threshold effects), these specific optimizations may require adjustment. Additionally, DSDs require accurate "plug-in" estimates of cluster statistics; incorrect estimates could lead to sub-optimal performance. Future research may explore model-assisted estimators that remain robust even when the underlying interference model is misspecified, as well as extending these results to more complex random graph models.

Summary updated

Abstract

Randomized saturation designs are a family of designs which assign a possibly different treatment proportion to each cluster of a population at random. As a result, they generalize the well-known (stratified) completely randomized designs and the cluster-based randomized designs, which are included as special cases. We show that, under the stable unit treatment value assumption, either the cluster-based or the stratified completely randomized design are in fact optimal for the bias and variance of the difference-in-means estimator among randomized saturation designs. However, this is no longer the case when interference is present. We provide the closed form of the bias and variance of the difference-in-means estimator under a linear model of interference and investigate the optimization of each of these objectives. In addition to the randomized saturation designs, we propose a deterministic saturation design, where the treatment proportion for clusters are fixed, rather than randomized, in order to further improve the estimator under correct model specification. Through simulations, we illustrate the merits of optimizing randomized saturation designs to the graph and potential outcome structure, as well as showcasing the additional improvements yielded by well-chosen deterministic saturation designs.
BibTeX
@misc{cai2022optimizing,
  author = {Chencheng Cai and Jean Pouget-Abadie and Edoardo M. Airoldi},
  title = {{Optimizing Randomized and Deterministic Saturation Designs under Interference}},
  year = {2022},
  eprint = {2203.09682},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2203.09682},
}
Figure