Low-order outcomes and clustered designs: combining design and analysis for causal inference under network interference

Matthew Eichhorn, Samir Khan, Johan Ugander, Christina Lee YuarXiv 2024 · DOI · arXiv

Abstract

Variance reduction for causal inference in the presence of network interference is often achieved through either outcome modeling, typically analyzed under unit-randomized Bernoulli designs, or clustered experimental designs, typically analyzed without strong parametric assumptions. In this work, we study the intersection of these two approaches and make the following threefold contributions. First, we present an estimator of the total treatment effect (or global average treatment effect) in low-order outcome models when the data are collected under general experimental designs, generalizing previous results for Bernoulli designs. We refer to this estimator as the pseudoinverse estimator and give bounds on its bias and variance in terms of properties of the experimental design. Second, we evaluate these bounds for the case of Bernoulli graph cluster randomized (GCR) designs. Its variance scales like the smaller of the variance obtained by the estimator derived under a low-order assumption, and the variance obtained from cluster randomization, showing that combining these variance reduction strategies is preferable to using either individually. When the order of the potential outcomes model is correctly specified, our estimator is always unbiased, and under a misspecified model, we upper bound the bias by the closeness of the ground truth model to a low-order model. Third, we give empirical evidence that our variance bounds can be used to select a good clustering that minimizes the worst-case variance under a cluster randomized design from a set of candidate clusterings. Across a range of graphs and clustering algorithms, our method consistently selects clusterings that perform well on a range of response models, suggesting the practical use of our bounds.

Introduction

In traditional causal inference, researchers often rely on the Stable Unit Treatment Value Assumption (SUTVA), which posits that one individual's treatment status does not influence another's outcome. However, in interconnected systems—such as social networks, infectious disease pathways, or digital marketplaces—this assumption frequently fails. This phenomenon, known as network interference, makes estimating the Total Treatment Effect (TTE) exceptionally difficult. Because the outcome of a single node can depend on the treatment status of every other node in the network, the number of possible treatment combinations grows exponentially with the size of the population.

To manage this complexity, researchers typically adopt one of two strategies: outcome modeling (assuming simple, localized spillover effects) or clustered experimental design (grouping neighbors into clusters to minimize boundary interference). Historically, these two strategies have been studied in isolation. This paper introduces a unified framework that combines these approaches through a new "pseudoinverse estimator," achieving significant variance reduction and providing a robust method for experimental design selection.

The Pseudoinverse Estimator

The core contribution of this work is the $\beta$-order pseudoinverse estimator. This estimator bridges the gap between structural assumptions and experimental design through two pillars:

  1. Low-Order Interaction Models: The authors assume that an individual's outcome is a linear combination of effects from subsets of their neighbors of size at most $\beta$. For instance, a $\beta=1$ model assumes an outcome depends on the individual's treatment and the individual treatments of their neighbors, but ignores complex interactions between pairs or triplets of neighbors.
  2. General Experimental Designs: Unlike previous estimators limited to simple Bernoulli (coin-flip) randomization, the pseudoinverse estimator is compatible with complex designs like Graph Cluster Randomized (GCR) designs, where treatment is assigned at the cluster level.

The estimator utilizes the Moore-Penrose pseudoinverse of a design matrix that captures the joint treatment probabilities of the network. This mathematical approach ensures the estimator remains valid even when the experimental design makes certain treatment combinations impossible, resulting in a rank-deficient design matrix.

Theoretical Insights: The Synergy Effect

A key finding of the research is the "synergy effect" between modeling and design. The variance of the pseudoinverse estimator scales with the minimum of the variance derived from the model and the variance derived from the design. This creates a dual layer of protection: if the clustering is suboptimal, the low-order model maintains precision; if the model is slightly misspecified, the clustered design keeps the variance in check.

This synergy is particularly visible when examining different parts of the network. For units in the "interior" of a cluster (influenced primarily by members of their own cluster), the experimental design performs the heavy lifting. For units on the "boundaries" (influenced by multiple clusters), the low-order model prevents the variance from exploding, which is a common failure point for traditional design-based estimators.

Furthermore, the estimator is robust to misspecification. Even if a researcher assumes a simple $\beta=1$ model in a world that is actually $\beta=2$, the bias is bounded by the magnitude of the higher-order effects. In many practical scenarios, the massive reduction in variance offered by a simple model outweighs the minor increase in bias.

Empirical Validation and Design Selection

The authors validated their approach using synthetic Stochastic Block Models (SBM) and real-world datasets, including a village social network from China (3,677 individuals) and the Caltech Facebook friendship graph (762 individuals). They tested these against various response models, including those with no effect, weak effects, and degree-proportional effects.

The results demonstrated that the pseudoinverse estimator consistently outperformed the standard Horvitz-Thompson estimator—the traditional benchmark for design-based inference—across all tested graphs and clusterings. Notably, the estimator proved far less sensitive to the quality of the clustering than traditional methods, making it highly suitable for real-world graphs where perfect clusters are rare.

Beyond estimation, the authors show that their theoretical variance bounds can serve as a "fitness score" to select the best clustering algorithm (e.g., Louvain vs. METIS) for a specific graph before an experiment begins. In simulations, this method consistently selected designs that performed nearly as well as an "oracle" with prior knowledge of the true response model.

Conclusion

By combining low-order outcome modeling with clustered designs, this work provides a practical and theoretically grounded framework for causal inference in networks. The pseudoinverse estimator offers a path forward for researchers to achieve high-precision estimates of treatment effects even in the presence of complex interference. Future work may further refine these results by exploring the spectral properties of design matrices and investigating alternative basis functions for defining "low-order" interactions.

Summary updated

Abstract

Variance reduction for causal inference in the presence of network interference is often achieved through either outcome modeling, typically analyzed under unit-randomized Bernoulli designs, or clustered experimental designs, typically analyzed without strong parametric assumptions. In this work, we study the intersection of these two approaches and make the following threefold contributions. First, we present an estimator of the total treatment effect (or global average treatment effect) in low-order outcome models when the data are collected under general experimental designs, generalizing previous results for Bernoulli designs. We refer to this estimator as the pseudoinverse estimator and give bounds on its bias and variance in terms of properties of the experimental design. Second, we evaluate these bounds for the case of Bernoulli graph cluster randomized (GCR) designs. Its variance scales like the smaller of the variance obtained by the estimator derived under a low-order assumption, and the variance obtained from cluster randomization, showing that combining these variance reduction strategies is preferable to using either individually. When the order of the potential outcomes model is correctly specified, our estimator is always unbiased, and under a misspecified model, we upper bound the bias by the closeness of the ground truth model to a low-order model. Third, we give empirical evidence that our variance bounds can be used to select a good clustering that minimizes the worst-case variance under a cluster randomized design from a set of candidate clusterings. Across a range of graphs and clustering algorithms, our method consistently selects clusterings that perform well on a range of response models, suggesting the practical use of our bounds.
BibTeX
@misc{eichhorn2024low,
  author = {Matthew Eichhorn and Samir Khan and Johan Ugander and Christina Lee Yu},
  title = {{Low-order outcomes and clustered designs: combining design and analysis for causal inference under network interference}},
  year = {2024},
  eprint = {2405.07979},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2405.07979},
}
Figure