On the admissibility of Horvitz-Thompson estimator for estimating causal effects under network interference

Vishesh Karwa, Edoardo M. AiroldiarXiv 2023 · DOI · arXiv

TL;DR

Researchers have discovered that the standard Horvitz-Thompson (H-T) estimator—a cornerstone of causal inference—is "inadmissible" (suboptimal in terms of Mean Squared Error) when estimating effects in networks. To fix this, they introduce the Conditional Horvitz-Thompson (CHT) estimator, which achieves both unbiasedness and optimality by conditioning on the realized number of units in different network exposure states.

What Problem Does This Paper Address?

In classical causal inference, we often assume "no interference" (SUTVA), meaning one person’s treatment doesn't affect another’s outcome. In the real world—social networks, infectious diseases, or marketplaces—this assumption fails. If I get vaccinated, your risk of infection drops; if my friend sees an ad, they might tell me about it.

To handle this "network interference," researchers use the Horvitz-Thompson (H-T) estimator because it provides unbiased estimates of causal effects even when the interference patterns are complex. However, the authors identify a major theoretical gap: the H-T estimator is often inadmissible. In statistical terms, an estimator is inadmissible if there exists another estimator that has a lower Mean Squared Error (MSE) across all possible scenarios.

The culprit? Random exposure designs. In a network, even if you randomly assign treatment to individuals (like a Bernoulli design), the number of people who end up with "at least one treated neighbor" is a random variable. The H-T estimator fails to account for this "extra" randomness, leading to unnecessarily high variance.

Methods

The authors propose a shift from unconditional to conditional inference. Their primary contribution is the Conditional Horvitz-Thompson (CHT) estimator.

The CHT estimator uses propensity scores (the probability of being in a certain exposure state) conditional on the realized number of units in each treatment-exposure combination. By treating these counts as an "ancillary statistic," the CHT effectively transforms a random exposure design into a fixed exposure design, which the authors prove makes the estimator admissible.

Because calculating these conditional probabilities exactly is computationally "hard" (NP-hard in many network settings), the authors develop three practical approximations:

  1. MCMC Approximation: Sampling from the conditional distribution of treatment assignments.
  2. Ratio-based Approximation: A closed-form approach that relates CHT to the well-known Ratio estimator.
  3. Difference-in-Means Approximation: A simple version that avoids computing propensity scores entirely under certain exchangeability conditions.

Datasets

The authors evaluate their theory and estimators using the Add Health (National Longitudinal Study of Adolescent Health) dataset.

  • Scope: They used 144 friendship networks from different schools, with an average size of approximately 600 nodes per network.
  • Setup: They simulated a "2x2 exposure model" (where a unit is "exposed" if at least one neighbor is treated) and applied various causal contrasts like Total Treatment Effect (TTE) and Average Direct Effect (ADE).

Key Insights and Ideas

  • The "Random Exposure" Trap: The paper highlights that standard experimental designs (like Bernoulli trials) become "random exposure designs" in networks. Because we don't directly control the network-level exposure counts, the standard H-T estimator "wastes" information.
  • Admissibility vs. Unbiasedness: While H-T is famous for being unbiased, the authors show that in network settings, you can trade off a tiny bit of complexity to get an estimator (CHT) that is both unbiased and has lower variance.
  • The Power of Conditioning: The "aha moment" is that the realized number of units in an exposure state doesn't carry information about the causal effect itself, but it does affect the precision of our estimate. Conditioning on it is a classic statistical "trick" that yields a more powerful tool.

Key Results

In numerical simulations on the Add Health networks, the results were striking:

  • MSE Reduction: The approximate CHT estimator consistently achieved significantly lower Root Mean Squared Error (RMSE) than the standard H-T estimator across almost all causal estimands.
  • Comparison with Hájek: The Hájek and Ratio estimators (which are commonly used as "better" versions of H-T) were also shown to be admissible, but the CHT provides a more formal, unbiased framework for achieving that optimality.
  • Practicality: The MCMC and analytical approximations performed nearly as well as the theoretical ideal, suggesting that practitioners don't need to solve the exact (and difficult) conditional probabilities to see the benefits.

Limitations and Future Work

  • Model Dependence: The CHT estimator (like H-T) requires the researcher to specify an "exposure model" (e.g., "I am affected if >2 neighbors are treated"). If this model is wrong, the estimator may become biased.
  • Computational Cost: While the approximations help, MCMC can still be slow for massive graphs (millions of nodes).
  • Unknown Networks: The current work assumes the network structure is known at the time of analysis. A major future direction is developing admissible estimators for cases where the network is only partially observed or estimated from data.

Summary updated

Abstract

The Horvitz-Thompson (H-T) estimator is widely used for estimating network causal effects. We study its optimality properties by embedding it in the class of all linear estimators. We show that, under any form of interference, the H-T estimator is inadmissible with respect to mean squared error for designs that generate a random number of units for a given treatment-exposure combination, which includes completely randomized and Bernoulli designs. In contrast, the Hajek and Ratio estimators are admissible under such designs as long as their weights exceed one. We show that the H-T estimator becomes admissible under restricted randomization schemes that fix treatment-exposure counts. These findings motivate a new estimator, the conditional horvitz-Thompson (CHT) estimator, which uses propensity scores conditional on the realized number of units in each treatment-exposure combination. We prove that the CHT estimator is unbiased and admissible for network causal effects, for any design, within the class of linear estimators. Because computing conditional propensity scores can be challenging in practice, we develop three practical approximations: an MCMC approximation and two closed-form approximations based on the difference-in-means and Ratio estimators, the first one avoids computing any propensity scores. Numerical experiments on the Add Health network illustrate the theoretical results.
BibTeX
@misc{karwa2023admissibility,
  author = {Vishesh Karwa and Edoardo M. Airoldi},
  title = {{On the admissibility of Horvitz-Thompson estimator for estimating causal effects under network interference}},
  year = {2023},
  eprint = {2312.01234},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.01234},
}
Figure