Individualized Policy Evaluation and Learning under Clustered Network Interference

Yi Zhang, Kosuke Imai

TL;DR

Researchers Yi Zhang and Kosuke Imai have developed a new framework for learning individualized treatment rules (ITRs) that accounts for "spillover effects"—where treating one person affects their neighbors. By assuming a flexible, additive structure for these effects within clusters (like households or classrooms), their new "addIPW" and "addDR" estimators achieve significantly higher statistical efficiency and better policy performance than standard methods that ignore network relationships.

What Problem Does This Paper Address?

In classical policy learning, we usually assume the "Stable Unit Treatment Value Assumption" (SUTVA), which is a fancy way of saying: "What happens to you only depends on the treatment you received."

In the real world, this is often false. If you give a student an anti-bullying intervention, it might improve the behavior of their friends, too. This is called interference or spillover effects. If a policy designer ignores these effects, they might miss out on "influential" individuals who could amplify the policy's impact.

However, accounting for interference is hard. In a cluster of 10 people, there are $2^{10}$ (1,024) possible treatment combinations. Standard methods (like Inverse Probability Weighting) become incredibly "noisy" because they often require the entire cluster's treatment to exactly match the proposed policy to learn anything. Most previous attempts to fix this assumed "anonymous interference"—the idea that it only matters how many of your neighbors are treated, not who they are. This paper aims to bridge the gap: accounting for interference without making overly restrictive assumptions about who influences whom.

Methods

The authors introduce a Heterogeneous Additive Outcome Model. The core idea is that an individual’s outcome is an additive function of their own treatment and the individual treatments of everyone else in their cluster. Crucially, the "weight" of each person's influence can be different, allowing the model to recognize that a best friend has more influence than a distant acquaintance.

Based on this model, they propose two key estimators:

  1. addIPW (Additive Inverse-Probability Weighting): Unlike standard IPW, which scales poorly as clusters get larger, addIPW leverages the additive structure to use data more efficiently. It allows the researcher to learn from clusters even if the realized treatment assignment doesn't perfectly match the target policy.
  2. addDR (Additive Doubly Robust): For observational studies where the "propensity score" (the probability of being treated) is unknown, this estimator combines the outcome model and the propensity model. It remains accurate if at least one of those models is correctly specified.

To actually find the best policy, the authors show that the problem can be framed as a Mixed-Integer Program (MIP), allowing researchers to use off-the-shelf optimization tools to find the optimal rule.

Datasets

The paper evaluates the methodology using:

  • Synthetic Simulations: Two scenarios (A and B) where they test the estimators against varying cluster sizes and degrees of model misspecification.
  • Colombian Schoolchildren Data: A dataset from a randomized experiment in Bogota, Colombia (Barrera-Osorio et al., 2011), involving 1,010 households and 2,129 students. The goal was to learn an ITR that maximizes school attendance by strategically assigning conditional cash transfers.

Key Insights and Ideas

  • Breaking the Curse of Dimensionality: The "Aha!" moment is that by assuming additivity, the complexity of the spillover effects grows linearly with cluster size rather than exponentially. This makes it possible to estimate effects that were previously too "thin" to see in the data.
  • Who Matters: By moving away from "anonymous interference," the model can prioritize treating "hubs" or influential units within a network.
  • Robustness to Misspecification: Even if the true world isn't perfectly additive (e.g., there are complex interactions between who gets treated), the additive model acts as a "best linear approximation." The authors show that this "biased but low-variance" approach often produces better real-world policies than "unbiased but high-variance" standard methods.

Key Results

  • Optimal Convergence: The authors prove that their learner achieves a "regret bound" (a measure of how much worse the learned policy is compared to the perfect one) that converges at a rate of $1/\sqrt{n}$. This matches the optimal rate for standard settings without interference.
  • Efficiency Gains: In simulations, the addIPW and addDR estimators showed substantially lower variance than standard IPW.
  • Empirical Success: In the Colombian study, the addIPW policy consistently outperformed policies learned via standard methods across different treatment cost scenarios. The learned rule suggested prioritizing children in lower grades who live in larger, poorer households—capturing the nuances of where the cash transfer would have the most significant "bang for the buck."

Limitations and Future Work

  • Clustered Only: The current framework assumes "partial interference," meaning people only influence others within their specific group (e.g., household). It does not yet account for "full" networks where everyone might influence everyone else (like a whole city or a social media platform).
  • Additivity Assumption: While robust, the model specifically rules out "interaction" effects (e.g., the effect of treating Child A might change depending on whether Child B is also treated). The authors suggest that extending this to polynomial models with interactions is a logical next step.
  • Computational Complexity: While MIPs are powerful, solving them for extremely large datasets or very complex policy classes can be computationally expensive. The authors suggest smooth stochastic approximations as a faster alternative.

Summary updated

Abstract

Although there is now a large literature on policy evaluation and learning, much of the prior work assumes that the treatment assignment of one unit does not affect the outcome of another unit. Unfortunately, ignoring interference can lead to biased policy evaluation and ineffective learned policies. For example, treating influential individuals who have many friends can generate positive spillover effects, thereby improving the overall performance of an individualized treatment rule (ITR). We consider the problem of evaluating and learning an optimal ITR under clustered network interference (also known as partial interference), where clusters of units are sampled from a population and units may influence one another within each cluster. Unlike previous methods that impose strong restrictions on spillover effects, such as anonymous interference, the proposed methodology only assumes a semiparametric structural model, where each unit's outcome is an additive function of individual treatments within the cluster. Under this model, we propose an estimator that can be used to evaluate the empirical performance of an ITR. We show that this estimator is substantially more efficient than the standard inverse probability weighting estimator, which does not impose any assumption about spillover effects. We derive the finite-sample regret bound for a learned ITR, showing that the use of our efficient evaluation estimator leads to the improved performance of learned policies. We consider both experimental and observational studies, and for the latter, we develop a doubly robust estimator that is semiparametrically efficient and yields an optimal regret bound. Finally, we conduct simulation and empirical studies to illustrate the advantages of the proposed methodology.
BibTeX
@misc{zhang2023individualized,
  author = {Yi Zhang and Kosuke Imai},
  title = {{Individualized Policy Evaluation and Learning under Clustered Network Interference}},
  year = {2023},
  eprint = {2311.02467},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2311.02467},
}
Figure