Treatment Effects with Targeting Instruments

Sokbae Lee, Bernard SalaniéarXiv 2020 · DOI · arXiv

Introduction

In policy evaluation, the standard approach often relies on binary models: a treatment is either received or not, and an instrument (such as a lottery) either encourages participation or does not. However, real-world scenarios are frequently more complex. Individuals often choose between multiple options—for instance, selecting between a public program, a private alternative, or no program at all. In their paper, "Treatment Effects with Targeting Instruments," Sokbae Lee and Bernard Salanié provide a new framework for these multivalued settings, introducing the concept of "targeting" to identify treatment effects without relying on restrictive parametric models.

The Challenge of Multivalued Treatments

Standard econometric tools, such as the Local Average Treatment Effect (LATE), are designed for binary settings. When an individual faces multiple choices, a single instrument can trigger complex movements between several categories simultaneously. Without a sufficient number of "levers" (instruments), it becomes difficult to isolate which specific transitions are occurring and why.

Previous attempts to address this complexity often relied on heavy parametric assumptions—such as assuming unobserved factors follow a normal distribution—or resulted in models too complex for transparent application. Lee and Salanié propose a "parsimonious framework" that maintains mathematical rigor while remaining accessible for applied research.

The Targeting Framework

The authors build their model on the Additive Random-Utility Model (ARUM). In this setup, individuals choose a treatment that maximizes their utility, which is composed of observed mean values and unobserved personal shocks. The core innovation lies in three levels of "Targeting Assumptions":

  • Strict Targeting: An instrument specific to treatment $A$ only affects the mean utility of $A$, leaving the appeal of other options unchanged.
  • One-to-One Targeting: Each instrument targets exactly one treatment, and each treatment is targeted by exactly one instrument.
  • Universal Targeting: A combination of the above, creating a clear mapping between policy interventions and choices.

Crucially, these targeting assumptions are testable. Researchers can examine propensity scores—the probability of choosing a treatment given an instrument—to verify if the data supports the targeting hypothesis.

Incorporating Positive Selection

To further refine their estimates, the authors introduce the concept of Positive Selection. This is the intuitive economic assumption that "always-takers" (those who would choose a treatment even without an incentive) likely derive higher utility or better outcomes from that treatment than "compliers" (those who only choose it due to the incentive).

By applying this logic, the authors can shrink the "identified set"—the range of possible treatment effects—into tighter, more informative bounds. This allows for meaningful inference without needing to assume a specific functional form (like a normal distribution) for the unobserved heterogeneity.

Empirical Application: The Head Start Impact Study

The authors demonstrate the utility of their framework by revisiting the Head Start Impact Study (HSIS). The HSIS was a randomized trial where children were offered or denied admission to Head Start. However, the treatment was effectively multivalued: children denied Head Start often enrolled in other "substitute" preschool centers rather than staying home. This created a ternary choice: Head Start, other centers, or no preschool.

In this context, the "substitution effect" is critical. The benefit of Head Start depends significantly on the alternative; the program is typically more beneficial for a child who would otherwise have had no preschool than for a child who would have attended a different high-quality center.

Key Results and Policy Implications

By applying their non-parametric bounds to the HSIS data, Lee and Salanié found that previous point estimates for Head Start’s benefits might have been overly optimistic. Their analysis suggests that the large differences between complier groups found in earlier studies (such as Kline and Walters, 2016) can only be justified if one assumes "negative selection" into Head Start—an assumption that is often less plausible than positive selection.

The authors also calculated the Marginal Value of Public Funds (MVPF) for expanding Head Start. Their estimated upper bound for the MVPF was 1.55, notably lower than the previous point estimate of 2.02. While still indicating a positive return on investment, this suggests a more conservative valuation of the program's marginal expansion.

Conclusion

Lee and Salanié’s framework offers a robust alternative to "black box" parametric models for evaluating complex policy choices. By focusing on the concepts of targeting and positive selection, they provide a method to derive informative bounds on treatment effects that are both economically grounded and empirically testable. This approach is particularly valuable for researchers evaluating programs where multiple substitutes exist and where understanding the heterogeneity of responses is critical for effective policy design.

Summary updated

Abstract

Multivalued treatments are commonplace in applications. We explore the use of discrete-valued instruments to control for selection bias in this setting. Our discussion revolves around the concept of targeting: which instruments target which treatments. It allows us to establish conditions under which counterfactual averages and treatment effects are point- or partially-identified for composite complier groups. We explore the additional identifying power of a positive selection assumption. We illustrate its usefulness by revisiting the findings of Kline and Walters (2016) on the Head Start Impact Study. We derive informative bounds that suggest less beneficial effects of Head Start expansions than their parametric estimates.
BibTeX
@misc{lee2020treatment,
  author = {Sokbae Lee and Bernard Salanié},
  title = {{Treatment Effects with Targeting Instruments}},
  year = {2020},
  eprint = {2007.10432},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2007.10432},
}
Figure