In an ideal experiment, every participant does exactly what they are told. If you assign a student to use a GPT-4 tutor, they use it; if you assign them to a control group, they don't. In reality, people are "noncompliant." Some students assigned the tutor will ignore it, while some in the control group might find a way to use it anyway.
When compliance is imperfect, the simple difference between the "assigned" groups no longer measures the effect of the treatment itself. Instead, it measures the effect of the offer of treatment. To find the actual effect of the intervention, researchers like Joshua D. Angrist and Guido Imbens developed a framework to isolate the effect for a specific group: the Compliers.
The Logic of the Instrument
To solve non-compliance, we use an Instrumental Variable (IV). An instrument is a "nudge"—something that affects whether a person takes the treatment but has no direct effect on the outcome.
For example, in a study on generative AI in the workplace, Eleanor Wiske Dillon, Sonia Jaffe, and Christopher Stanton used random assignment to a "Copilot" software rollout as an instrument. Being assigned the software makes you more likely to use it, but the assignment itself doesn't write your emails—only the usage of the software does.
As Joshua D. Angrist, Guido Imbens, and Donald B. Rubin formalized in Identification of Causal Effects Using Instrumental Variables, this setup allows us to partition the population into four latent groups based on how they respond to the instrument:
- Compliers: People who take the treatment if nudged, but not otherwise.
- Always-takers: People who take the treatment regardless of the nudge.
- Never-takers: People who refuse the treatment regardless of the nudge.
- Defiers: People who do the exact opposite of the nudge (usually assumed not to exist).
Identifying the Local Effect
The core insight of the Local Average Treatment Effect (LATE) is that the instrument only provides information about the Compliers.
Think of it as a ratio. If offering a GPT tutor increases average exam scores by 2 points (the "Intent-to-Treat" effect), but only 50% of the students actually used the tutor because of that offer (the "Compliance" rate), then the effect on those who actually changed their behavior must be 4 points ($2 / 0.5$).
This 4-point gain is the LATE. As Allen Nie and colleagues demonstrate in The GPT Surprise, this allows researchers to "scale up" the observed results to account for the fact that not everyone used the tool. This isolates the causal impact of usage from the noise of non-usage.
Why "Local" Matters
The effect is called "local" because it only applies to the compliers. We learn nothing about the "Always-takers" (who would have used GPT anyway) or the "Never-takers" (who would never use it).
This specificity is often a point of contention. Critics argue that policy-makers want to know the effect on the whole population. However, in Better LATE Than Nothing, Guido Imbens defends this focus. He argues that LATE provides a "gold-standard" of internal validity. It tells us exactly what happens to the people the policy actually reaches.
This framework has been extended to complex scenarios beyond simple binary choices:
- Variable Intensity: Joshua D. Angrist extended the logic to treatments with different "dosages," like years of schooling, in Average Causal Response with Variable Treatment Intensity.
- Distributional Impacts: Alberto Abadie and Joshua D. Angrist showed how to estimate how a treatment affects the entire distribution (e.g., the gap between high and low performers) for compliers in Instrumental Variables Estimation of Quantile Treatment Effects.
- Market Design: In Breaking Ties, Atila Abdulkadiroğlu and colleagues show that the "tie-breaking" lotteries used in school assignments act as natural instruments, allowing researchers to measure the effect of attending a specific school even when students aren't randomly assigned.
Intuition: The Nudge as a Filter
The LATE framework treats the instrument as a filter. By looking at how the outcome changes only when the instrument changes the treatment status, we filter out the "self-selection" bias of people who would have taken the treatment anyway.
Whether it is evaluating SNAP outreach programs (Elizabeth Stuart et al., /paper/art_5fb7154fb2c143ce9546b2151411bf0b) or the impact of civil communication on gaming platforms (Lyle Ungar et al., /paper/art_701a7332f2e747538350ee76a0c0818a), the goal remains the same: identifying the causal effect for the specific subpopulation whose behavior is actually modified by our interventions.