Causal Inference Under Outcome-Based Sampling with Monotonicity Assumptions

Sung Jae Jun, Sokbae Lee

TL;DR

This paper provides a robust framework for causal inference in "outcome-based" studies (like case-control designs) where rare events make random sampling impractical. By replacing the often-unrealistic assumption of "unconfoundedness" with intuitive "monotonicity" assumptions, the authors show that the standard odds ratio serves as a sharp upper bound for causal relative risk.

What Problem Does This Paper Address?

In an ideal world, causal inference relies on random samples from a population. However, for rare events—such as a specific cancer, entrance into an elite university, or involvement in a criminal gang—a random sample of 1,000 people might yield zero "cases." To solve this, researchers use outcome-based sampling, such as case-control studies, where they specifically recruit people who experienced the outcome (cases) and compare them to those who didn't (controls).

The challenge is that standard causal tools (like the "strong ignorability" assumption) don't work the same way here as they do in random samples. Often, the parameters we actually care about—Relative Risk (the ratio of probabilities) and Attributable Risk (the difference in probabilities)—cannot be "point-identified" (calculated as a single number). Furthermore, the assumption that there are no unobserved factors influencing both treatment and outcome is frequently violated in real-world observational data.

Methods

The authors move away from trying to find a single "true" effect and instead embrace partial identification. They focus on finding "sharp bounds"—the tightest possible interval that must contain the true causal effect.

Key methodological contributions include:

  • Monotonicity Assumptions: Instead of assuming we've measured every possible confounder, they use Monotone Treatment Response (MTR) (the treatment is potentially beneficial but never hurts) and Monotone Treatment Selection (MTS) (those who choose the treatment are naturally more likely to have a positive outcome anyway).
  • New Bounds: They prove that under these monotonicity assumptions, the standard Odds Ratio (OR)—a common measure of association—is actually the sharp upper bound for the Causal Relative Risk.
  • Aggregation Algorithms: They provide a "retrospective sieve logistic estimator" to average these causal effects over the entire population, even when the true distribution of characteristics in that population isn't directly observed.
  • Software: The authors released an R package, ciccr, to allow other researchers to apply these bounds to their own case-control or case-population data.

Datasets

The paper demonstrates the framework using three distinct empirical examples:

  1. Pakistan University Entrance: Examining the benefit of attending private school on entering a "Very Selective University" (VSU) using survey data from male students in Islamabad and Lahore.
  2. Brazilian Gang Involvement: Analyzing the link between staying in school and joining drug-trafficking gangs in Rio de Janeiro favelas, combining census data with a unique gang member survey.
  3. U.S. Physician Billing: Investigating whether physicians in smaller group practices are more likely to "overbill" Medicare (billing >100 hours/week), using a large dataset of 78,165 physicians.

Key Insights and Ideas

  • Ignorability isn't a silver bullet: In case-control studies, even if you assume "strong ignorability" (no hidden confounders), you still usually can't get a single point estimate for relative risk unless you know the true prevalence of the outcome in the general population.
  • The Odds Ratio as a Guardrail: For decades, researchers have used the odds ratio as a proxy for relative risk. This paper provides a formal causal justification: if you believe the treatment doesn't hurt (MTR) and people self-select into treatment (MTS), the odds ratio is the highest possible causal effect consistent with the data.
  • Design Matters: The authors show that "Case-Population" sampling (comparing cases to a sample of the general population) is mathematically "easier" and more informative for causal inference than traditional "Case-Control" sampling.

Key Results

  • Education (Pakistan): Attending private school increases the chance of entering a prestigious university by a factor of at most 1.26. The authors conclude the effect is modest at best.
  • Crime (Brazil): Not being in school has a massive potential impact; the relative risk of joining a gang is bounded by an odds ratio of 15.01. This suggests that keeping young men in school could significantly discourage criminal activity.
  • Medical Billing (USA): Working in a small group practice (fewer than 6 people) is associated with a higher likelihood of being "flagged" for high billing hours, with an upper bound relative risk of approximately 2.68.

Limitations and Future Work

  • Prevalence Knowledge: Some of the bounds (especially for Attributable Risk) require the researcher to provide a plausible upper bound on the true population prevalence ($p_0$). If $p_0$ is completely unknown and could be anything, the bounds for Attributable Risk become very wide.
  • Binary Constraints: The current framework is strictly for binary treatments (Yes/No) and binary outcomes. Extending this to continuous treatments or multi-valued outcomes remains a direction for future research.
  • Monotonicity Validity: The results rely entirely on the validity of the MTR and MTS assumptions. While often plausible in economics and medicine, they are not universal and must be defended by the researcher in each specific context.

Summary updated

Abstract

We study causal inference under case-control and case-population sampling. Specifically, we focus on the binary-outcome and binary-treatment case, where the parameters of interest are causal relative and attributable risks defined via the potential outcome framework. It is shown that strong ignorability is not always as powerful as it is under random sampling and that certain monotonicity assumptions yield comparable results in terms of sharp identified intervals. Specifically, the usual odds ratio is shown to be a sharp identified upper bound on causal relative risk under the monotone treatment response and monotone treatment selection assumptions. We offer algorithms for inference on the causal parameters that are aggregated over the true population distribution of the covariates. We show the usefulness of our approach by studying three empirical examples: the benefit of attending private school for entering a prestigious university in Pakistan; the relationship between staying in school and getting involved with drug-trafficking gangs in Brazil; and the link between physicians’ hours and size of the group practice in the United States.
BibTeX
@article{jun2023causal,
  author = {Sung Jae Jun and Sokbae Lee},
  title = {{Causal Inference Under Outcome-Based Sampling with Monotonicity Assumptions}},
  journal = {Journal of Business and Economic Statistics},
  year = {2023},
  doi = {10.1080/07350015.2023.2277164},
}

Loading related content...

Figure