Probabilities of Potential Outcome Types in Experimental Studies: Identification and Estimation Based on Proxy Covariate Information

Ryusei Shingaki, Manabu KurokiAAAI 2023 · DOI

TL;DR

This paper introduces a novel method to identify and estimate "potential outcome types" in randomized experiments, crucial for understanding individual treatment effects. By leveraging proxy covariate information, it overcomes limitations of previous methods that relied on strong assumptions, enabling more robust causal inference.

What Problem Does This Paper Address?

Causal inference aims to understand the effect of a treatment on an outcome. A key concept is "potential outcomes," which represent what would happen to an individual under different treatment scenarios (e.g., receiving a vaccine vs. a placebo). However, we can only observe one of these potential outcomes for any given individual. To fully understand treatment effects, especially at an individual level, researchers are interested in the probabilities of different "potential outcome types." These types categorize individuals based on how they would respond to both the treatment and the control (e.g., "doomed" if the outcome occurs regardless of treatment, "preventive" if the outcome is avoided only by receiving the treatment).

Previous methods for estimating these probabilities often required strong assumptions like monotonicity (e.g., treatment never makes things worse) or relied on extensive covariate information. In randomized experiments, while we can estimate average treatment effects, identifying these individual-level potential outcome type probabilities without strong assumptions has been challenging. This paper addresses this gap by proposing a new identification condition that uses proxy covariate information, relaxing previous stringent requirements.

Methods

The core contribution of this paper is a novel identification condition for the probabilities of potential outcome types in randomized experiments. This condition leverages proxy covariate information, specifically variables (Z and W) that are related to the unobserved confounders (U) but are not directly affected by the treatment (X) or outcome (Y).

The authors represent the causal relationships using directed acyclic graphs (DAGs) and structural causal models. They then derive a set of equations involving conditional probabilities of observed variables. The identification condition hinges on the invertibility of certain matrices derived from these probabilities, denoted as $Q_{xyw}$.

Crucially, the paper proposes new plug-in estimators for these probabilities. Unlike standard methods like maximum likelihood estimation, which are not directly applicable due to the nature of the problem (reducing to singular models), these plug-in estimators are designed to work with the derived identification conditions. The asymptotic normality of these proposed estimators is also demonstrated.

Datasets

Not discussed in the paper. The paper focuses on theoretical development and a simulated numerical experiment.

Key Insights and Ideas

  • Relaxing Monotonicity: The proposed method does not require the monotonicity assumption, which is a significant advantage as monotonicity is often violated in real-world scenarios.
  • Leveraging Proxy Covariates: The identification of potential outcome type probabilities is achieved by using proxy covariates (Z and W) that capture information about unobserved confounders (U). This is a key innovation for situations where full covariate information is unavailable.
  • Bridging Identification and Estimation: The paper clearly separates the problem of identifying whether these probabilities can be determined from data and estimating them once identified. It shows that identification is possible under certain conditions, and then tackles the estimation challenge.
  • Singular Models and Novel Estimators: The estimation problem reduces to that of "singular models," which are not amenable to standard statistical techniques. The development of new plug-in estimators is a direct response to this challenge.
  • Generalizability: While the paper focuses on randomized experiments, the authors suggest that their results can be extended to observational studies, broadening the applicability of their work.

Key Results

The paper presents a Theorem 1 that provides the conditions under which the probabilities of the four potential outcome types ($p(u_1), p(u_2), p(u_3), p(u_4)$) are identifiable. These conditions involve:

  1. Availability of positive probabilities $p(y, z, w|x)$.
  2. Independence assumptions between covariates and treatment/confounders.
  3. Invertibility of specific matrices ($Q_{xyw}$) derived from observed data.

Theorem 2 establishes the asymptotic normality of the proposed plug-in estimators for these probabilities.

A numerical experiment demonstrates the performance of the proposed estimation method. For simulated data, the sample means of the estimated probabilities ($p(u_2)$ and $p(u_2) - p(u_3)$) were close to their true values, and the sample standard errors decreased with larger sample sizes, suggesting the estimators are consistent. Box plots also indicated reasonable performance, though some outliers were observed, particularly when the identification conditions were borderline.

Limitations and Future Work

  • Data Sparseness: The authors acknowledge that extending their results to multivalued or continuous variables might lead to data sparseness, making it difficult to obtain reliable statistics. This is left for future work.
  • Variance Estimation: The paper focuses on point estimation and does not delve into variance estimation for the proposed estimators. This is also identified as a direction for future research.
  • Matrix Inversion Instability: The numerical experiment suggests that the proposed estimation methods can be unstable in practice due to matrix inversion, especially when the association between unobserved confounders and proxy covariates is weak or when sample sizes are small.
  • Assumptions on Proxy Covariates: The identification relies on the existence of proxy covariates (Z and W) that satisfy specific independence conditions. Finding such variables in practice can be a challenge.
  • Dichotomous Variables Focus: The primary focus is on dichotomous treatment and outcome variables, though the authors note that extensions to multivalued variables are possible under certain assumptions.

Future work includes addressing variance estimation, exploring extensions to multivalued and continuous variables, and investigating the practical challenges of finding suitable proxy covariates.

Summary updated

BibTeX
@inproceedings{shingaki2023probabilities,
  author = {Ryusei Shingaki and Manabu Kuroki},
  title = {{Probabilities of Potential Outcome Types in Experimental Studies: Identification and Estimation Based on Proxy Covariate Information}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i10.26448},
}
Figure