Commentary on "Principal Stratification -- a Goal or a Tool?" by Judea Pearl

Peter B. Gilbert, Michael G. Hudgens, Julian WolfsonThe International Journal of Biostatistics 2011 · DOI · OpenAlex

Abstract

This commentary takes up Pearl's welcome challenge to clearly articulate the scientific value of principal stratification estimands that we and colleagues have investigated, in the area of randomized placebo-controlled preventive vaccine efficacy trials, especially trials of HIV vaccines. After briefly arguing that certain principal stratification estimands for studying vaccine effects on post-infection outcomes are of genuine scientific interest, the bulk of our commentary argues that the "causal effect predictiveness" (CEP) principal stratification estimand for evaluating immune biomarkers as surrogate endpoints is not of ultimate scientific interest, because it evaluates surrogacy restricted to the setting of a particular vaccine efficacy trial, but is nevertheless useful for guiding the selection of primary immune biomarker endpoints in Phase I/II vaccine trials and for facilitating assessment of transportability/bridging surrogacy.

Introduction

In the field of causal inference, the utility of Principal Stratification (PS) has been a subject of significant debate. Judea Pearl recently critiqued the framework, questioning whether PS estimands represent genuine scientific goals or merely mathematical tools. This commentary, published in The International Journal of Biostatistics, defends the PS framework by demonstrating its critical role in solving complex problems within HIV vaccine research. Specifically, the authors argue that PS is essential for evaluating post-infection outcomes and identifying surrogate biomarkers—two pillars of modern vaccine development.

Addressing the "Truncation by Death" Problem

A primary challenge in randomized vaccine trials is the "truncation by death" or selection bias problem. When evaluating a vaccine's effect on post-infection outcomes (e.g., viral load), a simple comparison between infected vaccinees and infected placebo recipients is flawed. The vaccine may prevent infection in a specific subgroup (e.g., those with stronger innate immunity), meaning the infected vaccine group is not comparable to the infected placebo group.

PS addresses this by defining "principal strata"—subgroups based on potential outcomes. By focusing on the stratum of individuals who would have been infected regardless of treatment assignment, researchers can isolate the vaccine's true biological effect on the pathogen without the confounding effects of selection bias. This provides a scientifically meaningful measure of how a vaccine modifies disease progression in those it fails to protect from infection.

Evaluating Surrogate Endpoints via CEP

The authors introduce the Causal Effect Predictiveness (CEP) surface as a key PS-based metric. In vaccine development, identifying "surrogates"—biomarkers measured early in trials that predict clinical efficacy—is vital for accelerating the transition from Phase I/II to Phase III trials. The CEP estimand quantifies how well a vaccine-induced change in a biomarker predicts the actual clinical outcome (infection).

While Pearl argues that such trial-specific estimands may lack broad scientific interest, the authors contend they are indispensable for guiding the selection of primary endpoints and facilitating "bridging" or "transportability." This allows researchers to predict the efficacy of a new vaccine or the same vaccine in a different population based on observed biomarker distributions, as seen in the analysis of the Thai trial (RV144).

Methodological Advantages over Alternatives

The commentary contrasts PS with the "Natural Direct/Indirect Effects" (NDE) framework. NDE often requires hypothetical interventions—such as setting a placebo recipient's immune response to a level only achievable via vaccination—which the authors describe as "dubious" or "inconceivable" in the context of vaccine science. PS avoids these impossible manipulations by relying on observed strata. Furthermore, in trials involving pathogen-naive participants (common in HIV research), the baseline immune response is known to be zero, which simplifies the mathematical identification of PS estimands and makes the approach highly practical.

Conclusion

While acknowledging that PS estimands require additional identifiability assumptions or augmented study designs, the authors conclude that the framework passes the ultimate scientific litmus test: it provides actionable information that advances vaccine development. By formalizing the relationship between biomarkers and clinical protection, PS serves as a bridge between early-stage laboratory results and late-stage clinical success, particularly in the ongoing effort to develop an effective HIV vaccine.

Summary updated

Abstract

This commentary takes up Pearl's welcome challenge to clearly articulate the scientific value of principal stratification estimands that we and colleagues have investigated, in the area of randomized placebo-controlled preventive vaccine efficacy trials, especially trials of HIV vaccines. After briefly arguing that certain principal stratification estimands for studying vaccine effects on post-infection outcomes are of genuine scientific interest, the bulk of our commentary argues that the "causal effect predictiveness" (CEP) principal stratification estimand for evaluating immune biomarkers as surrogate endpoints is not of ultimate scientific interest, because it evaluates surrogacy restricted to the setting of a particular vaccine efficacy trial, but is nevertheless useful for guiding the selection of primary immune biomarker endpoints in Phase I/II vaccine trials and for facilitating assessment of transportability/bridging surrogacy.
BibTeX
@article{gilbert2011commentary,
  author = {Peter B. Gilbert and Michael G. Hudgens and Julian Wolfson},
  title = {{Commentary on "Principal Stratification -- a Goal or a Tool?" by Judea Pearl}},
  journal = {The International Journal of Biostatistics},
  year = {2011},
  doi = {10.2202/1557-4679.1341},
}
Figure