Recovery and inference of causal effects with sequential adjustment for confounding and attrition

Johan de Aguas, Johan Pensar, Tomás Varnet Pérez, Guido Biele

TL;DR

This paper introduces the Sequential Adjustment Criteria (SAC), a new graphical framework for recovering causal effects when data is plagued by both confounding and informative dropout (attrition). The authors also debut the Targeted Sequential Regression (TSR) estimator, a "multiply-robust" tool that provides reliable causal estimates even when several parts of the statistical model are incorrectly specified.

What Problem Does This Paper Address?

In the world of applied causal inference, two "boss-level" challenges often break our models: confounding bias (unaccounted factors influencing both treatment and outcome) and selection bias (specifically "attrition," where participants drop out of a study in a non-random way).

Standard tools like the "backdoor criterion" help with confounding, and "Inverse Probability Weighting" (IPW) helps with missing data. However, real-world scenarios are often messier. For example, in medical studies, a treatment might cause a side effect (a "post-exposure" variable) that then causes a patient to drop out. Standard rules often label these post-exposure variables as "forbidden" for adjustment because they can introduce new biases (like collider bias).

The authors identify a gap: we lack a constructive, graphical way to identify causal effects when we must navigate these "forbidden" nodes to account for attrition.

Methods

The paper makes two major contributions:

  1. Sequential Adjustment Criteria (SAC): This is a theoretical breakthrough using Directed Acyclic Graphs (DAGs). It defines how to pick two sets of variables—an "outer" set ($W$) and an "inner" set ($Z$)—to block confounding and account for attrition through a sequence of regressions. Crucially, it proves that you can adjust for certain descendants of the treatment if you follow this sequential logic.
  2. Targeted Sequential Regression (TSR) Estimator: Building on the Targeted Minimum Loss-based Estimation (TMLE) framework, the authors developed a new estimator.
  3. How it works: It uses two sequential regression models and "targets" them to minimize bias for the specific causal effect you care about (the Average Treatment Effect).
  4. Multiple Robustness: This is the "killer feature." Most modern estimators are "doubly robust" (they work if either the outcome model or the treatment model is right). TSR is multiply-robust: it remains consistent if any one of three different combinations of its sub-models is correctly specified.

Datasets

The authors validated their approach using both synthetic and real-world data:

  • Simulated Data: Two distinct "m-graph" (missingness graph) setups were used to test the estimator against "naïve" regressions and Double Inverse Probability Weighting (DIPW) under various levels of model misspecification.
  • ADHD Medication Study: The authors applied TSR to a massive observational dataset of 9,352 Norwegian schoolchildren diagnosed with ADHD. They linked data from several national registries (Prescription Database, National Patient Registry, etc.) to see how stimulant medication affected national test scores in grade 8, accounting for the fact that some students are exempted from or skip these tests.

Key Insights and Ideas

  • The "Forbidden" is Functional: The traditional "don't adjust for post-treatment variables" rule is too blunt. SAC provides the mathematical "permission" to use these variables to fix selection bias, provided they are handled in the correct sequential order.
  • Triple-Layer Protection: The multiple robustness of the TSR estimator makes it a "tank" for observational research. In their simulations, when both the selection model and the meta-regression were wrong, TSR still pulled through because the exposure model and mean-imputation were correct.
  • Smoothing Misspecification: One "aha moment" in the results shows that TSR can actually "smooth out" errors from a bad initial model by using well-specified propensity scores in its targeting step.

Key Results

  • Simulation Performance: In complex scenarios where standard methods (DIPW or simple sequential regression) failed due to model misspecification, TSR maintained low bias and high "coverage" (meaning its confidence intervals actually contained the truth).
  • ADHD Application: The study found that ADHD medication has a positive but small impact on academic achievement.
  • Numeracy: An increase of 1.06 raw score points (Cohen’s $d \approx 0.10$).
  • Reading: An increase of 0.66 raw score points (Cohen’s $d \approx 0.08$).
  • Methodological Edge: Interestingly, the TSR estimates were slightly higher than those from simpler methods. This suggests that by better handling the "informative missingness" (the kids who didn't take the test), TSR captured a more accurate picture of the medication's benefit.

Limitations and Future Work

  • Finite Sample Variance: The authors note that while TSR is theoretically efficient, in smaller datasets, the variance estimators can sometimes underestimate the true uncertainty.
  • Finding the "Best" Set: While SAC tells you if a set of variables is valid, it doesn't yet tell you which valid set is optimal (i.e., which one will give you the narrowest confidence intervals).
  • Computational Complexity: Implementing TSR/TMLE is significantly more complex than a standard OLS regression, requiring specialized software and more "tuning" by the researcher.

The authors suggest that future work should focus on automating the search for these "optimal" adjustment sets and extending the framework to handle situations where multiple different variables are missing simultaneously.

Summary updated

Abstract

Abstract Confounding bias and selection bias bring two significant challenges to the validity of conclusions drawn from applied causal inference. The latter can stem from informative missingness, such as in cases of attrition. We introduce the sequential adjustment criteria , which extend available graphical conditions for recovering causal effects from confounding and attrition using sequential regressions, allowing for the inclusion of postexposure and forbidden variables in the adjustment sets. We propose an estimator for the recovered average treatment effect based on targeted minimum-loss estimation , which exhibits multiple robustness under certain technical conditions. This approach ensures consistency even in scenarios where the double inverse probability weighting and the naïve plug-in sequential regressions approaches fall short. Through a simulation study, we assess the performance of the proposed estimator against alternative methods across different graph setups and model specification scenarios. As a motivating application, we examine the effect of pharmacological treatment for attention-deficit/hyperactivity disorder upon the scores obtained by diagnosed Norwegian schoolchildren in national tests using observational data ( <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>n</m:mi> <m:mo>=</m:mo> <m:mstyle> <m:mspace width="0.1em"/> <m:mtext>9,352</m:mtext> <m:mspace width="0.1em"/> </m:mstyle> </m:math> n=\hspace{0.1em}\text{9,352}\hspace{0.1em} ). Our findings align with the accumulated clinical evidence, affirming a positive but small impact of medication on academic achievement.
BibTeX
@article{aguas2025recovery,
  author = {Johan de Aguas and Johan Pensar and Tomás Varnet Pérez and Guido Biele},
  title = {{Recovery and inference of causal effects with sequential adjustment for confounding and attrition}},
  journal = {Journal of Causal Inference},
  year = {2025},
  doi = {10.1515/jci-2024-0009},
}
Figure