Abstract
Randomized controlled trials (RCTs) serve as the gold standard for causal inference, yet their findings often struggle with external validity due to the "research-to-practice gap." This paper introduces ROOT (Rashomon Set of Optimal Trees), a novel framework designed to identify underrepresented subgroups within trial data. By utilizing interpretable decision trees to characterize the Rashomon set of near-optimal models, ROOT allows researchers to refine target populations and improve the precision of treatment effect estimates. We demonstrate the framework's utility through synthetic experiments and a real-world case study on opioid use disorder treatment, showing that ROOT provides a principled, transparent method for navigating the tradeoff between generalizability and statistical precision.
Introduction
While randomized controlled trials (RCTs) provide high internal validity, the populations enrolled in these trials often differ significantly from the general patient populations they are intended to inform. This discrepancy—driven by strict eligibility criteria and volunteer bias—creates a significant hurdle for clinicians and policymakers. Existing methods to bridge this gap typically rely on "selection scores" to weight trial results toward a target population. However, these scores are often generated by "black box" models that lack interpretability, making it difficult for practitioners to understand exactly which subgroups are underrepresented or why certain predictions are less reliable.
This paper addresses the need for a principled, interpretable approach to characterizing underrepresented populations. We propose the ROOT (Rashomon Set of Optimal Trees) framework, which identifies subgroups where the trial data provides insufficient evidence, allowing for a more credible and precise estimation of treatment effects in the remaining population.
The ROOT Framework
The authors propose a two-stage approach to causal generalization:
- Design Stage: Identify a "sufficiently represented" subpopulation using the ROOT algorithm.
- Analysis Stage: Estimate the treatment effect specifically for that refined, well-supported population.
The Rashomon Set of Optimal Trees
The core innovation, ROOT, treats the identification of underrepresented groups as a functional optimization problem. Rather than seeking a single "best" model, ROOT identifies a Rashomon Set—a collection of different decision trees that are all nearly optimal at minimizing the variance of the treatment effect estimate.
By restricting the model space to sparse decision trees, ROOT generates "if-then" rules that are inherently interpretable. For example, instead of a single probability score, ROOT might identify that "Hispanic women with a history of amphetamine use" are a subgroup for whom the trial data is too sparse to provide a reliable estimate. The algorithm employs a stochastic search (explore-exploit) to navigate the massive space of possible tree structures, ensuring it finds robust representations without getting trapped in local minima.
Empirical Evaluation
Synthetic Data-Generating Processes
The framework was evaluated against four distinct synthetic data-generating processes (DGPs): Community, Box, Clone, and High-dimensional. These tests were designed to challenge the algorithm's ability to handle non-linearity and high-dimensional covariates. In these experiments, ROOT consistently outperformed traditional selection-score trimming and simple randomized trees, producing estimates with significantly lower standard errors.
Case Study: Opioid Use Disorder (MOUD)
To demonstrate real-world utility, the authors applied ROOT to the START Trial (N=1,271), which compared methadone and buprenorphine for opioid use disorder (MOUD). The target population was defined using the TEDS-A Dataset (N=740,015), representing individuals seeking treatment for opioid use across the United States.
ROOT identified that Black individuals with a history of amphetamine use but no cannabis use were significantly underrepresented in the START trial. By excluding these specific, poorly-supported subgroups (representing less than 1% of the target population), the researchers were able to provide a more precise and credible treatment effect estimate for the remaining 99.3% of the real-world population.
Key Insights
The Precision-Generalizability Tradeoff
ROOT highlights a fundamental tension in clinical research: the desire to generalize findings to everyone versus the need for statistical certainty. By pruning "high-variance" units—those for whom the trial provides little reliable information—ROOT allows researchers to navigate this tradeoff transparently.
Focus on Effect Modifiers
Unlike standard methods that prune based solely on rarity, ROOT prioritizes the removal of individuals who are both rare in the trial and likely to be effect modifiers (those whose treatment response differs significantly from the average). This ensures that the refinement of the target population is driven by the goal of accurate causal inference rather than simple demographic matching.
Limitations and Future Directions
While ROOT offers a significant step forward, it faces challenges in extremely high-dimensional settings where the number of effect modifiers is large; in such cases, the "interpretable" tree can become overly complex. Additionally, the framework relies on the assumption of "S-admissibility"—that all variables differentiating the trial group from the general population have been measured. Future work is also needed to determine how statistical inference should account for the uncertainty introduced by the data-adaptive process of defining the target population.
Conclusion
ROOT provides a principled and interpretable bridge between clinical trials and real-world practice. By characterizing the "Rashomon Set" of underrepresented populations, it empowers researchers to make more informed decisions about the limits of their data, ultimately leading to more reliable and actionable evidence for patient care.
Summary updated