Nonparametric Bounds for Evaluating the Clinical Utility of Treatment Rules

Johannes Hruza, Erin E. Gabriel, Arvid Sjölander, Samir Bhatt, Michael C. SachsarXiv 2025 · DOI · arXiv

Abstract

Evaluating the value of new clinical treatment rules based on patient characteristics is important but often complicated by hidden confounding factors in observational studies. Standard methods for estimating the average patient outcome if a new rule were universally adopted typically rely on strong, untestable assumptions about these hidden factors. This paper tackles this challenge by developing nonparametric bounds - a range of plausible values - for the expected outcome under a new rule, even with unobserved confounders present. We propose and investigate two main strategies for derivation of these bounds. We extend these techniques to incorporate Instrumental Variables (IVs), which can help narrow the bounds, and to directly estimate bounds on the difference in expected outcomes between the new rule and an existing clinical guideline. In simulation studies we compare the performance and width of bounds generated by the reduction and conditioning strategies in different scenarios. The methods are illustrated with a real-data example about prevention of peanut allergy in children. Our bounding frameworks provide robust tools for assessing the potential impact of new clinical treatment rules when unmeasured confounding is a concern.

Introduction

Individualized Treatment Rules (ITRs) are decision rules that recommend medical treatments based on individual patient characteristics ($X$). A critical task in clinical research is evaluating the "clinical utility" of a proposed ITR—defined as the expected outcome if the rule were implemented across a population. While randomized controlled trials (RCTs) are the gold standard for such evaluations, researchers often must rely on observational data.

The primary obstacle in observational studies is unmeasured confounding. Standard estimation techniques often assume "no unmeasured confounding," implying that all factors influencing both treatment assignment ($A$) and outcome ($Y$) are observed. However, in practice, latent variables ($U$) such as a clinician's intuitive assessment or a patient's unrecorded lifestyle factors often bias these estimates. This paper proposes a robust alternative: using nonparametric bounds to provide a range of possible outcomes that are consistent with the observed data, without relying on untestable assumptions.

Causal Models and Identification

The authors represent the relationships between observed and unobserved variables using Directed Acyclic Graphs (DAGs). These models account for patient covariates ($X$), treatment ($A$), outcome ($Y$), and unmeasured confounders ($U$). Furthermore, they consider scenarios where additional functions of covariates ($g(X)$ or $f(X)$) are observed, and where an Instrumental Variable ($Z$) is available.

Causal structures for evaluating treatment rules under unmeasured confounding, including scenarios with and without Instrumental Variables (IVs).

To handle the complexity of these models, the paper introduces two strategies for partial identification:

  1. Reduction Strategy: This approach simplifies the causal graph by marginalizing over the covariates $X$, treating them as part of the unmeasured confounding. This allows the use of standard linear programming (LP) techniques to derive bounds, though it may sacrifice precision by ignoring the specific information contained in $X$.
  2. Conditioning Strategy: This approach calculates sharp bounds within each stratum of $X$ and then averages these bounds over the population distribution of $X$. This strategy is generally more computationally efficient and produces tighter bounds.

The Role of Instrumental Variables

Instrumental Variables (IVs) are factors that influence treatment assignment but have no direct effect on the outcome except through the treatment. In the context of bounding, IVs are highly valuable because they impose additional constraints on the possible joint distributions of $(U, A, Y)$, which can significantly narrow the resulting bounds. The paper demonstrates how to integrate IVs into both the Reduction and Conditioning frameworks to improve the precision of clinical utility estimates.

Empirical Evaluation

Simulation Studies

The authors conducted simulations to compare the Reduction and Conditioning strategies. A key finding was that the Conditioning strategy consistently yielded bounds that were at least as narrow as, and often significantly tighter than, those from the Reduction strategy. This confirms that preserving the granular information in patient covariates is essential for accurate utility assessment. Moreover, the Conditioning approach proved more scalable for high-dimensional data, where direct LP on the full model becomes computationally prohibitive.

Real-World Application: The LEAP Study

The methods were applied to data from the "Learning Early About Peanut Allergy" (LEAP) study, which investigated whether early introduction of peanuts reduces allergy risk in high-risk infants. Using the study's randomization as an IV, the authors evaluated different treatment rules.

  • For a simple rule ($f_1$) recommending peanut exposure for all children, the bounds were entirely negative (approx. -15% to -11%), providing strong evidence that the rule reduces allergy risk compared to avoidance.
  • For a more complex, dose-dependent rule ($f_2$), the bounds were wider and included zero, indicating that the available data could not definitively prove its superiority over the standard of care under all confounding scenarios.

Conclusion

This research provides a rigorous framework for evaluating clinical treatment rules when unmeasured confounding cannot be ruled out. By moving from point estimates to nonparametric bounds, clinicians and policymakers can obtain a more "honest" assessment of uncertainty. The Conditioning strategy, in particular, offers a computationally feasible and precise method for leveraging patient data and instrumental variables to inform better medical decision-making.

Summary updated

Abstract

Evaluating the value of new clinical treatment rules based on patient characteristics is important but often complicated by hidden confounding factors in observational studies. Standard methods for estimating the average patient outcome if a new rule were universally adopted typically rely on strong, untestable assumptions about these hidden factors. This paper tackles this challenge by developing nonparametric bounds - a range of plausible values - for the expected outcome under a new rule, even with unobserved confounders present. We propose and investigate two main strategies for derivation of these bounds. We extend these techniques to incorporate Instrumental Variables (IVs), which can help narrow the bounds, and to directly estimate bounds on the difference in expected outcomes between the new rule and an existing clinical guideline. In simulation studies we compare the performance and width of bounds generated by the reduction and conditioning strategies in different scenarios. The methods are illustrated with a real-data example about prevention of peanut allergy in children. Our bounding frameworks provide robust tools for assessing the potential impact of new clinical treatment rules when unmeasured confounding is a concern.
BibTeX
@misc{hruza2025nonparametric,
  author = {Johannes Hruza and Erin E. Gabriel and Arvid Sjölander and Samir Bhatt and Michael C. Sachs},
  title = {{Nonparametric Bounds for Evaluating the Clinical Utility of Treatment Rules}},
  year = {2025},
  eprint = {2508.14653},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.14653},
  howpublished = {arXiv},
}
Figure