Methods for Estimating Complier Average Causal Effects for Cost-Effectiveness Analysis

Karla Díaz-Ordaz, A Franchini, Richard Grieve

TL;DR

This paper tackles the challenge of estimating the true cost-effectiveness of a treatment in randomized controlled trials (RCTs) where participants don't always adhere to their assigned treatment. They propose and compare novel Bayesian and frequentist methods that correctly account for non-compliance and the correlation between costs and health outcomes, leading to more reliable estimates than traditional approaches.

What Problem Does This Paper Address?

In clinical trials, researchers often want to understand the cost-effectiveness of a new treatment. A standard approach is to compare the costs and health outcomes of participants assigned to the new treatment versus a control group. However, a significant problem arises when participants don't follow their assigned treatment (non-compliance). This non-compliance can bias the results, making it difficult to determine the true causal effect of receiving the treatment.

Traditional methods like intention-to-treat (ITT) analysis, which looks at the effect of assignment to treatment, don't fully address this. Per-protocol analyses, which exclude non-compliant individuals, can also lead to biased estimates. Furthermore, cost-effectiveness analyses (CEAs) often involve multiple outcomes (like cost and quality-adjusted life-years or QALYs) that are correlated. Accurately estimating the causal effect of treatment receipt on these correlated outcomes, while accounting for non-compliance, is a complex methodological challenge that this paper aims to solve.

Methods

The authors propose and evaluate three main methods for estimating Complier Average Causal Effects (CACE) in cost-effectiveness analyses with bivariate outcomes and non-compliance:

  • Three-Stage Least Squares (3SLS): This is a frequentist approach that extends traditional two-stage least squares (2SLS) to handle systems of simultaneous equations with endogenous regressors. It jointly estimates the effect of treatment assignment on treatment received, and then the effect of received treatment on both cost and QALY outcomes, while accounting for the correlation between these outcomes. This is achieved by first estimating the treatment received, then estimating the outcomes based on the predicted treatment, and finally using the residuals to estimate the covariance matrix of the errors.
  • Naive Bayesian (uBN and uBGN): These are Bayesian approaches that extend existing methods for Mendelian randomization. They model the treatment received and the outcomes separately, but crucially, they ignore the correlation between the outcomes and the treatment received. The uBN model assumes normal distributions for both outcomes, while uBGN uses a gamma distribution for costs and a normal distribution for QALYs. While flexible, their "unadjusted" nature is a key limitation.
  • Bayesian Full Likelihood (BFL): This is a more sophisticated Bayesian approach that jointly models the treatment received and the bivariate outcomes. By modeling these components simultaneously, it correctly accounts for the correlation between costs, QALYs, and treatment received, providing a more robust estimation of the causal effect. This is described as an extension of the "unrestricted reduced form" in econometrics.

Datasets

The primary dataset used for illustration and application is the REFLUX study. This was a multicenter randomized controlled trial and cost-effectiveness analysis comparing laparoscopic surgery with medical management for patients with gastro-oesophageal reflux disease. The study collected data on individual resource use (costs) and health-related quality of life (HRQOL) measured by EQ5D over a 5-year follow-up period.

The paper also uses a simulation study to rigorously compare the performance of the proposed methods across various scenarios. These simulations vary:

  • Levels of non-compliance (30% and 70%)
  • Distributions of cost data (normal, gamma, inverse Gaussian)
  • Levels of correlation between costs and QALYs (±0.4, ±0.8)
  • Sample sizes (n=100 and n=1000)

Key Insights and Ideas

  • Ignoring correlations is problematic: The most striking insight is that failing to correctly model the correlation between costs, QALYs, and treatment received leads to biased estimates and poor confidence interval coverage. This is particularly true for the "unadjusted" Bayesian methods (uBN, uBGN).
  • CACE is crucial for policy: When compliance in an RCT differs from routine practice, the CACE provides a more relevant estimand for policymakers than ITT, as it reflects the effect of receiving the intervention.
  • Bayesian Full Likelihood (BFL) and 3SLS are robust: Both the BFL and 3SLS methods demonstrated good performance in simulations, providing unbiased estimates and appropriate confidence interval coverage, even under challenging conditions like high non-compliance and non-normal cost distributions.
  • Model misspecification has consequences: The uBGN model, when misspecified regarding the cost distribution, showed significant bias, highlighting the importance of choosing appropriate distributional assumptions.
  • Sample size matters: While BFL performed well with larger sample sizes, its performance was less optimal with smaller samples, leading to wider confidence intervals.

Key Results

  • Simulation Study:
    • The three-stage least squares (3SLS) and Bayesian Full Likelihood (BFL) methods consistently provided low bias and good confidence interval coverage across various scenarios.
  • The "unadjusted" Bayesian methods (uBN and uBGN) showed substantial biases and poor coverage, especially with higher levels of non-compliance and non-normal cost distributions. For instance, uBGN exhibited up to 10% bias for incremental cost and 20-40% bias for INB in certain scenarios.
  • The two-stage least squares (2SLS) approach, when applied separately to outcomes, resulted in confidence intervals that diverged from nominal levels due to ignoring the correlation between costs and QALYs.
  • REFLUX Study Application:
    • The CACE estimates from 3SLS and BFL suggested that laparoscopic surgery is more cost-effective than medical management for compliers, with incremental costs and QALYs being larger than ITT estimates.
  • The CACE for INB from BFL was £13340 (95% CI: 1406, 26315), indicating cost-effectiveness.
  • The uBGN model produced a significantly smaller INB estimate (£5869) with a CI that included zero, suggesting potential bias due to model misspecification.

Limitations and Future Work

  • Missing Data Handling: The paper acknowledges that missing data can impact results. While they used multiple imputation (MI) and Bayesian approaches to handle missingness, they note that the results can be sensitive to the assumptions made about the missing data mechanism. Future work could explore more advanced methods for handling missing data, especially missingness not at random.
  • Instrument Strength: The study did not explicitly consider scenarios where the instrument (random assignment) is weakly correlated with treatment received. While such scenarios are less common in practice, Bayesian IV methods have shown better performance than 2SLS in these cases.
  • Model Complexity: The paper focused on linear additive treatment effects. Future research could extend these methods to handle non-linear relationships and more complex systems of simultaneous equations.
  • Prior Sensitivity: While the BFL analysis was found to be relatively robust to the choice of weakly informative priors, the paper notes that posterior distributions can be sensitive to prior choices, especially with large amounts of missing data.
  • Time-Varying Non-Compliance: The current methods assume non-compliance occurs within a specific timeframe. Extending these to handle time-varying non-compliance would be a valuable future direction.

Summary updated

Abstract

In randomized controlled trials with treatment non-compliance, instrumental variable approaches are used to estimate complier average causal effects. We extend these approaches to cost-effectiveness analyses, where methods need to recognize the correlation between cost and health outcomes. We propose a Bayesian full likelihood approach, which jointly models the effects of random assignment on treatment received and the outcomes, and a three-stage least squares method, which acknowledges the correlation between the end points and the endogeneity of the treatment received. This investigation is motivated by the REFLUX study, which exemplifies the setting where compliance differs between the randomized controlled trial and routine practice. A simulation is used to compare the methods' performance. We find that failure to model the correlation between the outcomes and treatment received correctly can result in poor confidence interval coverage and biased estimates. By contrast, Bayesian full likelihood and three-stage least squares methods provide unbiased estimates with good coverage.
BibTeX
@article{diazordaz2017methods,
  author = {Karla Díaz-Ordaz and A Franchini and Richard Grieve},
  title = {{Methods for Estimating Complier Average Causal Effects for Cost-Effectiveness Analysis}},
  journal = {Journal of the Royal Statistical Society Series A (Statistics in Society)},
  year = {2017},
  doi = {10.1111/rssa.12294},
}
Figure