Abstract
Non-adherence to assigned treatment is a common issue in cluster randomised trials (CRTs). In these settings, the efficacy estimand—the Local Average Treatment Effect (LATE)—is often of primary interest alongside the standard Intention-to-Treat (ITT) analysis. This paper demonstrates that the LATE can be effectively estimated via two-stage least squares (TSLS) regression using cluster-level summaries of the outcome and treatment received. We propose the use of baseline variables to adjust cluster-level summaries to improve efficiency and emphasize that implementation must account for reduced sample sizes and heteroscedasticity to obtain valid inferences. Through simulations and a re-analysis of the TXT4FLUJAB trial, we show that TSLS using cluster-level summaries provides estimates with negligible bias and nominal coverage, provided appropriate small-sample degrees of freedom corrections and robust standard errors are employed.
Introduction
In a Cluster Randomised Trial (CRT), groups such as schools, hospitals, or geographical areas are randomised to intervention or control arms. While this design is robust for evaluating policy-level changes, it frequently encounters the challenge of non-adherence. Non-adherence can occur at the cluster level (e.g., a clinic fails to implement a protocol) or the individual level (e.g., a patient within a clinic opts out of the treatment).
Standard practice typically relies on Intention-to-Treat (ITT) analysis, which compares outcomes based on the original random assignment. While ITT provides a conservative estimate of the "offer" of treatment, it often dilutes the true clinical efficacy because it includes participants who never received the intervention. To estimate the effect on those who actually complied—the LATE—researchers often turn to Instrumental Variables (IV) methods. However, the inherent dependency of individuals within clusters complicates standard IV approaches. This paper outlines a simplified, cluster-level approach to TSLS that maintains statistical rigor while remaining accessible to practitioners.
Methodology
The Cluster-Level Summary Approach
The proposed method shifts the unit of analysis from the individual to the cluster. This simplification bypasses the need for complex multilevel models that can become unstable in the presence of non-adherence and small cluster counts. The process involves four primary steps:
- Summarization: For each cluster, calculate a summary statistic (typically the mean or proportion) for both the outcome of interest and the actual treatment received.
- Covariate Adjustment (adCL): To account for baseline individual-level differences (e.g., age, gender, or baseline health status), a two-step residual-based approach is used. Individual outcomes are regressed on baseline covariates, and the resulting residuals are then averaged at the cluster level to create "adjusted" summaries.
- Two-Stage Least Squares (TSLS):
- Stage 1: The cluster-level treatment received is regressed on the cluster-level random assignment (the instrument).
- Stage 2: The cluster-level outcome is regressed on the predicted treatment values generated in the first stage.
- Inference Corrections: Because the effective sample size is the number of clusters ($J$) rather than the number of individuals, standard asymptotic assumptions often fail. The methodology requires Huber-White (robust) standard errors to account for heteroscedasticity and Small-Sample Degrees of Freedom (SSDF) corrections (using $t$-distributions with $J - p$ degrees of freedom) to ensure valid confidence intervals.
Empirical Evaluation
Simulation Results
The performance of the cluster-level TSLS approach was evaluated using Monte Carlo simulations across 2,500 replicates. The simulations varied the number of clusters ($J = 10$ to $400$), intra-cluster correlation (ICC), and confounding strength.
The results indicated that TSLS on cluster-level summaries produced negligible bias, even with as few as 10 clusters. However, the simulations highlighted a critical pitfall: using standard normal-based approximations for inference led to significant under-coverage (as low as 84% for a 95% CI). When SSDF corrections and robust standard errors were applied, coverage returned to the nominal 95% level. Furthermore, the adCL adjustment successfully improved precision without introducing bias.
Case Study: The TXT4FLUJAB Trial
The method was applied to the TXT4FLUJAB trial, a large-scale UK CRT involving 116 GP practices and over 100,000 patients. The trial tested whether text message reminders increased flu vaccine uptake.
- ITT Analysis: Showed a modest 2.89% increase in vaccination rates.
- LATE Analysis: Using the cluster-level TSLS approach, the LATE was estimated at approximately 14%.
This discrepancy illustrates the "masking" effect of non-adherence; while the policy of sending texts had a small overall impact, the effect on those who actually received and engaged with the text was substantially higher.
Discussion and Limitations
While the cluster-level summary approach is robust and easy to implement, it is generally less efficient than individual-level multilevel IV models, resulting in wider confidence intervals. The validity of the LATE estimate also rests on two fundamental assumptions: the Exclusion Restriction (the random assignment only affects the outcome through the treatment) and Monotonicity (the absence of "defiers").
Future research should address the impact of missing data at the individual level on cluster-level summaries and explore the stability of these estimates in scenarios with "weak instruments" (where the correlation between assignment and treatment received is very low).
Conclusion
For researchers conducting cluster randomised trials with non-adherence, TSLS on cluster-level summaries offers a practical and consistent method for estimating treatment efficacy. To ensure valid results, practitioners must move beyond software defaults and manually apply small-sample degrees of freedom corrections and robust standard errors, particularly when the number of clusters is small.
Summary updated