Multilevel Intervention Stepped Wedge Designs (MLI-SWDs)

John Sperger, Michael R. Kosorok, Laura Linnan, Shawn M. KneippPrevention 2024 · DOI

TL;DR

This paper introduces the Multilevel Intervention Stepped Wedge Design (MLI-SWD), a novel hybrid experimental design that combines cluster-level Stepped Wedge Designs with individual-level randomization. This approach allows for the precise estimation of both individual and cluster-level intervention effects, their interactions, and the overall intervention impact, addressing a critical gap in the methodology for studying complex multilevel interventions aimed at reducing health inequities.

What Problem Does This Paper Address?

Multilevel Interventions (MLIs) are crucial for addressing health inequities by targeting various social determinants of health. However, designing studies for MLIs, especially those involving interventions at both individual (IL) and cluster (CL) levels, presents significant methodological challenges. A key hurdle is the lack of robust tools for calculating the necessary sample size and power for such complex designs. This limits the development and rigorous evaluation of MLIs. The authors aim to bridge this gap by proposing a new design, the MLI-SWD, and providing the statistical framework and software for its implementation and sample size determination.

Methods

The core contribution is the Multilevel Intervention Stepped Wedge Design (MLI-SWD). This design is a hybrid approach that integrates:

  • Cluster-Level (CL) Randomization via Stepped Wedge Design (SWD): Clusters (e.g., communities, clinics) are randomized to receive the CL intervention at different time points, following a stepped wedge pattern. This allows for gradual rollout and accounts for time-varying effects.
  • Individual-Level (IL) Randomization: Within each cluster, individuals are independently randomized to receive the IL intervention or a control. This is suitable when the risk of interference between individuals receiving the IL intervention within the same cluster is low.

The MLI-SWD accommodates:

  • Cross-sectional and cohort designs: Data can be collected from the same individuals over time (cohort) or from different individuals at each time point (cross-sectional).
  • Complete and incomplete observation patterns: Clusters may be observed at all time points or have unobserved periods.

The statistical framework for sample size calculation is adapted from existing Generalized Estimating Equations (GEE) methods for SWDs, extended to the multilevel context. The authors also developed an R package to facilitate these calculations. They further explore extensions for scenarios where individuals join clusters over time, which is common in interventions targeting dynamic populations like the unemployed.

Datasets

The paper does not present results from a specific dataset for evaluation. Instead, it uses the ongoing NC Works 4 Health (NCW4H) study as a motivating example and a basis for discussing extensions to the MLI-SWD. The NCW4H study aims to develop and evaluate an MLI to reduce negative health consequences of unemployment, involving both individual-level (Diabetes Prevention Program modification) and cluster-level (Supervising for Success program) interventions.

Key Insights and Ideas

  • Synergistic Potential of MLIs: MLIs hold significant promise for tackling complex health inequities by addressing multiple social determinants of health simultaneously.
  • Methodological Bottleneck: The lack of appropriate study designs and sample size calculation tools has been a major barrier to advancing MLI research.
  • Hybrid Design Power: Combining SWD for CL interventions with individual randomization for IL interventions offers a powerful way to disentangle and estimate the effects of each component, their interaction, and the overall intervention.
  • Flexibility of MLI-SWD: The design's adaptability to different data collection (cohort/cross-sectional) and observation patterns (complete/incomplete) makes it broadly applicable.
  • Importance of Individual-Level Randomization: Randomizing IL interventions at the individual level, when feasible, can lead to substantial efficiency gains in estimating IL effects, interaction effects, and overall intervention effects compared to cluster-randomizing the IL intervention.
  • Addressing Dynamic Populations: The extension to accommodate individuals joining clusters over time is crucial for interventions targeting populations with fluid membership, such as the unemployed.

Key Results

The paper's primary contribution is the introduction and methodological framework for the MLI-SWD. While not presenting empirical results from a completed study, it provides:

  • A detailed description of the MLI-SWD, including its rationale and suitability for various MLI contexts.
  • An adaptation of GEE-based sample size calculation methods for this multilevel setting.
  • An R package (swtgeepower) for performing power and sample size calculations for continuous, binary, and count outcomes with various link functions.
  • A detailed example of how to parameterize the MLI-SWD and conduct a sample size calculation for a hypothetical trial, demonstrating its practical application.
  • Discussion of extensions for incomplete designs and scenarios where individuals join clusters over time.

Limitations and Future Work

  • Assumptions for Open Cohorts: When individuals join clusters over time (open cohort designs), additional causal assumptions are required regarding the conditional independence of treatment assignment given past history, which may not always hold and could necessitate alternative analysis methods like g-computation.
  • Spillover Effects: The MLI-SWD is vulnerable to bias from spillover effects at the individual level, a concern not present when IL interventions are cluster-randomized. Researchers must carefully consider the risk of contamination.
  • Time-Varying Treatment Effects: The current sample size calculation framework primarily considers linear transformations of time-varying treatments. Future work could expand this to include functional summaries like the Area Under the Curve (AUC).
  • Noncompliance: The paper acknowledges that noncompliance, a common issue in cluster-randomized trials, was not addressed and is an area for future investigation.
  • CL Parameter Estimation Efficiency: Incorporating cluster-total estimators could potentially improve the efficiency of CL parameter estimation.
  • Broader Applicability: Expanding the design to accommodate an arbitrary number of levels and interventions would increase its applicability.
  • Sensitivity Analyses: The authors recommend performing sensitivity analyses with unequal cluster sizes and different correlation values, as these can significantly impact results.

Summary updated

Abstract

Multilevel interventions (MLIs) hold promise for reducing health inequities by intervening at multiple types of social determinants of health consistent with the socioecological model of health. In spite of their potential, methodological challenges related to study design compounded by a lack of tools for sample size calculation inhibit their development. We help address this gap by proposing the Multilevel Intervention Stepped Wedge Design (MLI-SWD), a hybrid experimental design which combines cluster-level (CL) randomization using a Stepped Wedge design (SWD) with independent individual-level (IL) randomization. The MLI-SWD is suitable for MLIs where the IL intervention has a low risk of interference between individuals in the same cluster, and it enables estimation of the component IL and CL treatment effects, their interaction, and the combined intervention effect. The MLI-SWD accommodates cross-sectional and cohort designs as well as both incomplete (clusters are not observed in every study period) and complete observation patterns. We adapt recent work using generalized estimating equations for SWD sample size calculation to the multilevel setting and provide an R package for power and sample size calculation. Furthermore, motivated by our experiences with the ongoing NC Works 4 Health study, we consider how to apply the MLI-SWD when individuals join clusters over the course of the study. This situation arises when unemployment MLIs include IL interventions that are delivered while the individual is unemployed. This extension requires carefully considering whether the study interventions will satisfy additional causal assumptions but could permit randomization in new settings.
BibTeX
@inproceedings{sperger2024multilevel,
  author = {John Sperger and Michael R. Kosorok and Laura Linnan and Shawn M. Kneipp},
  title = {{Multilevel Intervention Stepped Wedge Designs (MLI-SWDs)}},
  booktitle = {Prevention},
  year = {2024},
  doi = {10.1007/s11121-024-01657-y},
}
Figure