Combining Cox Regressions Across a Heterogeneous Distributed Research\n Network Facing Small and Zero Counts

Martijn J. Schuemie, Yong Chen, David Madigan, Marc A. SuchardarXiv 2021 · DOI · arXiv

Abstract

Studies of the effects of medical interventions increasingly take place in distributed research settings using data from multiple clinical data sources including electronic health records and administrative claims. In such settings, privacy concerns typically prohibit sharing of individual patient data, and instead, cross-network analyses can only utilize summary statistics from the individual databases such as hazard ratios and standard errors. In the specific but very common context of the Cox proportional hazards model, we show that combining such per site summary statistics into a single network-wide estimate using standard meta-analysis methods leads to substantial bias when outcome counts are small. This bias derives primarily from the normal approximations of the per site likelihood that the methods utilized. Here we propose and evaluate methods that eschew normal approximations in favor of three more flexible approximations: a skew-normal, a one-dimensional grid, and a custom parametric function that mimics the behavior of the Cox likelihood function. In extensive simulation studies, we demonstrate how these approximations impact bias in the context of both fixed-effects and (Bayesian) random-effects models. We then apply these approaches to three real-world studies of the comparative safety of antidepressants, each using data from four observational health care databases.

Introduction

Distributed Research Networks (DRNs) have become essential for large-scale medical studies, allowing researchers to analyze data from diverse sources like electronic health records and insurance claims without compromising patient privacy. In these settings, individual sites typically share summary statistics rather than raw patient-level data. For time-to-event analysis, the Cox proportional hazards model is the standard. However, when outcomes are rare or group sizes are highly unbalanced, the traditional "normal approximation" used to share results breaks down, leading to significant statistical bias.

The Problem of Rare Events and Zero Counts

Standard meta-analysis methods rely on the assumption that the likelihood function of a treatment effect follows a normal distribution (a bell curve). This assumption holds well with large sample sizes and frequent events. However, in studies of rare adverse drug effects, many sites may record very few events, or even zero. In such cases, the likelihood function becomes highly skewed or lacks a defined mode, making the normal approximation—which only uses the mode and its curvature (Fisher information)—highly inaccurate. Discarding sites with zero events, a common practice, further exacerbates bias by ignoring valuable information about the upper bounds of risk.

Proposed Methods

To address these limitations, the authors propose moving away from simple normal approximations in favor of four more robust methods:

  1. Normal Approximation (The Baseline): Uses only the mode and Fisher information; fails if a site has zero outcomes because the mode is undefined.
  2. Skew-normal Approximation: Adds a "shape" parameter to account for asymmetry in the likelihood function.
  3. Custom Parametric Function: A novel function designed specifically to mimic the "double exponential" behavior often seen in Cox likelihood functions.
  4. Grid Approximation: A high-resolution sampling of the log-likelihood across 1,000 equally spaced points, serving as a near-exact representation.

Evaluation and Key Results

The researchers tested these methods through extensive simulations (over 750 scenarios) and real-world applications using four massive US databases (CCAE, MDCR, MDCD, and Optum). They examined factors such as the number of sites and the "treated fraction"—the proportion of patients receiving the treatment versus the comparator.

The simulations revealed a "Treated Fraction Trap": bias becomes severe when group sizes are highly unequal (e.g., 90% vs 10%). In these scenarios, the normal approximation's coverage of the true hazard ratio plummeted to as low as 25%.

In contrast, the skew-normal, grid, and custom parametric methods maintained near-perfect 95% coverage and minimal bias across all scenarios. Furthermore, the methods proved robust as the number of sites increased, consistently outperforming traditional meta-analysis.

Real-World Impact

The methods were applied to three safety studies of antidepressants. In a comparison of Amitriptyline versus Citalopram, the traditional normal-based meta-analysis incorrectly yielded a statistically significant risk. However, the authors' non-normal approximations correctly identified the risk as non-significant, matching the results of a "gold standard" analysis where all data was pooled—a process usually impossible in practice due to privacy constraints.

Conclusion

By moving beyond simple normal approximations, researchers can significantly improve the accuracy of distributed medical studies, especially for rare outcomes. The custom parametric function offers a balance of precision and communication efficiency. To facilitate adoption, the authors have released the EvidenceSynthesis R package on CRAN. Future work will look to extend these techniques to multi-variable adjustments and other regression models like Poisson and logistic regression.

Summary updated

Abstract

Studies of the effects of medical interventions increasingly take place in\ndistributed research settings using data from multiple clinical data sources\nincluding electronic health records and administrative claims. In such\nsettings, privacy concerns typically prohibit sharing of individual patient\ndata, and instead, analyses can only utilize summary statistics from the\nindividual databases. In the specific but very common context of the Cox\nproportional hazards model, we show that standard meta analysis methods then\nlead to substantial bias when outcome counts are small. This bias derives\nprimarily from the normal approximations that the methods utilize. Here we\npropose and evaluate methods that eschew normal approximations in favor of\nthree more flexible approximations: a skew-normal, a one-dimensional grid, and\na custom parametric function that mimics the behavior of the Cox likelihood\nfunction. In extensive simulation studies we demonstrate how these\napproximations impact bias in the context of both fixed-effects and (Bayesian)\nrandom-effects models. We then apply these approaches to three real-world\nstudies of the comparative safety of antidepressants, each using data from four\nobservational healthcare databases.\n
BibTeX
@misc{schuemie2021combining,
  author = {Martijn J. Schuemie and Yong Chen and David Madigan and Marc A. Suchard},
  title = {{Combining Cox Regressions Across a Heterogeneous Distributed Research\textbackslash\{\}n Network Facing Small and Zero Counts}},
  year = {2021},
  eprint = {2101.01551},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2101.01551},
}
Figure