Abstract
Replacing Relative Risk (RR) or Risk Difference (RD) with the Odds Ratio (OR) is currently unadvisable in clinical trials and meta-analyses. It is possible that no effect measure is truly "portable" across different study populations. In cases where the portability of the effect measure is challenging to satisfy, we suggest presenting the conditional effect based on the baseline risk using a bivariate generalized linear mixed model (BGLMM). This model accounts for the correlation between the effect measure and baseline disease risk, allowing investigators to report effects conditioned on baseline risk in addition to the marginal effect.
The Portability Debate in Clinical Research
In clinical trials and meta-analyses, researchers typically describe treatment effects using Risk Difference (RD), Relative Risk (RR), or the Odds Ratio (OR). A recent controversial proposal suggested that the OR should be the universal standard because it is supposedly "portable"—meaning the ratio remains constant regardless of whether the baseline risk of the population is high or low.
However, this "portability" is a mathematical assumption that rarely holds up in real-world clinical settings. Furthermore, the OR presents two major challenges for practitioners:
- Interpretability: Clinicians and patients naturally think in terms of "risk," not "odds." Misinterpreting an OR of 6.0 as a six-fold increase in risk (when the RR might only be 2.0) can lead to dangerously exaggerated clinical conclusions.
- Non-collapsibility: Unlike RR and RD, the OR has a mathematical property where the "crude" effect for a whole group can differ from the "stratum-specific" effects for subgroups, even in the absence of confounding factors.
Empirical Evaluation of the Portability Myth
To test the claim of OR portability, researchers conducted a massive empirical evaluation using the Cochrane Database of Systematic Reviews (CDSR). They analyzed 40,243 meta-analyses (updated to January 2020) containing at least three studies with binary outcomes.
By calculating the Spearman’s rank correlation coefficient between study-specific ORs and baseline risks, the researchers found that in nearly 50% of the meta-analyses, there was a moderate to strong correlation (absolute values of Spearman’s $\rho > 0.5$). Most of these correlations were negative, meaning that as the baseline risk of a disease increases, the observed Odds Ratio tends to decrease. This evidence suggests that the OR is not a "plug-and-play" metric that remains stable across different populations.
A Better Path: Bivariate Generalized Linear Mixed Models (BGLMM)
When portability fails, the variability of an effect measure with baseline risk should be treated as a feature, not a bug. Clinical meaning should change based on how common a disease is; an effect that doubles a one-in-a-million risk is clinically distinct from one that doubles a 40% risk.
To address this, the authors propose using a Bivariate Generalized Linear Mixed Model (BGLMM). Unlike standard meta-regression, which treats baseline risk as a fixed covariate—a practice often criticized for statistical pitfalls—BGLMM jointly models the underlying event risks from both the control and treatment groups as random effects. This allows researchers to estimate "conditional effects," showing how the treatment effect changes based on a specific baseline risk.
Case Study: Sumatriptan for Migraines
The researchers re-analyzed a meta-analysis of 20 studies comparing oral sumatriptan (100 mg) vs. placebo for acute migraine relief. The OR was far from stable across studies:
- At a 0.10 baseline risk, the OR estimate was 6.73.
- At a 0.43 baseline risk, the OR estimate dropped to 2.56.
The BGLMM matched the observed data well, providing a more nuanced view than traditional random-effects models that ignore the correlation between effect size and baseline risk.
Conclusion
The study concludes that it is possible that no effect measure (OR, RR, or RD) is truly portable across all meta-analyses. Biological and environmental differences between study populations often mean that a treatment's impact is inherently tied to the baseline risk. Instead of searching for a "perfect" portable measure, the field should move toward reporting conditional effects that reflect the reality of diverse patient populations using robust statistical frameworks like BGLMM.
Summary updated