The impact on clinical success from the 23andMe cohort

Xin Wang, Sotiris Karagounis, Suyash Shringarpure, Rohith Srivas, Qiaojuan Jane Su, Vladimir Vacic, Steven J. Pitts, Adam AutonmedRxiv 2024 · DOI

TL;DR

By leveraging a dataset of 15 million individuals, 23andMe researchers demonstrate that drug targets supported by human genetics are 2–3x more likely to succeed in clinical trials. Crucially, they show that self-reported data is just as predictive as clinical records, and that high-confidence gene mapping combined with rare, large-effect variants can boost success enrichment by up to 5x.

What Problem Does This Paper Address?

The pharmaceutical industry faces a staggering 90% failure rate for new therapeutic programs entering clinical trials. To improve these odds, researchers have turned to "natural experiments"—human genetic variations that mimic the effects of a drug. While previous studies suggested that targets with genetic support are twice as likely to succeed, several questions remained:

  1. Does the "noisy" nature of self-reported data (common in consumer genetics) diminish its predictive value compared to clinical records?
  2. Does the massive scale of a cohort (moving from 1 million to 15 million people) actually uncover more actionable targets?
  3. Do rare variants with large effects provide better "blueprints" for success than common variants with tiny effects?

Methods

The authors employed a multi-step analytical pipeline to link genetic associations to clinical outcomes:

  • Phenotype Mapping: They mapped 23andMe’s self-reported survey data to the Experimental Factor Ontology (EFO) to ensure compatibility with clinical trial databases like Citeline Pharmaprojects.
  • Massive GWAS: They conducted Genome-Wide Association Studies (GWAS) on ~7.5 million consented individuals across five ancestry groups, utilizing both genotyped and imputed variants (over 172 million variants total).
  • Variant-to-Gene (V2G) Scoring: Instead of just looking at the nearest gene to a genetic hit, they built a machine-learning model (stochastic gradient boosting) to prioritize causal genes. This model integrated coding variants, expression/protein QTLs, and colocalization data.
  • Relative Success (RS) Analysis: They calculated the "Relative Success" metric—the ratio of the probability of clinical success for targets with genetic support versus those without.

Datasets

  • 23andMe Cohort: ~15 million individuals (7.5 million used for GWAS), representing the world's largest genotype-phenotype database.
  • Public GWAS Data: GWAS Catalog and Open Target Genetics (OTG).
  • Biobanks: UK Biobank, FinnGen, and All of Us.
  • Clinical Data: Citeline Pharmaprojects (used to track the progress of target-indication pairs through Phase I, II, III, and Launch).
  • Gold Standards: Open Targets and Genebass were used to train the V2G models.

Key Insights and Ideas

  • Self-Reported Data is High-Fidelity: One of the most significant "aha" moments is that genetic associations derived from entirely self-reported phenotypes (e.g., "Have you been diagnosed with Type 2 Diabetes?") are just as predictive of clinical success (2–3x enrichment) as those derived from rigorous electronic medical records.
  • Rare Variants Matter More: Contrary to some recent high-profile publications, this study found that rare variants (Minor Allele Frequency < 0.1%) and those with large effect sizes (Odds Ratio > 2) are significantly better predictors of success, offering ~4x enrichment.
  • The Power of Mapping: The "nearest gene" is often not the causal gene. By using advanced V2G mapping to identify the likely causal gene with high confidence, the researchers achieved up to a 5x enrichment in trial success.
  • Doubling the Target Space: By adding 23andMe’s data to the existing public domain data, the researchers effectively doubled the number of target-indication pairs in the clinic that are supported by human genetics.

Key Results

  • Increased Coverage: 23andMe data provided genetic support for 745 unique clinical indications, a 12% increase over all public data combined.
  • Success Multipliers: Genetically supported target-indication pairs had a Relative Success (RS) of 2.17.
  • High-Confidence Enrichment: When focusing on the top tier of their V2G scoring model (99.999th percentile), the enrichment for clinical approval jumped to 4.95x.
  • Rare Variant Discovery: The study estimates that a sequencing cohort of ~3 million individuals is required to discover 95% of the rare variant associations currently detectable via 23andMe’s imputation-based approach.

Limitations and Future Work

  • Ancestry Bias: While 23andMe has large non-European cohorts (over 500,000 African American participants), the majority of the genetically supported targets identified still come from the European dataset due to higher discovery power.
  • Imputation Limits: Imputation quality remains lower in non-European populations, highlighting a need for more diverse reference panels.
  • False Positives: Rare-variant associations are still prone to false positives at standard significance thresholds ($5 \times 10^{-8}$), and the "winner’s curse" may inflate effect size estimates.
  • Future Directions: The authors suggest that as cohort sizes continue to grow and gene-mapping algorithms improve, the industry will be able to move beyond "low-hanging fruit" to identify highly specific, large-effect targets for complex diseases.

Summary updated

Abstract

Abstract 90% of therapeutic programs that enter clinical trials ultimately fail. Human genetic variation provides a set of “natural experiments” that can inform successful strategies for therapeutic discovery. Previous work has estimated that drug targets with human genetics supported mechanisms have a 2-3x increased likelihood of succeeding in the clinic compared to those without. 23andMe, Inc. is a direct-to-consumer genetics company that has created a human genetics dataset approximately an order of magnitude larger in sample size than current publically available cohorts. As of 2024, 23andMe has approximately 15 million individuals with genotype and phenotype data, of which ∼80% consent to participation in research. In this work, we explore how both the scale of the genetic data and improved methods to link genetic associations to putative causal genes impact the prediction of clinical success. Comparing the total number of target-indication pairs that have reached at least phase I that are also supported by genetic evidence, the number of target-indication pairs with support from 23andMe is 60% greater than that with support from all GWAS datasets in the public domain. Including 23andMe genetic evidence approximately doubles the number of target-indication pairs in the clinic that are supported by human genetics. Furthermore, we show that genetic associations derived from entirely self-reported phenotypes are 2-3x enriched for clinical success, just as for clinically derived phenotypes. In contrast to conclusions from the recent publication of Minikel et al. , we found that minor allele frequencies and effect sizes from GWAS influence the relative success estimates for program approvals, and that drug programs supported by rare and large effect associations have greater (3-4x) likelihood to be approved compared to common variant associations with small effects. Finally, improved gene mapping to identify the likely causal genes underlying genetic associations can result in up to 4-5x enrichment for trial success. With the increased power and scale of the 23andMe genetic dataset, we identify an expansive set of opportunities that may be pursued in the clinic, emphasizing the importance of cohort size and gene mapping confidence in deriving clinical value.
BibTeX
@misc{wang2024impact,
  author = {Xin Wang and Sotiris Karagounis and Suyash Shringarpure and Rohith Srivas and Qiaojuan Jane Su and Vladimir Vacic and Steven J. Pitts and Adam Auton},
  title = {{The impact on clinical success from the 23andMe cohort}},
  year = {2024},
  doi = {10.1101/2024.06.17.24309059},
  howpublished = {medRxiv},
}
Figure