Improving power for rare variant tests by integrating external controls

Seunggeun Lee, Sehee Kim, Christian Fuchsberger

TL;DR

This paper introduces a new statistical method, iECAT, that effectively integrates external control samples into rare variant association tests. This significantly boosts the power to detect genetic associations for complex diseases while crucially maintaining control over false positive rates, even in the presence of batch effects.

What Problem Does This Paper Address?

The cost of sequencing genomes has plummeted, leading to a massive increase in available genomic data. This presents a huge opportunity to improve the power of rare variant association tests by leveraging "external" control samples from other studies, in addition to a study's own "internal" controls. However, a major hurdle arises: external control samples often come from different sequencing platforms or analysis pipelines, introducing "batch effects." These batch effects can drastically inflate the rate of false positives (Type I error), making it difficult to trust the results. The paper aims to solve this by developing a method that can harness the power of external controls without succumbing to the pitfalls of batch effects.

Methods

The core innovation is the integrating External Controls into Association Test (iECAT) framework. The key insight is that batch effects can be detected by comparing the estimated effect size (odds ratio) of a variant using only internal controls versus using both internal and external controls. If these estimates differ significantly, it suggests a batch effect.

iECAT uses an Empirical Bayesian approach to estimate and adjust for this bias. It first calculates an initial estimate of the bias and then uses a shrinkage method to combine this bias estimate with the original estimate, effectively "shrinking" the estimate towards a more reliable value. This approach allows the method to adaptively decide how much to rely on external controls based on the evidence of batch effects.

The method is extended from single-variant tests to gene- or region-based tests, incorporating popular frameworks like Burden, SKAT, and SKAT-O. This allows for a more comprehensive analysis of genetic regions rather than individual variants.

Datasets

The methods were evaluated using:

  • Simulated datasets: Extensive simulations were performed to assess Type I error rates and power under various scenarios, including different sample sizes, case-control ratios, and levels of batch effects and population stratification.
  • Real-world datasets:
    • Age-related macular degeneration (AMD) study: Targeted sequencing data was analyzed.
  • Genetics of Type-2 Diabetes (GoT2D) study: Deep whole exome sequencing data was analyzed.
  • European samples from the Exome Variant Server (ESP): These were used as external controls for both real data analyses.

Key Insights and Ideas

  • Batch effects are a major threat: The study clearly demonstrates that ignoring batch effects when integrating external controls leads to dramatically inflated Type I error rates, rendering results unreliable.
  • Adaptive bias correction is crucial: iECAT's ability to detect and adjust for batch effects in an adaptive manner is key to its success. It doesn't blindly use external controls but intelligently weighs their contribution based on evidence of bias.
  • External controls offer substantial power gains: When batch effects are properly handled, integrating external controls can lead to significant improvements in the power to detect associations, which is critical for finding associations with rare variants.
  • Allele counts are sufficient: A major advantage is that iECAT only requires allele count information from external studies, making it highly practical given the availability of such data in public repositories.

Key Results

  • Type I Error Control: iECAT consistently maintained well-controlled or only slightly inflated Type I error rates across various simulation scenarios, even in the presence of batch effects and population stratification. In contrast, methods that ignored batch effects showed massively inflated error rates.
  • Power Improvement: In simulations, iECAT demonstrated substantially improved power compared to using internal controls alone. For example, with a sample size of 10,000, iECAT was 38-60% more powerful.
  • Real Data Validation:
    • In the AMD study, iECAT identified two well-known AMD-related genes (C3 and CFH) with highly significant p-values, which were not as significant when using SKAT-O with internal controls alone.
  • In the GoT2D study, QQ plots showed that iECAT was well-calibrated, while the unadjusted method (iECAT-ONoadj) showed significant inflation, highlighting the importance of the proposed adjustment.

Limitations and Future Work

  • Covariate Adjustment: The current methods cannot adjust for covariates like age and gender. This is a key area for future research.
  • Hardy-Weinberg Equilibrium (HWE) Assumption: The method assumes HWE holds. While the authors note that violation might have limited effects for rare variants, it's still an assumption.
  • Singletons and Doubletons: The method is not recommended for singletons or doubletons due to unstable log odds ratio estimates. Future work will extend the method to include these variants.
  • Filtering External Samples: A potential issue with using variant servers is the inability to filter samples by genetic background, which could lead to slightly inflated Type I error rates. The authors believe this will become less problematic as servers provide finer-scale ancestry information.

Summary updated

Abstract

Abstract Due to the drop in sequencing cost, the number of sequenced genomes is increasing rapidly. To improve power of rare variant tests, these sequenced samples could be used as external control samples in addition to control samples from the study itself. However, when using external controls, possible batch effects due to the use of different sequencing platforms or genotype calling pipelines can dramatically increase type I error rates. To address this, we propose novel summary statistics-based single and gene- or region-based rare-variant tests that allow the integration of external controls while controlling for type I error. Our approach is based on the insight that batch effects on a given variant can be assessed by comparing odds ratio estimates using internal controls only vs. using combined control samples of internal and external controls. From simulation experiments and the analysis of data from age related macular degeneration and type 2 diabetes studies, we demonstrate that our method can substantially improve power while controlling for type I error rate.
BibTeX
@inproceedings{lee2017improving,
  author = {Seunggeun Lee and Sehee Kim and Christian Fuchsberger},
  title = {{Improving power for rare variant tests by integrating external controls}},
  booktitle = {Genetic},
  year = {2017},
  doi = {10.1002/gepi.22057},
}
Figure