An Efficient Test for Comparing Sequence Diversity between Two Populations

Peter B. Gilbert, Vladimir Novitsky, Monty Montano, Max EssexJournal of Computational Biology 2001 · DOI

TL;DR

This paper introduces a novel statistical test designed to compare genetic sequence diversity between two populations, specifically addressing the challenge of multiple sequences per individual. Their method, a "global test," efficiently combines all possible pairwise comparisons, overcoming the limitations of traditional tests that either ignore valuable data or suffer from inflated Type I error rates. This approach offers a more powerful, objective, and efficient way to analyze sequence diversity, with direct applications in fields like HIV research.

What Problem Does This Paper Address?

When studying genetic diversity within populations, especially for rapidly evolving viruses like HIV, researchers often collect multiple genetic sequences from each individual. This is crucial because intra-individual diversity can be significant. However, comparing the overall genetic diversity between two populations becomes tricky when you have multiple sequences per person. Standard statistical tests, like the t-test, assume independence between data points. When you calculate distances between all possible pairs of sequences within a group, these distances are no longer independent – they are "data-dependent" due to the shared sequences from the same individual. This dependency can lead to an inflated Type I error rate, meaning you're more likely to incorrectly conclude there's a difference between populations when there isn't.

A common workaround has been to simplify the data by using only one sequence or a consensus sequence from each individual. However, this approach has two major drawbacks: 1) the results can be highly dependent on which arbitrary sequence or consensus is chosen, and 2) it sacrifices statistical power by discarding valuable information from the extra sequences. This paper aims to solve this by developing a statistical test that can effectively utilize all available sequences, even with multiple per individual, without compromising the validity of the test or losing statistical power.

Methods

The core of the paper is the development of a "global test" statistic. This test is based on the idea of linearly combining multiple standard two-sample test statistics (like t-statistics or Wilcoxon rank sum statistics) that can be calculated from all possible unique pairs of sequences within and between the two populations.

Here's a breakdown of the approach:

  • U-statistics: The foundation of the test relies on U-statistics, a general class of statistics used for comparing distributions. For every possible pair of sequences within group 1 (say, sequence k and k') and every possible pair within group 2 (sequence l and l'), a standard two-sample U-statistic can be computed to test for differences in their distributions.
  • Linear Combination: The "global test" statistic is a weighted sum of all these individual U-statistics. The weights can be chosen in various ways, including equal weights for all U-statistics (simplest approach) or weights that optimize the statistical power of the test.
  • Addressing Data Dependency: The key innovation is how this combination accounts for the data dependencies. By carefully constructing the statistic and its variance, the test remains valid even with multiple sequences per individual. The paper provides detailed mathematical derivations for the covariance matrix of these combined statistics and how to estimate it.
  • Asymptotic Normality: Under certain conditions, the global test statistic is shown to be asymptotically normally distributed, allowing for standard hypothesis testing and confidence interval construction.
  • Monte Carlo Permutation: For situations where the normality assumption might be in doubt (e.g., small sample sizes), a Monte Carlo permutation procedure is proposed to determine significance levels.

Datasets

The paper uses two primary datasets for illustration and evaluation:

  1. HIV-1 Subtype C from Botswana: This dataset consists of nearly full-length HIV-1 genome sequences from 23 patients in Botswana. The sequencing yielded varying numbers of clones per patient (ranging from one to five).
  2. HIV-1 Subtype B from Los Alamos Database: This dataset comprises 27 full-length subtype B sequences from 23 patients, mostly with only one sequence per patient, but some with two or three.

The authors also refer to data from Wolinsky et al. (1996) for a comparison of partial envelope sequences from HIV-1 B strains in Chicago MACS cohort.

Key Insights and Ideas

  • The "Curse of Dimensionality" in Sequence Data: The paper highlights a common problem in sequence analysis: as you collect more data (more sequences per individual), standard statistical tools can break down or become less reliable due to inherent data dependencies.
  • Leveraging All Data: The most significant insight is that by intelligently combining information from all available sequences, you can achieve a more powerful and objective test than by arbitrarily selecting a subset. This is particularly relevant for rapidly evolving pathogens where intra-patient diversity is a key feature.
  • Efficiency Gains: The global test can offer substantial statistical power advantages, especially when there are multiple sequences per patient and these sequences are not highly correlated. The paper quantifies these potential efficiency gains, showing they can be several-fold.
  • Objectivity and Robustness: The global test is presented as maximally efficient and objective because its conclusion doesn't depend on the arbitrary choice of which sequences to include. It provides a single, unified result.
  • Broad Applicability: While the paper focuses on HIV, the methodology is general and can be applied to comparing sequence diversity in any organism where multiple samples per individual are available.

Key Results

  • Demonstrated Superiority: The paper demonstrates through theoretical analysis and application to HIV data that the global test can be more statistically powerful than traditional methods that use only one sequence per patient.
  • HIV-1 Subtype Comparison: Applying the test to Botswana subtype C and Los Alamos subtype B HIV-1 sequences, the authors found statistically significant differences in interpatient genetic distance across all 12 genomic regions examined.
  • Efficiency Estimates: For the Botswana dataset, the observed asymptotic relative efficiency (ARE) of the global test compared to a t-test using only the first sequence was modest (around 0.48-0.92 for different genes). However, this was because the subtype B dataset had very few individuals with multiple sequences. When comparing to U-statistics that consider second or third sequences, the efficiency advantage became substantial (2 to 10 times more efficient).
  • Confidence Intervals: The paper shows how to construct confidence intervals for the difference in location parameters, providing a range of plausible values for the true difference in diversity between populations.

Limitations and Future Work

  • Computational Complexity: While the method is powerful, the number of possible U-statistics can grow rapidly with the number of sequences per patient and the number of patients, potentially leading to high computational demands. The paper mentions that the covariance matrix calculation can be complex.
  • Assumptions: The test relies on the assumption of exchangeability of marginal distributions of interpatient pairwise distances. While generally tenable, it's a theoretical underpinning.
  • Weighting Schemes: The paper discusses different weighting schemes for the global statistic, noting that the choice of weights can affect the results. Further research could explore optimal weighting strategies for various scenarios.
  • Generalization to Other Data Types: While the paper focuses on nucleotide sequence distances, the framework of combining U-statistics could potentially be extended to other types of genetic distance measures or even other biological data where repeated measurements are taken.
  • Applications Beyond HIV: The authors suggest broader applications in vaccine development, particularly for "sieve analysis" hypotheses, and in selecting viral antigens for vaccines. Further exploration of these applications would be valuable.

Summary updated

Abstract

We address the problem of comparing interindividual genomic sequence diversity between two populations. Although the methods are general, for concreteness we focus on comparing two human immunodeficiency virus (HIV) infected populations. From a viral isolate(s) taken from each individual in a sample of persons from each population, suppose one or multiple measurements are made on the genetic sequence of a coding region of HIV. Given a definition of genetic distance between sequences, the goal is to test if the distribution of interindividual distances differs between populations. If distances between all pairs of sequences within each group are used, then data-dependencies arising from the use of multiple sequences from individuals invalidates the use of a standard two-sample test such as the t-test. Where this problem has been recognized, a typical solution has been to apply a standard test to a reduced dataset comprised of one sequence or a consensus sequence from each patient. Disadvantages of this procedure are that the conclusion of the test depends on the choice of utilized sequences, often an arbitrary decision, and exclusion of replicate sequences from the analysis may needlessly sacrifice statistical power. We present a new test free of these drawbacks, which is based on a statistic that linearly combines all possible standard test statistics calculated from independent sequence subsamples. We describe statistical power advantages of the test and illustrate its use by application to nucleotide sequence distances measured from HIV-1 infected populations in southern Africa (GenBank accession numbers AF110959--AF110981) and North America/Europe. The test makes minimal assumptions, is maximally efficient and objective, and is broadly applicable.
BibTeX
@article{gilbert2001efficient,
  author = {Peter B. Gilbert and Vladimir Novitsky and Monty Montano and Max Essex},
  title = {{An Efficient Test for Comparing Sequence Diversity between Two Populations}},
  journal = {Journal of Computational Biology},
  year = {2001},
  doi = {10.1089/106652701300312904},
}
Figure