A two-sample test for high-dimensional data with applications to gene-set testing

Song Xi Chen, Yingli QinThe Annals of Statistics 2010 · DOI

TL;DR

This paper introduces a novel two-sample test designed for "large p, small n" scenarios, where the number of features (p) far exceeds the number of observations (n). This test overcomes the limitations of traditional methods like Hotelling's T2 test, which fail in such high-dimensional settings, and is particularly useful for analyzing complex biological data, such as testing the significance of gene sets.

What Problem Does This Paper Address?

In many modern scientific fields, particularly in biology and finance, researchers are faced with datasets where the number of measured variables (features or dimensions, denoted by 'p') is vastly larger than the number of samples or observations ('n'). This is commonly referred to as the "large p, small n" problem. Traditional statistical tests, like Hotelling's T2 test for comparing means of two groups, are designed for situations where 'p' is much smaller than 'n' and often assume equal covariance matrices. These assumptions break down in high-dimensional settings, leading to unreliable results or even making the tests impossible to compute (e.g., when the sample covariance matrix is not invertible). The paper specifically addresses the challenge of performing a two-sample test for the means of such high-dimensional data, where the goal is to determine if two groups of observations come from populations with the same mean vector. A key application highlighted is in gene-set testing, where researchers want to know if a group of genes (a "gene set") collectively shows differential expression between two conditions, rather than focusing on individual genes.

Methods

The core contribution of this paper is a new two-sample test statistic, denoted as $T_n$. This statistic is designed to be robust in high-dimensional settings and does not require explicit conditions on the relationship between 'p' and 'n', offering significant flexibility. The proposed test statistic is derived by modifying Hotelling's T2 statistic to avoid the problematic inversion of the sample covariance matrix. Instead of relying on the inverse, it uses a quadratic form that is more amenable to high-dimensional analysis. The authors establish the asymptotic normality of this test statistic under certain conditions, which allows for the construction of a hypothesis test. They also develop methods for estimating the variance of the test statistic in a way that is consistent even in high dimensions. A key aspect of their approach is the development of estimators for the trace of covariance matrices and their products, which are crucial for calculating the test statistic's variance. The paper also discusses how to apply this test to gene-set testing by treating each gene set as a single multivariate observation.

Datasets

The paper uses an acute lymphoblastic leukemia (ALL) data set for an empirical study on gene-set testing. This dataset contains microarray expressions for 128 patients, with two sub-classes: T-cell and B-cell type leukemia. Within the B-cell type, there are BCR/ABL and NEG molecular classes. The analysis focuses on comparing gene expression levels between the BCR/ABL (n₁=37) and NEG (n₂=42) molecular sub-classes for gene sets defined by Gene Ontology (GO) terms.

Key Insights and Ideas

  • Beyond Hotelling's T2: The paper demonstrates that traditional multivariate tests like Hotelling's T2 are fundamentally ill-suited for high-dimensional data ("large p, small n") due to issues with covariance matrix estimation and invertibility.
  • Flexibility in p vs. n: The proposed test offers a significant advantage by not requiring strict relationships between 'p' and 'n' (e.g., p growing at a specific rate relative to n), making it applicable to a wider range of high-dimensional scenarios.
  • Gene-Set Testing Power: The test is particularly effective for gene-set testing, allowing researchers to assess the collective significance of groups of genes, which is biologically more meaningful than analyzing individual genes in isolation.
  • Robustness to Sparsity: Simulation studies suggest the proposed test maintains good power even when the number of truly different means is sparse (i.e., only a small fraction of the p dimensions have different means).
  • Improved Variance Estimation: The paper introduces novel estimators for the variance of the test statistic that are ratio-consistent, meaning they accurately estimate the true variance even with high dimensionality.

Key Results

The paper establishes the asymptotic normality of the proposed two-sample test statistic ($T_n$) under general conditions, allowing for hypothesis testing. Simulation studies comparing the proposed test with the Bai-Saranadasa test and multiple comparison procedures (Bonferroni, FDR) show that the proposed test is generally more powerful, especially in high-dimensional settings. For instance, in the simulation studies, the proposed test consistently outperformed the Bai-Saranadasa test and multiple testing procedures, maintaining a reasonably sized approximation to the nominal 5% significance level. In the gene-set testing application on the ALL data, the test identified a substantial number of significant gene sets across different biological categories (BP, MF, CC), highlighting its utility in biological discovery.

Limitations and Future Work

  • Identifying Significant Components: While the proposed test is powerful for detecting differences in means of high-dimensional data or gene sets, it does not inherently identify which specific components (individual genes or dimensions) are responsible for the rejection of the null hypothesis. Further follow-up procedures would be needed for this.
  • Sparsity Impact: Although the test performs well under sparsity, the paper notes that extreme sparsity (very few non-zero differences) can lead to a reduction in power, which is quantified and discussed.
  • Paired Data Extension: The paper briefly mentions that the two-sample test can be modified for paired observations, effectively turning it into a one-sample problem with high-dimensional data. This suggests a potential avenue for future work to fully develop and explore this extension.
  • Finite Dimension Case: The paper acknowledges that when 'p' is fixed (finite dimension), the asymptotic normality guarantees might not hold, and traditional tests like Hotelling's might be more appropriate. However, it suggests the proposed test can still be used with estimated critical values.

Summary updated

Abstract

We propose a two-sample test for the means of high-dimensional data when the data dimension is much larger than the sample size. Hotelling’s classical T2 test does not work for this “large p, small n” situation. The proposed test does not require explicit conditions in the relationship between the data dimension and sample size. This offers much flexibility in analyzing high-dimensional data. An application of the proposed test is in testing significance for sets of genes which we demonstrate in an empirical study on a leukemia data set.
BibTeX
@article{chen2010two,
  author = {Song Xi Chen and Yingli Qin},
  title = {{A two-sample test for high-dimensional data with applications to gene-set testing}},
  journal = {The Annals of Statistics},
  year = {2010},
  doi = {10.1214/09-aos716},
}
Figure