A simple and flexible test of sample exchangeability with applications to statistical genomics

Alan J. Aw, Jeffrey P. Spence, Yun S. Song

Abstract: In scientific studies involving analyses of multivariate data, basic but important questions often arise for the researcher: Is the sample exchangeable, meaning that the joint distribution of the sample is invariant to the ordering of the units? Are the features independent of one another, or perhaps the features can be grouped so that the groups are mutually independent? In statistical genomics, these considerations are fundamental to downstream tasks such as demographic inference and the construction of polygenic risk scores. We propose a non-parametric approach, which we call the V test, to address these two questions, namely, a test of sample exchangeability given dependency structure of features, and a test of feature independence given sample exchangeability. Our test is conceptually simple, yet fast and flexible. It controls the Type I error across realistic scenarios, and handles data of arbitrary dimensions by leveraging large-sample asymptotics. Through extensive simulations and a comparison against unsupervised tests of stratification based on random matrix theory, we find that our test compares favorably in various scenarios of interest. We apply the test to data from the 1000 Genomes Project, demonstrating how it can be employed to assess exchangeability of the genetic sample, or find optimal linkage disequilibrium (LD) splits for downstream analysis. For exchangeability assessment, we find that removing rare variants can substantially increase the p-value of the test statistic. For optimal LD splitting, the V test reports different optimal splits than previous approaches not relying on hypothesis testing. Software for our methods is available in R (CRAN: flintyR) and Python (PyPI: flintyPy).

Introduction

In fields like statistical genomics, researchers often rely on two fundamental assumptions: sample exchangeability and feature independence. Sample exchangeability implies that individuals in a study come from a single, well-mixed population without hidden structure. Feature independence assumes that genetic markers, such as Single Nucleotide Polymorphisms (SNPs), are independent or can be partitioned into independent blocks (Linkage Disequilibrium blocks).

If these assumptions are violated, downstream analyses—such as predicting disease risk via polygenic risk scores or inferring demographic history—can produce wildly inaccurate results. Existing tools, such as the Tracy-Widom test from random matrix theory, often struggle when data is "unbalanced" (e.g., having many individuals from one population and only a few from another) or when the data does not perfectly fit specific distribution shapes. There is a critical need for a test that is distribution-free and robust to the messy realities of biological data.

Allele dosage and frequency across populations (LWK and YRI) illustrating population structure.

Methods: The V Test

The authors introduce the V test, a non-parametric statistical framework designed to check if a multivariate sample is exchangeable or if its features are truly independent. The test is built on a simple intuition: in an exchangeable sample, the "spread" of distances between individuals should be predictable.

The Statistic and Null Model

The test calculates the empirical variance ($V$) of all pairwise distances (such as Hamming distance for DNA) between individuals in a sample. The null hypothesis, termed "Exchangeable Sample and Independent Features" (ES&IF), posits that if you shuffle the data within each feature, the variance of distances should not change significantly if the original sample was exchangeable.

Conceptual overview of the V test intuition and null model construction.

Computational Efficiency

While shuffling data (permutation testing) is computationally expensive, the authors provide mathematical proofs showing that for high-dimensional data (large $P$), the null distribution follows a weighted sum of chi-square distributions. This asymptotic approximation allows the test to run approximately 2000x faster than raw permutations. For a dataset with 500 individuals and 500 features, the chi-square approximation took roughly 0.04 seconds, compared to 96 seconds for the standard permutation-based approach.

Key Results and Applications

The V test was validated using synthetic binary data and real-world genetic data from the 1000 Genomes Project, including populations such as the Yoruba (YRI) and Luhya (LWK).

  • Robustness to Imbalance: In head-to-head simulations against the Tracy-Widom (TW) test, the V test consistently achieved an AUROC of at least 0.5, whereas the TW test frequently dipped below 0.5 in unbalanced scenarios where one population was vastly overrepresented.
  • The Rare Variant Effect: The authors discovered that removing rare genetic variants significantly increases the $p$-value of the test. This suggests that much of the signal for non-exchangeability is carried by rare mutations specific to small sub-groups or lineages.
  • Validating LD Splits: The test provides a formal way to solve the "optimal LD splitting problem"—deciding where to partition the genome into independent blocks for parallel processing—reporting different optimal splits than previous heuristic approaches.

Conclusion

The V test offers a flexible and efficient alternative to traditional spectral tests for assessing population stratification. While it requires the user to specify the dependency structure of features, its non-parametric nature makes it highly adaptable to various genomic contexts. Future work may extend this framework to incorporate feature weights or handle more complex data types beyond binary SNPs.

Software Availability: The methods are implemented in the open-source packages flintyR (CRAN) and flintyPy (PyPI).

Summary updated

Abstract

In scientific studies involving analyses of multivariate data, basic but important questions often arise for the researcher: Is the sample exchangeable, meaning that the joint distribution of the sample is invariant to the ordering of the units? Are the features independent of one another, or perhaps the features can be grouped so that the groups are mutually independent? In statistical genomics, these considerations are fundamental to downstream tasks such as demographic inference and the construction of polygenic risk scores. We propose a non-parametric approach, which we call the V test, to address these two questions, namely, a test of sample exchangeability given dependency structure of features, and a test of feature independence given sample exchangeability. Our test is conceptually simple, yet fast and flexible. It controls the Type I error across realistic scenarios, and handles data of arbitrary dimensions by leveraging large-sample asymptotics. Through extensive simulations and a comparison against unsupervised tests of stratification based on random matrix theory, we find that our test compares favorably in various scenarios of interest. We apply the test to data from the 1000 Genomes Project, demonstrating how it can be employed to assess exchangeability of the genetic sample, or find optimal linkage disequilibrium (LD) splits for downstream analysis. For exchangeability assessment, we find that removing rare variants can substantially increase the <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>p</mml:mi></mml:math>-value of the test statistic. For optimal LD splitting, the V test reports different optimal splits than previous approaches not relying on hypothesis testing. Software for our methods is available in R (CRAN: flintyR) and Python (PyPI: flintyPy).
BibTeX
@article{aw2021simple,
  author = {Alan J. Aw and Jeffrey P. Spence and Yun S. Song},
  title = {{A simple and flexible test of sample exchangeability with applications to statistical genomics}},
  journal = {The Annals of Applied Statistics},
  year = {2021},
}
Figure