TL;DR
This paper introduces a significantly faster and more memory-efficient algorithm for calculating polygenic risk scores (PRSs) from massive genomic datasets. By developing novel data structures for linkage disequilibrium (LD) matrices and optimizing inference algorithms, they enable whole-genome PRS calculations on tens of millions of variants, drastically reducing computational costs and making PRSs more accessible.
What Problem Does This Paper Address?
Genome-wide association studies (GWASs) have identified millions of genetic variants associated with various traits and diseases. However, translating these individual associations into a single, comprehensive "polygenic risk score" (PRS) that captures an individual's overall genetic predisposition has been computationally challenging. Existing methods often struggle with the sheer volume of genetic data, either by simplifying the problem (e.g., using only a small subset of variants) or by requiring excessive computational resources (time and memory). This limits the ability to perform accurate and scalable PRS inference, especially as sequencing and imputation technologies continue to generate ever-larger datasets.
Methods
The core of this work lies in two main areas:
- Data Structures for LD Matrices:
- Compression: They introduce a highly compressed format for Linkage Disequilibrium (LD) matrices. This involves:
- Storing only the upper triangular portion of the symmetric LD matrix.
- Quantizing matrix entries to lower-precision floating-point or integer types (e.g.,
int8reduces storage by 8x). - Utilizing a cloud-native Compressed Sparse Row (CSR) format (Zarr) for efficient storage and access.
- Numerical Stability: They address numerical instabilities in PRS inference by adding a penalty term derived from the smallest eigenvalue of the LD matrix. This helps constrain parameter estimates and prevent exploding values, particularly in high-dimensional settings.
- Algorithmic Improvements:
- Memory-Efficient Inference: They developed a "low-memory" or "triangular LD mode" for their variational Bayesian inference algorithm (VIPRS). This allows the algorithm to operate directly on the compressed, upper-triangular LD matrix, significantly reducing memory footprint.
- Parallelism: They implemented two layers of parallelism:
- Across Chromosomes: Using a process pool to analyze chromosomes independently.
- Within Chromosomes: Parallelizing the coordinate-ascent updates using OpenMP threads.
- Dequantize-on-the-Fly (DQF): An option to dequantize LD matrix entries only when needed during computation, further reducing memory usage at a slight runtime cost.
Datasets
- UK Biobank: Used extensively for training and evaluating PRS models. They utilized summary statistics for 75 heritable, continuous phenotypes.
- CARTAGENE Biobank: Used for cross-biobank validation to assess the transferability of PRS models.
- HapMap3+ reference panel: A standard reference panel for LD information, used in various sizes (e.g., ~1.1 million variants, ~13.5 million variants, ~18 million variants).
- LD matrices: Pre-computed LD matrices for six ancestry groups were generated using the Pan-UKB genotype data.
Key Insights and Ideas
- LD Matrix Compression is Crucial: The paper strongly emphasizes that the way LD matrices are stored and compressed is a major bottleneck. Their novel format achieves dramatic storage reductions (up to 50x), making it feasible to handle much larger variant sets.
- Numerical Stability is Not Just About Data Quality: While data quality is important, the spectral properties of the LD matrix itself (specifically negative eigenvalues) are a significant source of numerical instability in PRS methods. Their proposed solution directly addresses this.
- Parallelism and Memory Efficiency are Synergistic: Combining multi-threading within the inference algorithm with memory-efficient data structures (like the compressed LD matrix) leads to substantial gains in both speed and memory usage.
- Dense Variant Sets Offer Marginal Gains (Sometimes): While using more variants generally improves accuracy, the gains are often modest (3-5%) and can be overshadowed by model assumptions or the quality of input data, especially for well-established methods.
- Cloud-Native Storage is the Future: Their adoption of Zarr for LD matrix storage highlights the trend towards cloud-native solutions for large-scale genomics data.
Key Results
- Storage Reduction: Over 50-fold reduction in LD matrix storage requirements using their compressed format (e.g.,
int8quantization). - Speedup: Orders-of-magnitude improvements in runtime and memory efficiency. VIPRS v.0.1 is up to 80 times faster than popular baselines and uses significantly less memory.
- Scalability: The updated VIPRS can perform variational Bayesian regression on up to 18 million variants in under an hour, using less than 15 GB of RAM.
- Accuracy: Achieved prediction accuracy comparable to state-of-the-art tools, even with massive datasets.
- Numerical Stability: The method demonstrates stable convergence even with ultra-high dimensional data.
Limitations and Future Work
- Hyperparameter Tuning Cost: While grid search improves accuracy, it significantly increases runtime (3-4 fold). They suggest future work could explore more efficient tuning methods.
- Cross-Population Transferability: While their models perform well, the gains from using denser variant sets don't always translate to better cross-ancestry prediction. This suggests that factors beyond variant density, like genetic architecture differences, play a significant role.
- Functional Annotations: The paper notes that incorporating functional genomic annotations could further improve PRS accuracy, a direction they suggest for future research.
- Generalizability of Parallelism: While their parallelization strategies are effective, they acknowledge potential issues like "staleness" in asynchronous updates, which might require further refinement.
Summary updated