PRSformer: Disease Prediction from Million-Scale Individual Genotypes

Payam Dibaeinia, Chris German, Suyash Shringarpure, Adam Auton, Aly A. Khan

TL;DR

This paper introduces PRSformer, a novel deep learning architecture that leverages Transformers with a specialized "neighborhood attention" mechanism to predict disease risk directly from millions of individual genetic profiles. PRSformer significantly outperforms existing methods, especially at large sample sizes, by effectively modeling complex genetic interactions that linear models miss.

What Problem Does This Paper Address?

Predicting disease susceptibility from an individual's genetic makeup is a major goal in precision medicine. However, current approaches, primarily based on Polygenic Risk Scores (PRS) derived from GWAS summary statistics, have limitations. They are often linear, failing to capture complex genetic interactions (like epistasis), and discard valuable individual-level genotype data. As biobanks grow to millions of individuals with millions of genetic markers (ultra-high dimensionality), these limitations become more pronounced, capping predictive performance. The challenge is to develop scalable methods that can effectively utilize this massive amount of individual-level genetic data to model intricate genetic architectures.

Methods

The core innovation is PRSformer, a deep learning architecture built upon the Transformer model. To handle the massive genomic data (millions of variants), they adapt the Transformer's self-attention mechanism with Neighborhood Attention (NA). NA restricts attention computations to a local window of neighboring genetic variants, reducing the computational complexity from quadratic ($O(L^2)$) to linear ($O(L \cdot k)$), where $L$ is the number of variants and $k$ is the window size. This aligns with the genomic structure where variants in close proximity are often inherited together (linkage disequilibrium).

PRSformer uses a stacking of these NA layers to progressively increase the "receptive field," allowing it to model local interactions first and then integrate information across wider genomic regions. The model is designed for end-to-end, multitask learning, meaning it can predict multiple disease risks simultaneously from raw genotype data. Input genetic variants are ordered by chromosomal position, and positional encodings are omitted, relying on the inherent positional information and NA.

Datasets

The primary dataset used is a unique large private cohort with approximately 3.8 million European-ancestry individuals. This cohort was used for training, validation, and testing. For cross-ancestry generalization experiments, an additional ~1.1 million non-European individuals were included, creating a multi-ancestry training set of approximately 5 million individuals. The genetic data consisted of approximately 140,000 variants per individual.

Key Insights and Ideas

  • Non-linearity Matters at Scale: The advantage of non-linear models like PRSformer over linear models only emerges significantly when sample sizes exceed one million individuals. This suggests that complex genetic interactions are difficult to resolve with smaller datasets.
  • Multitask Learning Boosts Performance: Training a single model to predict multiple related autoimmune and inflammatory conditions consistently improved performance compared to training separate models for each disease. This highlights the benefit of learning a shared genetic representation.
  • Genomic Structure is Key: The Neighborhood Attention mechanism, by focusing on local genomic regions, effectively leverages the structure of linkage disequilibrium, proving more beneficial than simply using all variants without considering their spatial relationships.
  • Cross-ancestry Training Enhances Generalization: Training a multi-ancestry model (PRSformer-ME) significantly improved prediction accuracy for non-European populations without sacrificing performance on European individuals, paving the way for more equitable genomic prediction.

Key Results

  • PRSformer consistently outperformed highly optimized linear models and the state-of-the-art summary-statistic method LDPred2 across 18 autoimmune and inflammatory conditions, achieving higher AUROC scores.
  • The performance gain from non-linearity became apparent only when training datasets exceeded 1 million individuals, with the advantage widening as sample size increased.
  • Multitask learning improved AUROC compared to single-task training for both PRSformer and the linear baseline.
  • The multi-ancestry trained PRSformer-ME showed significantly improved generalization to non-European populations, with higher AUROC scores in AFR, LAT, EAS, and SAS individuals compared to a European-only trained model.

Limitations and Future Work

  • Computational Cost: While PRSformer is scalable due to NA, deep learning models generally have higher computational demands than linear models, which might limit immediate clinical scalability.
  • Interpretability: The authors acknowledge the need for methods to interpret the learned non-linear interactions, which is crucial for biological hypothesis generation and experimental validation.
  • Phenome-Scale Prediction: A promising future direction is to extend the framework to predict a wider range of traits (phenome-scale) to capture shared genetic underpinnings across diverse biological systems.
  • Data Availability: The authors note that while their code is released, the specific data and trained models are not publicly available due to privacy concerns.

Summary updated

Abstract

Abstract Predicting disease risk from DNA presents an unprecedented emerging challenge as biobanks approach population scale sizes ( N > 10 6 individuals) with ultra-high-dimensional features ( L > 10 5 genotypes). Current methods, often linear and reliant on summary statistics, fail to capture complex genetic interactions and discard valuable individual-level information. We introduce PRSformer , a scalable deep learning architecture designed for end-to-end, multitask disease prediction directly from million-scale individual genotypes. PRSformer employs neighborhood attention, achieving linear O ( L ) complexity per layer, making Transformers tractable for genome-scale inputs. Crucially, PRSformer utilizes a stacking of these efficient attention layers, progressively increasing the effective receptive field to model local dependencies (e.g., within linkage disequilibrium blocks) before integrating information across wider genomic regions. This design, tailored for genomics, allows PRSformer to learn complex, potentially non-linear and long-range interactions directly from raw genotypes. We demonstrate PRSformer’s effectiveness using a unique large private cohort ( N ≈ 5M) for predicting 18 autoimmune and inflammatory conditions using L ≈ 140k variants. PRSformer significantly outperforms highly optimized linear models trained on the same individual-level data and state-of-the-art summary-statistic-based methods (LDPred2) derived from the same cohort , quantifying the benefits of non-linear modeling and multitask learning at scale. Furthermore, experiments reveal that the advantage of non-linearity emerges primarily at large sample sizes ( N > 1M), and that a multi-ancestry trained model improves generalization, establishing PRSformer as a new framework for deep learning in population-scale genomics.
BibTeX
@inproceedings{dibaeinia2025prsformer,
  author = {Payam Dibaeinia and Chris German and Suyash Shringarpure and Adam Auton and Aly A. Khan},
  title = {{PRSformer: Disease Prediction from Million-Scale Individual Genotypes}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=n1hYQmpCSW},
}
Figure