TL;DR
IsoAlign is a multimodal framework that significantly boosts the performance of genomic foundation models (GFMs) by aligning DNA sequences with their specific protein isoforms. By utilizing Joint-Embedding Predictive Architectures (JEPA), the model moves beyond simple sequence reconstruction to capture the functional complexity of alternative splicing, achieving state-of-the-art results in predicting mutation fitness, variant pathogenicity, and gene expression.
What Problem Does This Paper Address?
Most current genomic foundation models treat DNA like a "flat" text document, using self-supervised techniques like Masked Language Modeling (MLM) to learn sequence patterns. However, biology isn't flat—it's hierarchical. A single genomic locus can produce multiple different protein "isoforms" through a process called alternative splicing.
Existing models often overlook this one-to-many relationship, leading to representations that struggle to map genetic variations to their actual functional consequences in a cell. Because they don't explicitly model how DNA translates into diverse protein products, these general-purpose models often fail to outperform smaller, task-specific "specialist" models on clinically relevant tasks like variant effect prediction.
Methods
The authors introduce IsoAlign, a dual-encoder framework that pairs a trainable DNA encoder with a frozen, high-performance protein encoder (ESM2). To bridge these modalities, they propose two novel architectures based on the Joint-Embedding Predictive Architecture (JEPA) paradigm:
- DP-JEPA (DNA-Protein JEPA): This model learns to predict high-level protein representations directly from DNA sequences in a shared latent space using contrastive alignment.
- IM-JEPA (Isoform-Masked JEPA): This extension models the relationships between different isoforms of the same gene. It masks approximately 50% of the protein isoforms in a set and tasks the model with predicting them using the remaining "visible" protein context and DNA "anchors."
To prevent the latent space from collapsing (a common issue where the model gives the same output for every input), they employ SIGReg, an anti-collapse regularization term that encourages the predicted embeddings to follow a standard Gaussian distribution.
Datasets
- Training: DNA-protein pairs were constructed from the Broad’s Gene-to-Protein (G2P) database, linking Ensembl transcripts to UniProt protein isoforms.
- Evaluation:
- Deep Mutational Scanning (DMS): Five datasets (CBS, P53, PDE3A, GDI1) measuring how mutations affect protein fitness.
- Variant Effect Prediction (VEP): ClinVar Splicing (clinically annotated variants) and MFASS (functional assay of splicing disruption).
- TFLink: A database of 8.8M validated human transcription factor (TF) binding sites.
- GTEx: Tissue-specific transcript expression data across five human tissues.
Key Insights and Ideas
- Beyond the Codon Table: A critical "aha moment" came from sequence ablation studies. If the model were just memorizing the 3-to-1 genetic code (codons to amino acids), shuffling codons while preserving the protein sequence shouldn't hurt performance. Instead, codon shuffling degraded performance significantly (19%), proving that IsoAlign captures regulatory syntax—like codon usage bias and translational efficiency—rather than just a simple lookup table.
- Protein-Informed DNA Representations: By "grounding" DNA pretraining in protein space, the model learns which parts of the DNA sequence are functionally critical. This cross-modal alignment provides a much stronger inductive bias than DNA-only masking.
- Isoform Sensitivity: Unlike models that aggregate data at the gene level, IsoAlign distinguishes between subtle isoform-level differences. This is vital for understanding diseases caused by splicing errors.
Key Results
IsoAlign consistently outperformed both unimodal DNA baselines and existing cross-modal models across the board:
-
Fitness Prediction: Best-performing models achieved a median Spearman correlation of 0.557, outperforming 16 other GFMs and 5 protein foundation models (PFMs). Spearman correlation results showing IsoAlign (blue) consistently outperforming DNA-only baselines (gray).
-
Variant Effect Prediction: On the ClinVar Splicing dataset, IsoAlign improved the AUPRC of the NTv2-500M backbone from 0.32 to 0.63, a massive leap in functional discriminability. Table showing significant AUPRC gains for IsoAlign models compared to parent DNA backbones.
-
Transcription Factor Binding: IsoAlign achieved an AUROC of 0.934, significantly outperforming specialist models like BPNet and ChromBPNet. Comparison of TFLink binding prediction performance.
-
Gene Expression: On the GTEx benchmark, IsoAlign reached an overall Spearman correlation of 0.725, demonstrating superior modeling of tissue-specific expression.
Limitations and Future Work
The authors note that IsoAlign is currently limited by the completeness of DNA-protein databases. Curated benchmarks may not yet capture the full spectrum of context-specific splicing or rare patient-derived variations.
Future directions include:
- Incorporating epigenomic data (like chromatin accessibility) and 3D structural information.
- Evaluating the model on patient-derived cohorts to test clinical utility.
- Developing safeguards to ensure the models are robust against population bias and protect genomic privacy.
Summary updated