EpiFoundation: A Foundation Model for Single-Cell ATAC-seq via Peak-to-Gene Alignment

Jun-Cheng Wu, Changxin Wan, Zhicheng Ji, Yuyin Zhou, Wenpin HoubioRxiv 2025 · DOI

TL;DR

Researchers have developed EpiFoundation, a novel foundation model specifically designed for single-cell ATAC-seq (scATAC-seq) data. By leveraging paired scRNA-seq data for supervision and focusing on non-zero peaks, EpiFoundation effectively learns cell representations and achieves state-of-the-art performance in crucial downstream tasks like cell type annotation, batch correction, and gene expression prediction.

What Problem Does This Paper Address?

Single-cell ATAC-seq (scATAC-seq) is a powerful technology for understanding gene regulation by measuring chromatin accessibility at the single-cell level. However, analyzing scATAC-seq data presents significant challenges: the data is extremely high-dimensional (millions of genomic "peaks") and highly sparse (many peaks are not accessible in any given cell). This sparsity makes it difficult to model relationships between peaks and to extract meaningful biological signals. While foundation models have revolutionized scRNA-seq analysis, a dedicated foundation model for scATAC-seq, capable of handling its unique data characteristics and enabling diverse downstream applications, was lacking.

Methods

EpiFoundation tackles these challenges with a novel cross-modality pre-training approach:

  • Non-zero Peak Focus: Instead of processing all possible peaks, EpiFoundation focuses exclusively on the "non-zero" peaks (those showing accessibility in a given cell). This significantly reduces the input dimensionality and enhances the density of cell-specific information, making the model more efficient and effective.
  • Peak-to-Gene Alignment: The model uses paired scRNA-seq data as a supervisory signal during pre-training. This "peak-to-gene alignment" forces the model to learn representations that link chromatin accessibility (ATAC-seq) to gene expression (RNA-seq), a crucial step for understanding gene regulation.
  • Transformer Architecture: A Transformer-based architecture is employed to process the input embeddings of non-zero peaks and their corresponding chromosome information, generating a robust cell representation.
  • Batch Effect Mitigation: Batch information is incorporated during pre-training to ensure that the learned cell representations are unbiased and can be effectively used for downstream tasks like batch correction.

Datasets

  • MiniAtlas: The authors introduce a new benchmark dataset called MiniAtlas, comprising over 100,000 single cells with paired scATAC-seq and scRNA-seq measurements across 19 tissues and 56 cell types. This dataset is crucial for training and validating EpiFoundation.
  • Test Sets: Diverse test sets from kidney, peripheral blood mononuclear cells (PBMC), and bone marrow mononuclear cells (BMMC) tissues, as well as an "ALLTissue" set, were used for robust evaluation.

Key Insights and Ideas

  • Focusing on the "Active" Peaks is Sufficient: The paper's core hypothesis is that modeling only the accessible (non-zero) peaks is enough to capture essential cell-specific information for representation learning, overcoming the sparsity challenge.
  • Cross-Modality Supervision is Key: Using gene expression data to guide the learning of ATAC-seq representations is a powerful strategy for linking epigenetic information to functional outcomes.
  • Foundation Models for Epigenomics are Feasible and Powerful: EpiFoundation demonstrates that the foundation model paradigm, successful in scRNA-seq, can be effectively adapted to scATAC-seq, unlocking new possibilities for analyzing epigenetic data.

Key Results

EpiFoundation achieved state-of-the-art performance across multiple downstream tasks:

  • Cell Type Annotation: Demonstrated high accuracy (e.g., ACC up to 0.9135 on Kidney, 0.8837 on PBMC, 0.7615 on BMMC) across various tissues.
  • Batch Correction: Outperformed established methods like Harmony, LIGER, and scANVI on biological conservation and batch integration metrics, showing better preservation of cell-specific variations while removing batch effects. For instance, it achieved the highest NMI (0.5681) and ASWb (0.9069) on the Kidney dataset.
  • Gene Expression Prediction: Showed superior performance compared to the Gene Activity method, indicating a more accurate modeling of peak-to-gene correlations. For example, it achieved lower MSE values (e.g., 6.7642 on PBMC) than Gene Activity (10.2098).

Limitations and Future Work

  • Limitations: The paper acknowledges that while EpiFoundation effectively handles scATAC-seq data, its current architecture is primarily focused on peak-to-gene alignment. The authors also note that the model's performance can be impacted by the exclusion of batch or chromosome information, as shown in ablation studies.
  • Future Work: The authors plan to develop a more comprehensive single-cell foundation model that unifies multiple modalities, including scRNA-seq, scATAC-seq, and nucleotide sequences. They also aim to explore further applications of their model.

Summary updated

Abstract

Foundation models exhibit strong capabilities for downstream tasks by learning generalized representations through self-supervised pre-training on large datasets. While several foundation models have been developed for single-cell RNA-seq (scRNA-seq) data, there is still a lack of models specifically tailored for single-cell ATAC-seq (scATAC-seq), which measures epigenetic information in individual cells. The principal challenge in developing such a model lies in the vast number of scATAC peaks and the significant sparsity of the data, which complicates the formulation of peak-to-peak correlations. To address this challenge, we introduce <b>EpiFoundation</b>, a foundation model for learning cell representations from the high-dimensional and sparse space of peaks. EpiFoundation relies on an innovative cross-modality pre-training procedure with two key technical innovations. First, EpiFoundation exclusively processes the non-zero peak set, thereby enhancing the density of cell-specific information within the input data. Second, EpiFoundation utilizes dense gene expression information to supervise the pre-training process, aligning peak-to-gene correlations. EpiFoundation can handle various types of downstream tasks, including cell-type annotation, batch correction, and gene expression prediction. To train and validate EpiFoundation, we curated <b>MiniAtlas</b>, a dataset of 100,000+ single cells with paired scRNA-seq and scATAC-seq data, along with diverse test sets spanning various tissues and cell types for robust evaluation. EpiFoundation demonstrates state-of-the-art performance across multiple tissues and diverse downstream tasks.
BibTeX
@misc{wu2025epifoundation,
  author = {Jun-Cheng Wu and Changxin Wan and Zhicheng Ji and Yuyin Zhou and Wenpin Hou},
  title = {{EpiFoundation: A Foundation Model for Single-Cell ATAC-seq via Peak-to-Gene Alignment}},
  year = {2025},
  doi = {10.1101/2025.02.05.636688},
  howpublished = {bioRxiv},
}
Figure