TL;DR
Researchers have developed EpiFoundation, a novel foundation model specifically designed for single-cell ATAC-seq (scATAC-seq) data. By leveraging paired scRNA-seq data for supervision and focusing on non-zero peaks, EpiFoundation effectively learns cell representations and achieves state-of-the-art performance in crucial downstream tasks like cell type annotation, batch correction, and gene expression prediction.
What Problem Does This Paper Address?
Single-cell ATAC-seq (scATAC-seq) is a powerful technology for understanding gene regulation by measuring chromatin accessibility at the single-cell level. However, analyzing scATAC-seq data presents significant challenges: the data is extremely high-dimensional (millions of genomic "peaks") and highly sparse (many peaks are not accessible in any given cell). This sparsity makes it difficult to model relationships between peaks and to extract meaningful biological signals. While foundation models have revolutionized scRNA-seq analysis, a dedicated foundation model for scATAC-seq, capable of handling its unique data characteristics and enabling diverse downstream applications, was lacking.
Methods
EpiFoundation tackles these challenges with a novel cross-modality pre-training approach:
- Non-zero Peak Focus: Instead of processing all possible peaks, EpiFoundation focuses exclusively on the "non-zero" peaks (those showing accessibility in a given cell). This significantly reduces the input dimensionality and enhances the density of cell-specific information, making the model more efficient and effective.
- Peak-to-Gene Alignment: The model uses paired scRNA-seq data as a supervisory signal during pre-training. This "peak-to-gene alignment" forces the model to learn representations that link chromatin accessibility (ATAC-seq) to gene expression (RNA-seq), a crucial step for understanding gene regulation.
- Transformer Architecture: A Transformer-based architecture is employed to process the input embeddings of non-zero peaks and their corresponding chromosome information, generating a robust cell representation.
- Batch Effect Mitigation: Batch information is incorporated during pre-training to ensure that the learned cell representations are unbiased and can be effectively used for downstream tasks like batch correction.
Datasets
- MiniAtlas: The authors introduce a new benchmark dataset called MiniAtlas, comprising over 100,000 single cells with paired scATAC-seq and scRNA-seq measurements across 19 tissues and 56 cell types. This dataset is crucial for training and validating EpiFoundation.
- Test Sets: Diverse test sets from kidney, peripheral blood mononuclear cells (PBMC), and bone marrow mononuclear cells (BMMC) tissues, as well as an "ALLTissue" set, were used for robust evaluation.
Key Insights and Ideas
- Focusing on the "Active" Peaks is Sufficient: The paper's core hypothesis is that modeling only the accessible (non-zero) peaks is enough to capture essential cell-specific information for representation learning, overcoming the sparsity challenge.
- Cross-Modality Supervision is Key: Using gene expression data to guide the learning of ATAC-seq representations is a powerful strategy for linking epigenetic information to functional outcomes.
- Foundation Models for Epigenomics are Feasible and Powerful: EpiFoundation demonstrates that the foundation model paradigm, successful in scRNA-seq, can be effectively adapted to scATAC-seq, unlocking new possibilities for analyzing epigenetic data.
Key Results
EpiFoundation achieved state-of-the-art performance across multiple downstream tasks:
- Cell Type Annotation: Demonstrated high accuracy (e.g., ACC up to 0.9135 on Kidney, 0.8837 on PBMC, 0.7615 on BMMC) across various tissues.
- Batch Correction: Outperformed established methods like Harmony, LIGER, and scANVI on biological conservation and batch integration metrics, showing better preservation of cell-specific variations while removing batch effects. For instance, it achieved the highest NMI (0.5681) and ASWb (0.9069) on the Kidney dataset.
- Gene Expression Prediction: Showed superior performance compared to the Gene Activity method, indicating a more accurate modeling of peak-to-gene correlations. For example, it achieved lower MSE values (e.g., 6.7642 on PBMC) than Gene Activity (10.2098).
Limitations and Future Work
- Limitations: The paper acknowledges that while EpiFoundation effectively handles scATAC-seq data, its current architecture is primarily focused on peak-to-gene alignment. The authors also note that the model's performance can be impacted by the exclusion of batch or chromosome information, as shown in ablation studies.
- Future Work: The authors plan to develop a more comprehensive single-cell foundation model that unifies multiple modalities, including scRNA-seq, scATAC-seq, and nucleotide sequences. They also aim to explore further applications of their model.
Summary updated