Exploring Single-Cell Data with Deep Multitasking Neural Networks

Matthew Amodio, David van Dijk, Krishnan Srinivasan, William S. Chen, Hussein Mohsen, Kevin R. Moon, Allison M. Campbell, Yujiao Zhao, Xiaomei Wang, Manjunatha M. Venkataswamy, Anita Desai, Vasanthapuram Ravi, Priti Kumar, Ruth R. Montgomery, Guy Wolf, Smita KrishnaswamybioRxiv 2017 · DOI

Abstract

Biomedical researchers are generating high-throughput, high-dimensional single-cell data at an unprecedented rate. As the costs of data generation decrease, experimental designs are increasingly incorporating hundreds of samples across different patients, conditions, and treatments. This shift presents significant computational challenges, specifically regarding scalability and the management of batch effects. Current analysis pipelines are often fragmented, utilizing disparate tools for denoising, batch correction, visualization, and clustering, which can lead to inconsistent results. We present SAUCIE (Sparse Autoencoder for Unsupervised Clustering, Imputation, and Embedding), a deep multitasking neural network designed to perform these tasks simultaneously on a unified data representation. By employing novel regularizations, SAUCIE renders the hidden layers of a neural network interpretable, allowing for efficient processing of datasets containing millions of cells. We demonstrate SAUCIE’s capabilities on an 11-million-cell dataset of T cells from dengue patients, identifying distinct immune signatures and stratifying patient responses.

Introduction: The Challenge of Scale and Fragmentation

Single-cell technologies, such as mass cytometry (CyTOF) and single-cell RNA sequencing (scRNA-seq), allow for the molecular profiling of individual cells, providing deep insights into cellular heterogeneity. However, the field faces two primary bottlenecks:

  1. Scalability: Modern datasets now encompass millions of cells across hundreds of patients. Traditional algorithms, such as t-SNE or Phenograph, often struggle with the computational complexity required to process data at this scale.
  2. Pipeline Fragmentation: A typical analysis involves a "Frankenstein" pipeline—using one tool for denoising, another for batch correction, a third for visualization, and a fourth for clustering. Because these tools operate on different mathematical representations, the final results can be mismatched or inconsistent.

SAUCIE addresses these issues by leveraging the parallelization and scalability of deep learning to perform all these tasks within a single, synchronized architecture.

The SAUCIE Architecture: Multitasking via Regularization

SAUCIE is built on a deep autoencoder architecture. While standard autoencoders are often viewed as "black boxes," SAUCIE utilizes specific architectural constraints and novel regularizations to make its hidden layers interpretable and task-oriented.

Information Dimension (ID) Regularization

To perform unsupervised clustering, SAUCIE introduces Information Dimension (ID) regularization. This penalty is applied to a hidden layer to encourage "binarizable" activations. By forcing cells into discrete binary codes, the network effectively performs an analog-to-digital conversion, where each unique code represents a cluster. This allows the network to learn the underlying discrete structure of the data without requiring a pre-specified number of clusters.

Batch Correction via MMD

To handle technical variations between experimental runs (batch effects), SAUCIE employs a Maximal Mean Discrepancy (MMD) penalty. This forces the internal representations of different batches to follow the same distribution while preserving the underlying biological signals. By treating batch correction and clustering as joint optimization problems, SAUCIE ensures that biological variation is not accidentally erased during the normalization process.

Visualization and Imputation

SAUCIE includes a dedicated two-dimensional bottleneck layer that provides a direct visualization of the data manifold, similar to PCA or UMAP but learned non-linearly. Simultaneously, the reconstruction loss of the autoencoder naturally denoises the data and imputes missing values, which is particularly beneficial for addressing "dropout" events in scRNA-seq data.

Results and Performance

Massive Scalability

SAUCIE’s use of stochastic gradient descent allows it to process data in small "mini-batches," making it highly memory-efficient. In benchmarks, SAUCIE processed a massive 11-million-cell Dengue dataset in just 45 minutes. In comparison, standard metaclustering approaches required over 12 hours. Furthermore, SAUCIE demonstrated a 100x speedup over Nearest Neighbor Completion and a 600x speedup over scImpute for imputation tasks.

High-Fidelity Visualization and Batch Correction

On both synthetic and real-world datasets, SAUCIE achieved superior "mixing scores," indicating better blending of batches with less distortion of the original biological structure compared to methods like MNN or CCA. In visualization tasks, SAUCIE achieved a high average precision-recall AUC (0.9342), outperforming traditional methods like PCA, t-SNE, and PHATE.

Biological Discovery in Dengue Infection

Applying SAUCIE to a 180-sample dataset of T cells from dengue patients in India, the researchers were able to identify specific signatures of acute infection. SAUCIE successfully identified an enrichment of CD4-CD8- γδ T cells and cells involved in Type I interferon signaling. These signatures allowed for the creation of a "patient manifold," stratifying immune responses based on single-cell measurements and providing a high-level view of how different patients react to the virus.

Conclusion

SAUCIE represents a significant step forward in single-cell analysis by providing a unified, scalable, and interpretable framework for high-dimensional data. By moving away from fragmented pipelines and toward integrated multitasking neural networks, researchers can process millions of cells with higher consistency and speed. This "glass box" approach to deep learning opens new doors for exploring complex biological systems and identifying clinical signatures in large-scale patient cohorts.

Summary updated

Abstract

Abstract Biomedical researchers are generating high-throughput, high-dimensional single-cell data at a staggering rate. As costs of data generation decrease, experimental design is moving towards measurement of many different single-cell samples in the same dataset. These samples can correspond to different patients, conditions, or treatments. While scalability of methods to datasets of these sizes is a challenge on its own, dealing with large-scale experimental design presents a whole new set of problems, including batch effects and sample comparison issues. Currently, there are no computational tools that can both handle large amounts of data in a scalable manner (many cells) and at the same time deal with many samples (many patients or conditions). Moreover, data analysis currently involves the use of different tools that each operate on their own data representation, not guaranteeing a synchronized analysis pipeline. For instance, data visualization methods can be disjoint and mismatched with the clustering method. For this purpose, we present SAUCIE, a deep neural network that leverages the high degree of parallelization and scalability offered by neural networks, as well as the deep representation of data that can be learned by them to perform many single-cell data analysis tasks, all on a unified representation. A well-known limitation of neural networks is their interpretability. Our key contribution here are newly formulated regularizations (penalties) that render features learned in hidden layers of the neural network interpretable. When large multi-patient datasets are fed into SAUCIE, the various hidden layers contain denoised and batch-corrected data, a low dimensional visualization, unsupervised clustering, as well as other information that can be used to explore the data. We show this capability by analyzing a newly generated 180-sample dataset consisting of T cells from dengue patients in India, measured with mass cytometry. We show that SAUCIE, for the first time, can batch correct and process this 11-million cell data to identify cluster-based signatures of acute dengue infection and create a patient manifold, stratifying immune response to dengue on the basis of single-cell measurements.
BibTeX
@misc{amodio2017exploring,
  author = {Matthew Amodio and David van Dijk and Krishnan Srinivasan and William S. Chen and Hussein Mohsen and Kevin R. Moon and Allison M. Campbell and Yujiao Zhao and Xiaomei Wang and Manjunatha M. Venkataswamy and Anita Desai and Vasanthapuram Ravi and Priti Kumar and Ruth R. Montgomery and Guy Wolf and Smita Krishnaswamy},
  title = {{Exploring Single-Cell Data with Deep Multitasking Neural Networks}},
  year = {2017},
  doi = {10.1101/237065},
  howpublished = {bioRxiv},
}
Figure