Deep generative models for T cell receptor protein sequences

Kristian Davidsen, Branden J Olson, William S. DeWitt, Jean Feng, Elias Harkins, Philip Bradley, F. A. MatseneLife 2019 · DOI

TL;DR

This paper introduces Variational Autoencoders (VAEs) as a powerful new tool for modeling T cell receptor (TCR) repertoires. These deep learning models can accurately predict TCR frequencies, learn the complex rules of TCR gene recombination, and generate realistic TCR sequences, outperforming traditional methods.

What Problem Does This Paper Address?

The immune system's T cells use T cell receptors (TCRs) to recognize foreign invaders. Each TCR is built through a highly variable process called V(D)J recombination, creating a vast repertoire of unique TCRs. Understanding the distribution of these TCR sequences is crucial for various applications, such as identifying T cells that respond to specific diseases or vaccines, and understanding how TCRs are shared between individuals. Traditional methods for modeling these repertoires often rely on explicit, mechanistic models of V(D)J recombination and selection. However, these models can be complex to build and may not fully capture the intricate biological processes. This paper explores whether deep generative models, specifically VAEs, can learn these distributions directly from data without explicit biological priors, and if they can achieve comparable or better performance.

Methods

The core of this work involves applying and adapting Variational Autoencoder (VAE) models to TCR β protein sequences. VAEs are a type of deep generative model that learn a compressed, lower-dimensional "latent space" representation of data.

  • VAE Architecture: The authors use VAEs with neural networks for both the encoder (mapping TCR sequences to the latent space) and the decoder (mapping latent space representations back to TCR sequences). They experiment with two VAE variants:
  • Basic VAE: A simpler model trained on TCR sequences without explicit knowledge of germline gene information.
  • Count-match VAE: A more complex model that incorporates information about germline V and J gene sequences and their match to the CDR3 region.
  • Loss Function: The VAEs are trained using a loss function that balances two objectives: accurately reconstructing the input TCR sequences and ensuring the learned latent space distribution is close to a prior distribution (typically a multivariate normal).
  • Comparison Model: For benchmarking, they use a sophisticated recombination model called OLGA, combined with a simplified selection model (OLGA.Q), which represents a more traditional, mechanistic approach.

Datasets

The study utilizes several publicly available TCR repertoire datasets for training and evaluation:

  • De Neuter et al. (2019) dataset: Used extensively for evaluating VDJ recombination learning, generalization, and latent space structure. This dataset consists of TCR β sequences from 33 subjects.
  • Emerson et al. (2017) dataset: Used for cohort frequency estimation.
  • DeWitt et al. (2018) dataset: Used for model design and parameter tuning.
  • Emerson et al. (2013) dataset: Used for supplementary analysis of the effect of the $\beta$ parameter.

Key Insights and Ideas

  • Data-Driven Learning: The paper strongly suggests that deep learning models like VAEs can learn the complex rules of TCR generation directly from large datasets, potentially rivaling or surpassing models that explicitly encode biological knowledge.
  • Generalization Power: VAEs demonstrate a remarkable ability to generalize to unseen TCR sequences, indicating they are learning underlying biological principles rather than simply memorizing training data.
  • Latent Space Structure: The latent space learned by the VAEs shows clear organization based on biological features like V and J gene usage, and even CDR3 length, suggesting these models capture meaningful biological relationships.
  • Simplicity and Accessibility: The authors highlight that even a basic VAE implementation can be surprisingly concise (around 100 lines of Python code), making these powerful models more accessible to researchers.

Key Results

  • Cohort Frequency Estimation: VAE models achieved accurate cohort frequency estimation, outperforming the OLGA.Q model, especially as the number of training sequences increased. R² values for the VAE were higher than for OLGA.Q on test sets.
  • VDJ Recombination Learning: VAE-generated sequences showed distributions of VDJ recombination probabilities (as assessed by OLGA.Q) that closely matched those of real experimental repertoires, indicating they learned the rules of recombination. The basic VAE performed nearly as well as the count-match VAE, suggesting explicit germline information wasn't strictly necessary.
  • Generalization to Unseen Sequences: VAEs assigned probabilities to unseen sequences that were consistent with real TCRs, and importantly, they could distinguish between real sequences and those generated by the OLGA.Q model.
  • Latent Space Insights: Principal Component Analysis (PCA) on the VAE's latent space revealed clear clustering based on V and J gene identity, and further analysis showed CDR3 length as another important determinant.

Limitations and Future Work

  • Interpretability: A significant drawback noted is the inherent lack of interpretability of neural network models. While structure was found in the latent space, understanding the precise function of neural network weights remains challenging.
  • Nucleotide Level: The current models operate on amino acid sequences and cannot directly shed light on the VDJ recombination process, which occurs at the nucleotide level.
  • Computational Cost: Calculating PVAE using importance sampling is more computationally expensive than calculating OLGA.Q probabilities.
  • Future Directions:
    • Further unraveling the latent space structure by comparing repertoires and sequences labeled with external characteristics.
  • Developing pre-trained models to allow biologists to evaluate the probability of TCRs in different contexts.
  • Extending the methods to model TCR α chains and jointly modeling both α and β chains for a more complete picture.
  • Investigating the application of these models to other immune receptor repertoires, such as B cell receptors (BCRs).

Summary updated

Abstract

Probabilistic models of adaptive immune repertoire sequence distributions can be used to infer the expansion of immune cells in response to stimulus, differentiate genetic from environmental factors that determine repertoire sharing, and evaluate the suitability of various target immune sequences for stimulation via vaccination. Classically, these models are defined in terms of a probabilistic V(D)J recombination model which is sometimes combined with a selection model. In this paper we take a different approach, fitting variational autoencoder (VAE) models parameterized by deep neural networks to T cell receptor (TCR) repertoires. We show that simple VAE models can perform accurate cohort frequency estimation, learn the rules of VDJ recombination, and generalize well to unseen sequences. Further, we demonstrate that VAE-like models can distinguish between real sequences and sequences generated according to a recombination-selection model, and that many characteristics of VAE-generated sequences are similar to those of real sequences.
BibTeX
@inproceedings{davidsen2019deep,
  author = {Kristian Davidsen and Branden J Olson and William S. DeWitt and Jean Feng and Elias Harkins and Philip Bradley and F. A. Matsen},
  title = {{Deep generative models for T cell receptor protein sequences}},
  booktitle = {eLife},
  year = {2019},
  doi = {10.7554/elife.46935},
}
Figure