Disentangled Wasserstein Autoencoder for T-Cell Receptor Engineering

Tianxiao Li, Hongyu Guo, Filippo Grazioli, Mark Gerstein, Martin Renqiang Min

Introduction

T-cell receptors (TCRs) act as the "eyes" of the adaptive immune system, responsible for recognizing specific peptide antigens from viruses or tumors. Engineering TCRs to bind new targets is a highly sought-after goal for immunotherapy, but it remains notoriously difficult.

The primary challenge lies in the "structure-function" dichotomy: engineers must modify the specific residues that handle binding (the function) without accidentally breaking the overall protein fold (the structure). Traditional methods rely heavily on domain expertise or computationally expensive "black-box" simulations. Recent deep learning approaches often use reinforcement learning or iterative mutations, which are slow and lack interpretability regarding what the model is actually changing.

To address this, researchers have introduced TCR-dWAE, a new generative framework that treats protein engineering as a "style transfer" problem, disentangling a TCR's functional binding properties from its structural backbone. This approach allows for rapid, "one-pass" sequence editing that is significantly more effective at generating valid, high-affinity binders than traditional iterative methods.

Methodology: TCR-dWAE

The core idea behind the Disentangled Wasserstein Autoencoder (TCR-dWAE) is to map a protein sequence into two separate, independent latent spaces:

  1. Functional Embedding ($z_f$): Captures patterns related to peptide binding.
  2. Structural Embedding ($z_s$): Captures the generic structural backbone.

To achieve this disentanglement, the model uses a Wasserstein Autoencoder (WAE) framework with two key additions. First, an Auxiliary Classifier is trained to predict binding based only on $z_f$, forcing that embedding to capture functional information. Second, a Maximum Mean Discrepancy (MMD) Loss acts as a regularization term, forcing the joint distribution of $z_f$ and $z_s$ to match an isotropic Gaussian prior. This ensures the two embeddings are independent and makes the latent space easier to sample.

The TCR-dWAE framework (B) and the sequence engineering method (C).

For engineering, the model performs a process akin to "style transfer." It takes the structural embedding ($z_s$) from a template TCR and combines it with a functional embedding ($z_f$) known to bind a target peptide. A decoder then reconstructs the new, engineered sequence in a single pass.

Key Insights

  • Protein Design as Style Transfer: By framing the problem this way, the authors move away from slow, iterative "guess-and-check" mutations toward a more direct, interpretable manipulation of protein properties.
  • Theoretical Disentanglement: The authors use "Variation of Information" to prove that their objective function minimizes the mutual information between the functional and structural embeddings, providing a theoretical guarantee of disentanglement.
  • Interpretability: Attention maps demonstrate that the functional encoder focuses on the hypervariable regions (like CDR3β), while the structural encoder focuses on conserved regions, confirming the model learns biologically relevant features.

Results and Performance

The model was trained and evaluated on several major TCR-peptide interaction datasets (VDJDB, McPAS-TCR, NetTCR) and scored using ERGO, a state-of-the-art TCR binding predictor. TCR-dWAE was compared against several baselines, including standard VAEs and iterative search methods like Genetic algorithms and Monte Carlo Tree Search.

Performance comparison across different metrics and models.
  • Superior Quality: TCR-dWAE achieved a 23% success rate (the percentage of generated sequences that were both structurally valid and positive binders). This significantly outperformed the VAE baseline (16%) and crushed iterative mutation methods (which hovered around 2%).
  • Massive Speedup: Because it generates sequences in a single pass rather than through thousands of iterations, TCR-dWAE is 10 times faster than the baseline models.
  • Structural Integrity: The model successfully maintained the "backbone" of the template TCRs, evidenced by high validity scores and consensus motifs matching known natural TCRs.

Limitations and Future Work

Currently, the model only looks at the CDR3β region. Because real TCRs are heterodimers (alpha and beta chains), future work will need to incorporate both chains for higher accuracy. Additionally, the engineering process relies on having a "positive" $z_f$ to swap in, which requires a well-trained auxiliary classifier and high-quality labeled data. Finally, while the in silico results are strong, real-world wet-lab validation remains the necessary next step to confirm these engineered TCRs bind as predicted.

Summary updated

Abstract

In protein biophysics, the separation between the functionally important residues (forming the active site or binding surface) and those that create the overall structure (the fold) is a well-established and fundamental concept. Identifying and modifying those functional sites is critical for protein engineering but computationally non-trivial, and requires significant domain knowledge. To automate this process from a data-driven perspective, we propose a disentangled Wasserstein autoencoder with an auxiliary classifier, which isolates the function-related patterns from the rest with theoretical guarantees. This enables one-pass protein sequence editing and improves the understanding of the resulting sequences and editing actions involved. To demonstrate its effectiveness, we apply it to T-cell receptors (TCRs), a well-studied structure-function case. We show that our method can be used to alter the function of TCRs without changing the structural backbone, outperforming several competing methods in generation quality and efficiency, and requiring only 10\% of the running time needed by baseline models. To our knowledge, this is the first approach that utilizes disentangled representations for TCR engineering.
BibTeX
@inproceedings{li2023disentangled,
  author = {Tianxiao Li and Hongyu Guo and Filippo Grazioli and Mark Gerstein and Martin Renqiang Min},
  title = {{Disentangled Wasserstein Autoencoder for T-Cell Receptor Engineering}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=Eb74zfBkWa},
}
Figure