Abstract
We present a three-stage framework for training deep learning models specializing in antibody sequence-structure co-design. We first pre-train a language model using millions of antibody sequence data. Then, we employ the learned representations to guide the training of a diffusion model for joint optimization over both sequence and structure of antibodies. During the final alignment stage, we optimize the model to favor antibodies with low repulsion and high attraction to the antigen binding site, enhancing the rationality and functionality of the designs. To mitigate conflicting energy preferences, we extend AbDPO (Antibody Direct Preference Optimization) to guide the model towards Pareto optimality under multiple energy-based alignment objectives. Furthermore, we adopt an iterative learning paradigm with temperature scaling, enabling the model to benefit from diverse online datasets without requiring additional data. In practice, our proposed methods achieve high stability and efficiency in producing a better Pareto front of antibody designs compared to top samples generated by baselines and previous alignment techniques. Through extensive experiments, we showcase the superior performance of our methods in generating nature-like antibodies with high binding affinity consistently.
TL;DR
AlignAb is a three-stage deep learning framework that co-designs antibody sequences and structures by aligning them with physical energy constraints. By extending Direct Preference Optimization (DPO) to handle multiple conflicting objectives, the model learns to generate "nature-like" antibodies that maximize antigen attraction while minimizing structural repulsion, achieving a superior Pareto-optimal balance compared to existing baselines.
The Challenge of Antibody Design
Designing effective antibodies—specifically the Complementarity Determining Regions (CDRs) that bind to antigens—is a cornerstone of modern drug discovery. However, two major hurdles persist:
- The Data Gap: While millions of antibody sequences are available, high-quality 3D structures are scarce (only a few thousand). This scarcity makes training structure-aware models exceptionally difficult.
- The Metric Gap: Most deep learning models are optimized for proxy metrics like Amino Acid Recovery (AAR) or Root Mean Square Deviation (RMSD). While these measure similarity to known data, they often ignore underlying physics. A model might generate a sequence that appears correct statistically but is physically "irrational," featuring massive atomic clashes (high repulsion) or weak binding (low attraction).
The AlignAb Framework
The authors propose a three-stage pipeline to bridge these gaps:
- Pre-training: A BERT-based language model is trained on 50 million antibody sequences from the Observed Antibody Space (OAS), allowing the model to internalize the "grammar" of antibody sequences.
- Transferring: Learned representations from the BERT model guide a diffusion model (based on DiffAb). This model is trained on the SAbDab structural database to jointly predict amino acid types, coordinates, and orientations.
- Pareto-Optimal Energy Alignment (POEA): This core innovation adapts Direct Preference Optimization (DPO) to align the model using biophysical energy measurements from the Rosetta software suite. Since "high attraction" and "low repulsion" are often conflicting goals, POEA uses a multi-objective approach to find a Pareto front—a set of optimal trade-offs—rather than a single biased solution.
To prevent "mode collapse" (where the model generates repetitive, low-energy sequences), the framework implements Iterative Online Alignment with Temperature Scaling. This allows the model to explore new designs and iteratively improve itself using online-generated data.
Datasets
- Pre-training: 50 million heavy chain sequences from the Observed Antibody Space (OAS).
- Transferring/Fine-tuning: Structural data from SAbDab, filtered for high-resolution complexes.
- Evaluation: The RAbD benchmark, consisting of 42 diverse antigen-antibody complexes.
Experimental Performance
AlignAb consistently outperformed state-of-the-art baselines (including MEAN, HERN, and DiffAb) across several physical metrics. As shown in the results, AlignAb produced antibodies with significantly lower total energy ($E_{total}$) and better structural rationality compared to baselines, which often produced designs with massive structural clashes.
Key Insights
- Physics over Proxies: Energy functions (Lennard-Jones potentials, hydrogen bonding) are more reliable indicators of antibody quality than AAR or RMSD. AlignAb designs maintain energy stability close to natural reference levels.
- The Necessity of Pareto Optimality: In antibody design, "stickiness" (attraction) often comes at the cost of "clashiness" (repulsion). AlignAb navigates these trade-offs more effectively than models that simply sum different energy terms into a single scalar.
- Nature-Likeness: The generated CDR-H3 regions showed sequence diversity (entropy) and structural rationality that closely mimicked natural antibodies, narrowing the gap between in silico design and in vitro reality.
Limitations and Future Work
Current limitations include the use of fixed weighting for reward models across all test data due to computational constraints. Future work will focus on tuning reward weightings for individual antigen-antibody complexes and scaling the framework to larger fine-tuning datasets. The authors also aim to extend the Energy Alignment philosophy to other molecular design tasks, such as small molecule drug discovery.
Summary updated