MOFormer: navigating the antimicrobial peptide design space with Pareto-based multi-objective transformer

Li Wang, Xiangzheng Fu, Jiahao Yang, Xinyi Zhang, Xiucai Ye, Tetsuya Sakurai, Xiangxiang Zeng, Yiping LiuBriefings in Bioinformatics 2025 · DOI

TL;DR

This paper introduces MOFormer, a novel deep learning framework that uses a Transformer architecture to design antimicrobial peptides (AMPs) with multiple desirable properties simultaneously. By employing a Pareto-based multi-objective optimization strategy and a structured latent space, MOFormer effectively navigates the complex trade-offs between antimicrobial activity and safety (hemolysis and toxicity), outperforming existing methods in generating promising AMP candidates.

What Problem Does This Paper Address?

The urgent need for new antibiotics, driven by the rise of antimicrobial resistance, has highlighted antimicrobial peptides (AMPs) as promising alternatives. AMPs are naturally occurring molecules with potent antimicrobial activity. However, designing effective AMPs is a significant challenge because desirable properties often conflict. Specifically, AMPs that are highly effective against bacteria (high Minimum Inhibitory Concentration, MIC) can also cause hemolysis (HEMO) or general toxicity (TOXI) to host cells, limiting their clinical application. Traditional deep learning methods often struggle to optimize these multiple, conflicting objectives simultaneously, leading to a high attrition rate in drug development. This paper aims to address this gap by developing a model that can efficiently design AMPs that balance antimicrobial efficacy with reduced toxicity.

Methods

The core of this work is the MOFormer framework, a multi-objective Transformer-based model. It operates in two phases:

  1. Training and Optimization:
    • Multi-Fusion Descriptors: MOFormer goes beyond just sequence data by incorporating auxiliary descriptors that capture physical, chemical, and structural properties of AMPs. These include Amino Acid Composition (AAC) and Composition, Transformation, and Distribution (CTDC) features.
  2. Dimensionality Reduction: Principal Component Analysis (PCA) is used to reduce the dimensionality of these descriptors, retaining the most informative features.
  3. Transformer Architecture: A Transformer encoder is employed to process the sequence and compressed descriptors.
  4. Regularization: To prevent issues like posterior collapse and ensure a structured latent space, MOFormer uses KL divergence annealing and a novel Gaussian dropout strategy.
  5. Multi-Objective Loss Function: The training objective combines reconstruction loss, conditional loss, KL divergence, and regularization terms to guide the model towards optimizing multiple properties.
  6. Sampling and Screening:
    • Conditional Sampling: The trained model generates candidate AMP sequences based on desired property targets.
  7. Hierarchical Screening: A non-dominated sorting algorithm is used to rank candidates based on Pareto fronts, identifying solutions that offer the best trade-offs.
  8. Surrogate Models: Fine-tuned proxy models (e.g., MIC predictors and HEMO classifiers based on Prot-BERT-BFD) are used for rapid evaluation and ranking.
  9. Feedback Loop: Top-performing candidates are iteratively fed back into the training dataset to further refine the model's performance.

Datasets

The study utilized several datasets:

  • Training Data: A compiled dataset of 4096 AMP sequences, sourced from existing studies, filtered for low MIC against E. coli and low HEMO probability.
  • Additional Datasets:
    • 4547 AMP samples with antimicrobial activities.
  • 1006 balanced samples for HEMO and non-HEMO activity.
  • 201 TOXI samples with an equal number of non-TOXI controls.
  • Pre-trained Models: The Prot-BERT-BFD model was used for fine-tuning regression and classification models.

Key Insights and Ideas

  • The Power of Multi-Fusion Descriptors: Incorporating diverse physicochemical and structural descriptors alongside sequence information significantly improves the model's ability to understand and generate AMPs with desired properties.
  • Pareto Fronts for Trade-off Navigation: Framing AMP design as a multi-objective problem and using Pareto fronts is crucial for identifying peptides that strike an optimal balance between efficacy and safety, rather than just optimizing for a single metric.
  • Structured Latent Space is Key: Regularization techniques are vital for creating a well-behaved latent space that allows for precise sampling of desirable AMP candidates and prevents issues like posterior collapse.
  • Iterative Feedback Improves Performance: Re-incorporating high-performing candidates back into the training loop (feedback mechanism) demonstrably enhances the model's ability to generate AMPs with superior multi-objective profiles.
  • Interpretability through Attention: Analyzing attention scores within the Transformer architecture provides valuable insights into residue interactions, helping to understand why certain sequences are effective.

Key Results

  • Optimal Hypervolume (HV): MOFormer achieved the highest Hypervolume (HV) score across various multi-objective spaces, indicating superior coverage of the Pareto front and better balancing of conflicting objectives compared to six state-of-the-art methods.
  • Performance Comparison: In terms of interval satisfaction for MIC (<2) and HEMO (<0.5), MOFormer and its variants (w/o D, w/o F) showed strong performance, often outperforming other methods, especially in the combined optimization score. For example, MOFormer achieved an HV of 0.976 for MIC and HEMO.
  • TOXI Optimization: When extended to a tri-objective scenario (MIC, HEMO, TOXI), MOFormer also demonstrated excellent performance, with MOFormer and MOFormer (w/o F) outperforming others, though TOXI data scarcity presented a challenge.
  • Structural Validation: AlphaFold predictions for selected candidates showed high predicted local distance difference test (pLDDT) scores (70-87%), indicating reliable structural predictions. Molecular visualization revealed that most candidates adopted random coil structures, promoting dynamic interactions, while one exhibited a stable alpha-helical structure.
  • Novelty: The generated peptides showed significant structural novelty compared to the training data, with an average minimum Levenshtein distance of 3.672.

Limitations and Future Work

  • TOXI Data Scarcity: The authors acknowledge that TOXI data is notoriously scarce and challenging to gather, which impacts the validation process for models optimizing this property.
  • Mode Collapse in Extreme Optimization: When simultaneously minimizing MIC and TOXI, multi-objective optimization models can converge to a limited set of optimal regions, potentially sacrificing diversity.
  • Future Directions:
    • Enriching Training Data: Future work will focus on increasing the availability of extreme-case training data.
  • Advanced Architectures: Exploring classifier-free diffusion models is suggested to better preserve diversity under stringent constraints by integrating conditional and unconditional generative pathways.
  • Experimental Validation: The authors plan to conduct extensive experimental validation, including peptide synthesis, purification, in vitro assays (MIC, HEMO, TOXI), and molecular dynamics simulations to assess membrane-binding mechanisms and structural stability.

Summary updated

BibTeX
@article{wang2025moformer,
  author = {Li Wang and Xiangzheng Fu and Jiahao Yang and Xinyi Zhang and Xiucai Ye and Tetsuya Sakurai and Xiangxiang Zeng and Yiping Liu},
  title = {{MOFormer: navigating the antimicrobial peptide design space with Pareto-based multi-objective transformer}},
  journal = {Briefings in Bioinformatics},
  year = {2025},
  doi = {10.1093/bib/bbaf376},
}
Figure