LANTERN: Leveraging Large Language Models and Transformers for Enhanced Molecular Interactions

Cong Nga Ha, Phuong Viet Pham, Truong Son HybioRxiv 2025 · DOI

TL;DR

This paper introduces LANTERN, a novel deep learning framework that combines Large Language Models (LLMs) with Transformer architectures to predict molecular interactions like drug-target, protein-protein, and drug-drug interactions. By leveraging powerful LLM-generated embeddings and a Transformer's ability to capture complex relationships, LANTERN achieves state-of-the-art results across multiple benchmarks, offering a more scalable and accurate approach without relying on 3D structural data.

What Problem Does This Paper Address?

Understanding how molecules interact is fundamental to drug discovery, systems biology, and developing new treatments. However, the sheer scale of chemical and biological possibilities makes accurately predicting these interactions a significant challenge. Existing computational methods often struggle with scalability and can miss the subtle, intricate biochemical relationships that govern these interactions. Many approaches also rely on 3D structural data, which is not always available or can be computationally expensive to obtain and process. This paper aims to bridge this gap by developing a more efficient, accurate, and versatile framework for predicting diverse molecular interactions.

Methods

LANTERN's core innovation lies in its hybrid architecture:

  • LLM Embeddings: It first uses pre-trained Large Language Models (LLMs) to generate rich, context-aware "embeddings" for drugs (represented by SMILES strings) and proteins (represented by amino acid sequences). These embeddings capture essential biochemical and structural information from the sequences themselves.
  • Transformer Fusion: These LLM embeddings are then fed into a Transformer encoder. Transformers, known for their success in natural language processing, excel at modeling long-range dependencies and complex relationships. In LANTERN, the Transformer's attention mechanism dynamically fuses the drug and protein embeddings, allowing it to learn intricate interaction patterns between them.
  • Classification: Finally, a classifier takes the Transformer's output to predict the probability of an interaction.

This approach bypasses the need for explicit 3D structural data, relying solely on sequence information. The framework is also designed to be adaptable, allowing for different LLMs and Transformer configurations depending on the specific interaction task.

Datasets

The study evaluated LANTERN on several established benchmark datasets:

  • Drug-Target Interaction (DTI): DAVIS, KIBA, and BioSNAP.
  • Drug-Drug Interaction (DDI): DeepDDI.
  • Protein-Protein Interaction (PPI): Yeast PPI.

These datasets cover a range of interaction types and complexities, allowing for a comprehensive assessment of LANTERN's performance and generalizability.

Key Insights and Ideas

  • LLMs as Feature Extractors: The paper highlights the power of LLMs in generating highly informative sequence-based embeddings for both drugs and proteins, effectively encoding complex biochemical information without explicit structural data.
  • Transformer's Superiority for Fusion: The Transformer architecture, particularly its self-attention mechanism, proves significantly more effective than traditional Multi-Layer Perceptrons (MLPs) for fusing drug and protein representations. This is attributed to its ability to dynamically weight features and model long-range dependencies.
  • Sequence-Only Approach is Viable: LANTERN demonstrates that accurate molecular interaction prediction can be achieved using only sequence data, making it more broadly applicable and less dependent on the availability of costly 3D structural information.
  • Versatility Across Interaction Types: The framework's success on DTI, PPI, and DDI tasks underscores its robustness and adaptability to different biological interaction prediction challenges.

Key Results

LANTERN achieved state-of-the-art (SOTA) performance on multiple benchmarks:

  • DTI: Achieved SOTA on the BioSNAP, DAVIS, and KIBA datasets, with AUROC scores reaching up to 0.995 and AUPRC scores up to 0.996. For example, on BioSNAP, it surpassed the next best model by a significant margin.
  • DDI: Demonstrated superior performance on the DeepDDI dataset, achieving an AUROC of 0.998 and the highest AUPRC of 0.995.
  • PPI: Showed competitive performance on the Yeast PPI dataset, with specific LLM combinations (e.g., ProtT5 and ProtBERT) achieving near-perfect sensitivity.
  • Transformer vs. MLP: Ablation studies confirmed that the Transformer-based fusion significantly outperformed MLP-based fusion, with the full Transformer model achieving AUROC of 0.9953 and AUPRC of 0.9961, compared to an MLP replacement yielding 0.8485 and 0.8559 respectively.

Limitations and Future Work

  • LLM Selection: While the paper explores LLM choices, the optimal LLM can vary by dataset and task. Future work could involve more sophisticated methods for selecting or combining LLMs.
  • Computational Cost: While more efficient than 3D-based methods, Transformer architectures can still be computationally intensive.
  • Future Directions: The authors suggest exploring self-supervised pretraining strategies to further enhance adaptability and investigating the use of multiple LLMs for the same data type (e.g., drugs) to leverage complementary knowledge. They also envision LANTERN as a foundation model for molecular interactions.

Summary updated

Abstract

Abstract Understanding molecular interactions such as Drug-Target Interaction (DTI), Protein-Protein Interaction (PPI), and Drug-Drug Interaction (DDI) is critical for advancing drug discovery and systems biology. However, existing methods often struggle with scalability due to the vast chemical and biological space and suffer from limited accuracy when capturing intricate biochemical relationships. To address these challenges, we introduce LANTERN (Leveraging Large LAN guage Models and T ransformers for E nhanced molecula R interactio N s), a novel deep learning framework that integrates Large Language Models (LLMs) with Transformer-based architectures to model molecular interactions more effectively. LANTERN generates high-quality, context-aware embeddings for drug and protein sequences, enabling richer feature representations and improving predictive accuracy. By leveraging a Transformer-based fusion mechanism, our framework enhances scalability by efficiently integrating diverse interaction data while maintaining computational feasibility. Experimental results demonstrate that LANTERN achieves state-of-the-art performance on multiple DTI and DDI benchmarks, significantly outperforming traditional deep learning approaches. Additionally, LANTERN exhibits competitive performance on challenging PPI tasks, underscoring its versatility across diverse molecular interaction domains. The proposed framework offers a robust and adaptable solution for modeling molecular interactions, efficiently handling a diverse range of molecular entities without the need for 3D structural data and making it a promising framework for foundation models in molecular interaction. Our findings highlight the transformative potential of combining LLM-based embeddings with Transformer architectures, setting a new standard for molecular interaction prediction. The source code and relevant documentation are available at: https://github.com/HySonLab/LANTERN .
BibTeX
@misc{ha2025lantern,
  author = {Cong Nga Ha and Phuong Viet Pham and Truong Son Hy},
  title = {{LANTERN: Leveraging Large Language Models and Transformers for Enhanced Molecular Interactions}},
  year = {2025},
  doi = {10.1101/2025.02.10.637522},
  howpublished = {bioRxiv},
}
Figure