M^2LLM: Multi-view Molecular Representation Learning with Large Language Models

Jiaxin Ju, Yizhen Zheng, Huan Yee Koh, Can Wang, Shirui Pan

Abstract

Predicting molecular properties is a fundamental challenge in drug discovery and chemical research. While traditional methods rely on fixed fingerprints or data-driven Graph Neural Networks (GNNs), they often lack the chemical intuition and contextual flexibility required for complex tasks. This paper introduces M²LLM, a novel framework that transforms Large Language Models (LLMs) into sophisticated molecular encoders. By integrating three distinct perspectives—molecular structure, task-specific context, and expert-level scientific rules—M²LLM generates a holistic representation that achieves state-of-the-art performance across diverse chemical benchmarks, significantly reducing prediction errors in regression tasks.

Introduction

The cornerstone of drug discovery lies in accurately predicting properties such as solubility, toxicity, and blood-brain barrier penetration. Historically, researchers have utilized molecular fingerprints (rigid mathematical descriptions) or GNNs (which learn patterns directly from molecular graphs). However, fingerprints are non-adaptive, and GNNs are often "knowledge-poor," lacking the deep chemical reasoning found in scientific literature.

M²LLM addresses these limitations by leveraging the latent scientific knowledge within LLMs. Instead of treating a molecule as a simple text string (SMILES), M²LLM prompts the model to reason about the molecule through multiple "views," bridging the gap between symbolic chemical rules and deep learning.

Methodology

The M²LLM framework consists of two primary modules that feed into a dynamic fusion layer:

1. Molecular Embedding Generation

This module extracts semantic information from the LLM via two sub-views:

  • Molecular Structure View: The model is prompted with "insight questions" regarding the molecule's 3D geometry, intermolecular forces, and chemical equilibrium, forcing the LLM to activate its latent structural knowledge.
  • Molecular Task View: The LLM is provided with the specific downstream goal (e.g., predicting HIV inhibition) to generate an embedding tailored to that specific context.

2. Molecular Feature Curation

In the Molecular Rules View, the LLM acts as an expert chemist. It generates rules based on both general scientific principles (like Lipinski's Rule of Five) and patterns observed in the training data. These natural language rules are then converted into executable code via a Rule-to-Feature Code Translator, which extracts numerical features directly from the input molecules.

The M²LLM framework architecture, illustrating the parallel processes of Molecular Embedding Generation and Molecular Feature Curation followed by Multi-view Fusion.

Multi-view Fusion

A learnable fusion module dynamically weights the contributions of the Structure, Task, and Rule views. This ensures that the final representation is optimized for the specific requirements of the downstream task.

Experimental Results

M²LLM was evaluated on 8 datasets from the MoleculeNet benchmark, covering 34 subtasks using rigorous scaffold splitting. The results demonstrate a significant leap over traditional baselines:

  • Classification Excellence: On the ClinTox dataset, M²LLM achieved a near-perfect 99.5% ROC-AUC.
  • Regression Breakthroughs: The model reduced the Root Mean Square Error (RMSE) on the FreeSolv dataset by 53.3% compared to the strongest previous baseline.
  • Robust Generalization: The multi-view approach consistently outperformed "SMILES-only" LLM inputs, proving that structured reasoning is essential for chemical tasks.

Key Insights: The Importance of Views

A critical finding of this research is that the importance of each "view" is highly task-dependent. Analysis shows that the Structure View is often the primary driver for classification tasks, whereas the Rule and Task Views become significantly more influential in regression tasks, where continuous physical properties must be modeled.

Relative importance of Structure, Rule, and Task views across various datasets and LLM backbones, highlighting task-dependent feature weighting.

Conclusion and Future Work

M²LLM demonstrates that LLMs can serve as more than just text processors; they can function as expert feature engineers and reasoners in the chemical domain. While the model faced challenges with specific datasets like BACE—likely due to label ambiguity in binary potency thresholds—its overall performance sets a new standard for molecular representation learning.

Future research will focus on scaling M²LLM to larger datasets, incorporating direct 3D geometric and spectral data into the prompts, and further refining the Rule-to-Feature translator to handle increasingly complex chemical logic.

Summary updated

Abstract

Accurate molecular property prediction is a critical challenge with wide-ranging applications in chemistry, materials science, and drug discovery. Molecular representation methods, including fingerprints and graph neural networks (GNNs), achieve state-of-the-art results by effectively deriving features from molecular structures. However, these methods often overlook decades of accumulated semantic and contextual knowledge. Recent advancements in large language models (LLMs) demonstrate remarkable reasoning abilities and prior knowledge across scientific domains, leading us to hypothesize that LLMs can generate rich molecular representations when guided to reason in multiple perspectives. To address these gaps, we propose M^2LLM, a multi-view framework that integrates three perspectives: the molecular structure view, the molecular task view, and the molecular rules view. These views are fused dynamically to adapt to task requirements, and experiments demonstrate that M^2LLM achieves state-of-the-art performance on multiple benchmarks across classification and regression tasks. Moreover, we demonstrate that representation derived from LLM achieves exceptional performance by leveraging two core functionalities: the generation of molecular embeddings through their encoding capabilities and the curation of molecular features through advanced reasoning processes.
BibTeX
@inproceedings{ju2025m,
  author = {Jiaxin Ju and Yizhen Zheng and Huan Yee Koh and Can Wang and Shirui Pan},
  title = {{M^2LLM: Multi-view Molecular Representation Learning with Large Language Models}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/827},
}
Figure