Classical structural priors improve neural molecular representations in molecule language models

Donghyeok Lee, Sangkwon Park, Munyoung Chang, Sungroh YoonarXiv 2026 · DOI · arXiv

TL;DR

SAGA (Structure-Aware Guided Alignment) is a training-only regularizer that aligns neural molecular embeddings with classical chemical similarity (Tanimoto/fingerprints) using Centered Kernel Alignment (CKA). By enforcing this "classical" structural prior, molecule-language models become more chemically faithful, significantly improving their ability to retrieve structural analogues and generalize to new molecular scaffolds without adding any computational overhead at inference.

What Problem Does This Paper Address?

Modern drug discovery increasingly relies on multimodal "molecule-language models" (MLMs) that bridge chemical structures with natural language descriptions. These models are typically trained using contrastive learning, which teaches the model to pull "matching" structure-text pairs together and push "non-matching" pairs apart.

However, standard contrastive objectives are binary: they treat a structurally similar analogue (a "near-miss") exactly the same as a completely unrelated molecule. This ignores the fundamental chemical principle that molecules with similar substructures often share similar properties. Consequently, the latent spaces of these models don't necessarily reflect the graded continuum of molecular similarity that chemists have relied on for decades through tools like molecular fingerprints and Tanimoto similarity.

Methods

The authors introduce SAGA (Structure-Aware Guided Alignment), an auxiliary loss term added during pre-training.

The core innovation is the use of Centered Kernel Alignment (CKA) to perform "relational alignment." Instead of forcing a neural encoder to predict a specific fingerprint (point-wise alignment), SAGA ensures that the pairwise similarity patterns in the neural embedding space match the pairwise similarity patterns in a classical fingerprint space (like Morgan fingerprints).

Figure 1. Overview of SAGA. A CKA-based auxiliary loss aligns pairwise similarities among neural molecular embeddings with fingerprint-based structural similarities, transferring a classical chemoinformatics prior into molecule-language models without adding trainable parameters or requiring fingerprints at inference.

Key technical details include:

  • Training-Only: Fingerprints are only needed during training. At inference, the model uses only its learned neural representations.
  • Dimension Agnostic: CKA allows the comparison of two representation spaces even if they have different dimensions (e.g., a 768-d latent vector vs. a 2048-bit fingerprint).
  • Backbone Agnostic: The authors successfully integrated SAGA into both graph-based (MolCA) and SMILES-based (Atomas) model architectures.

Datasets

  • Pre-training: A curated corpus of 34,365 unique molecule-text pairs derived from PubChem.
  • Evaluation (Retrieval & Generation): ChEBI-20, a standard benchmark for molecule-to-text and text-to-molecule tasks.
  • Evaluation (Property Prediction): Eight classification tasks from MoleculeNet (e.g., BBBP, HIV, BACE) using scaffold splits to test generalization to entirely new chemical frameworks.
  • Diagnostic: 20 TDC ADMET datasets were used to validate that the chosen structural priors (fingerprints) actually correlate with real-world biological properties.

Key Insights and Ideas

  • The "Analogue Gap" in Contrastive Learning: The authors demonstrate that standard models struggle to organize their latent space by similarity. By adding SAGA, they bridge the gap between "exact match" and "unrelated," making the latent space more chemically intuitive.
  • Relational vs. Point-wise Alignment: The success of SAGA suggests that preserving the structure of the representation space is more important than forcing the model to reconstruct specific descriptors.
  • Cross-Modal Propagation: Interestingly, regularizing only the molecular encoder also improved the organization of the text encoder. Because the two are tied via a contrastive loss, making the molecular space more "chemically sensible" indirectly forced the text space to align better with chemical reality.
  • Model-Specific Priors: The researchers found that the "best" fingerprint depends on the encoder. Graph-based models (MolCA) benefited most from Morgan fingerprints, while SMILES-based models (Atomas) performed better with MACCS keys, which focus on predefined functional groups.

Key Results

SAGA delivered consistent improvements across multiple benchmarks:

  • Retrieval Gains: In Text-to-Molecule retrieval on ChEBI-20, SAGA improved R@1 scores significantly: MolCA (+4.34), Atomas-Base (+6.88), and Atomas-Large (+7.82).
  • Better "Near-Misses": In an "exclude-self" retrieval test (where the exact match is removed), SAGA-trained models retrieved molecules with much higher Tanimoto similarity to the target. For MolCA, the mean Tanimoto similarity of the top-3 hits jumped from 0.235 to 0.311.
  • Improved Generalization: On MoleculeNet property prediction, SAGA led to average ROC-AUC gains of up to 4.1% (MolCA). Crucially, these gains were observed on scaffold-split data, meaning the model is better at predicting properties for molecular structures it hasn't seen before.
  • Robust Generation: Molecular description and generation performance were either improved or preserved, indicating that the structural prior doesn't disrupt the model's ability to handle natural language.
SAGA improves the structural quality of retrieved neighbours, shifting non-exact hits toward closer structural analogues.

Limitations and Future Work

  • 2D vs. 3D: SAGA currently uses 2D topological fingerprints. While effective for many tasks, it may not capture nuances related to 3D conformation or specific protein-ligand binding mechanisms.
  • Hyperparameter Tuning: The effectiveness of SAGA is sensitive to the regularization weight ($\lambda$) and the CKA sample size ($n$), which may require careful tuning when applying it to new architectures.
  • Future Directions: The authors suggest exploring 3D-aware or target-aware relational priors to further enhance representations for specific drug discovery applications where spatial geometry is paramount.

Summary updated

Abstract

Classical molecular similarity is central to chemical information systems, yet molecule-language models trained on paired structure-text data do not necessarily organize molecular embeddings according to graded structural relationships. Here we introduce Structure-Aware Guided Alignment (SAGA), a training-only regularizer that uses Centered Kernel Alignment to align neural molecular representations with fingerprint-based Tanimoto similarity, without adding parameters or requiring fingerprints at inference. In MolCA and Atomas, SAGA improves molecule-text retrieval, shifts non-exact retrieval toward closer structural analogues, and increases average scaffold-split property-prediction performance, while molecular description and text-to-molecule generation are improved or preserved in single-run ablations. These results show that established chemoinformatics priors can make molecule-language representations more chemically faithful while retaining their multimodal utility.
BibTeX
@misc{lee2026classical,
  author = {Donghyeok Lee and Sangkwon Park and Munyoung Chang and Sungroh Yoon},
  title = {{Classical structural priors improve neural molecular representations in molecule language models}},
  year = {2026},
  eprint = {2611.21276},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21276},
}
Figure