Abstract
Cross-modal text-molecule retrieval model aims to learn a shared feature space of the text and molecule modalities for accurate similarity calculation, which facilitates the rapid screening of molecules with specific properties and activities in drug design. However, previous works have two main defects. First, they are inadequate in capturing modality-shared features considering the significant gap between text sequences and molecule graphs. Second, they mainly rely on contrastive learning and adversarial training for cross-modality alignment, both of which mainly focus on the first-order similarity, ignoring the second-order similarity that can capture more structural information in the embedding space. To address these issues, we propose a novel cross-modal text-molecule retrieval model with two-fold improvements. Specifically, on the top of two modality-specific encoders, we stack a memory bank based feature projector that contain learnable memory vectors to extract modality-shared features better. More importantly, during the model training, we calculate four kinds of similarity distributions (text-to-text, text-to-molecule, molecule-to-molecule, and molecule-to-text similarity distributions) for each instance, and then minimize the distance between these similarity distributions (namely second-order similarity losses) to enhance cross-modal alignment. Experimental results and analysis strongly demonstrate the effectiveness of our model. Particularly, our model achieves SOTA performance, outperforming the previously-reported best result by 6.4%.
Introduction
In drug discovery, identifying molecules with specific biological properties described in scientific literature is a fundamental challenge. While databases like PubChem are extensive, many molecules lack expert-annotated text descriptions, making traditional keyword searches ineffective. Cross-modal retrieval addresses this by learning a shared embedding space where text descriptions and molecular graphs are mapped close together.
However, existing models face two major hurdles: the Modality Gap, caused by the fundamental structural differences between 1D text and 2D/3D molecular graphs, and First-Order Limitations, where models focus only on pair-wise matches while ignoring the broader relational structure of the embedding space. This paper introduces a novel architecture designed to bridge these gaps using a memory-bank-based feature projector and second-order similarity losses.
Methodology
The proposed model enhances cross-modal alignment through two primary innovations integrated into a dual-encoder framework.
1. Memory Bank Based Feature Projector
To capture modality-shared features more effectively, the model introduces a Memory Bank based Feature Projector. Instead of standard linear layers, this module utilizes a set of learnable "memory vectors" that act as semantic anchors. These vectors serve as queries in a cross-attention module, interacting with the outputs of the text encoder (SciBERT) and the molecule encoder (GCN). This forced interaction through shared anchors helps bridge the topological differences between text and molecular structures.
2. Second-Order Similarity Losses ($L_{u2u}$ and $L_{u2c}$)
The core of the alignment strategy is the shift from first-order to second-order similarity. The model calculates four types of similarity distributions within each training batch: uni-modal (Text-to-Text, Molecule-to-Molecule) and cross-modal (Text-to-Molecule, Molecule-to-Text). By using Kullback-Leibler (KL) divergence to minimize the distance between these distributions, the model ensures that the relational structure of the embedding space is consistent across modalities—if two molecules are chemically similar, their text descriptions should be positioned similarly in the shared space.
Experimental Results
The model was evaluated on the ChEBI-20 and PCdes datasets, achieving new state-of-the-art (SOTA) performance. On ChEBI-20, the model reached a Hits@1 of 56.5%, a significant 6.4% improvement over the previous best of 50.1%. Additionally, the Mean Rank in text-to-molecule tasks dropped from 14.49 to 12.66, indicating more accurate candidate sorting.
Modality Gap Reduction
Qualitative analysis using Kernel Density Estimation (KDE) confirms that the proposed model significantly reduces the modality gap. The distance between corresponding text and molecule embeddings is substantially smaller compared to baseline models like AMAN, demonstrating superior alignment.
Conclusion
By introducing a memory-bank-based projection mechanism and enforcing second-order relational consistency, this model effectively bridges the gap between text and molecular modalities. The results demonstrate substantial improvements in retrieval accuracy and embedding space cohesion. Future work may involve integrating Large Language Models (LLMs) and external knowledge graphs to further enhance the semantic depth of the retrieval process.
Summary updated