Introduction
In drug discovery and materials science, a molecule's function is fundamentally dictated by its three-dimensional conformation. While massive datasets of 1D (SMILES) and 2D (graphs) representations exist, high-quality 3D geometric data linked to scientific literature is scarce. Existing multimodal models often rely on approximate 3D structures generated by fast but noisy algorithms, which lack the precision of "ground-state" geometries—the stable shapes molecules naturally assume.
GeomCLIP addresses this gap by introducing a multimodal pre-training framework that aligns high-fidelity 3D molecular geometries with biomedical text descriptions. By training on a newly curated dataset of over 200,000 ground-state structures, GeomCLIP achieves state-of-the-art performance across various molecular understanding tasks.
Methodology: The GeomCLIP Framework
GeomCLIP utilizes a dual-encoder architecture inspired by CLIP (Contrastive Language-Image Pre-training). The framework consists of a 3D Geometry Encoder (based on the Uni-Mol architecture) and a Text Encoder (utilizing Sci-BERT).
The model is trained using a hybrid objective. Beyond standard Contrastive Alignment, which pulls embeddings of matching molecule-text pairs closer in a shared latent space, GeomCLIP incorporates Unimodal Denoising. This task requires the 3D encoder to reconstruct original atom positions from noisy versions, ensuring the model retains "geometric intuition" and physical common sense, preventing it from over-simplifying structures just to match text features.
Data: The PubChem3D Dataset
A core contribution of this work is the creation of PubChem3D, a high-quality dataset containing 203,257 pairs of ground-state 3D geometries and biomedical text. Geometries were sourced from PubChemQC and GEOM, utilizing DFT-calculated (Density Functional Theory) structures. Text annotations were retrieved from PubChem, covering properties like toxicity, solubility, and biological roles. This high-fidelity data provides a significant performance boost over models trained on approximate 3D structures.
Evaluation and Key Results
GeomCLIP was evaluated across three primary downstream categories: property prediction, retrieval, and captioning.
- Property Prediction: On the QM9 benchmark, GeomCLIP achieved the lowest Mean Absolute Error (MAE) across all 12 quantum mechanics tasks, outperforming specialized 3D models like Uni-Mol and 3D-MoLM.
- Zero-Shot Retrieval: In molecule-to-text and text-to-molecule retrieval, GeomCLIP reached an accuracy of 53.50%, a significant improvement over the previous best of 51.52%.
- Molecule Captioning: The model produced more accurate and detailed natural language descriptions of 3D molecules, reflected in superior BLEU and ROUGE scores compared to baseline models.
Conclusion and Future Work
GeomCLIP demonstrates that ground-state geometries and scientific text act as powerful mutual supervisory signals. While the current dataset of 200k pairs is a milestone, it remains small compared to general-purpose AI datasets. Future work will focus on scaling the data, modeling the dynamic ensembles of molecular movement (conformer dynamics), and integrating more diverse literature-based descriptions to further refine the model’s understanding of the molecular world.
Summary updated