Abstract
SALSA (Semantically-Aware Latent Space Autoencoder) addresses the inherent "brittleness" of SMILES-based molecular representations by enforcing structural similarity within the model's latent space. By augmenting a standard Transformer-based autoencoder with a supervised contrastive loss derived from Graph Edit Distance (GED), SALSA produces a latent representation that is structurally, chemically, and biologically organized. Our results demonstrate that SALSA significantly outperforms naive encoders and traditional fingerprints in tasks such as virtual screening and semantic interpolation, providing a more robust foundation for AI-driven drug discovery.
Introduction: The Challenge of Molecular Representation
In the field of deep learning for chemistry, molecules are typically represented as SMILES (Simplified Molecular Input Line Entry System) strings. While these text-based representations allow for the use of powerful Natural Language Processing (NLP) architectures like Transformers, they are notoriously brittle. A single character change in a SMILES string can represent a profound structural shift, while two nearly identical molecules may have drastically different string representations.
When standard autoencoders are trained on SMILES, the resulting latent space—the model's internal mathematical map—becomes disorganized. Neighboring points in this space often lack chemical relationship, hindering tasks like property prediction and latent space interpolation. SALSA aims to resolve this by ensuring that structural similarity in the physical world corresponds to proximity in the latent space.
Methods: The SALSA Framework
The authors propose a dual-objective framework that modifies a standard Transformer-based sequence-to-sequence autoencoder to better capture chemical semantics.
Architecture and Objectives
The model utilizes a Transformer encoder and decoder separated by a bottleneck layer, which compresses the molecule into a fixed-size vector on a unit hypersphere. The training process involves two simultaneous tasks:
- SMILES Reconstruction: The model learns to decode the latent vector back into the original SMILES string.
- Supervised Contrastive (SupCon) Loss: This "secret sauce" explicitly instructs the model to minimize the distance between latent codes of structurally similar molecules while maximizing the distance between dissimilar ones.
Defining Similarity via Graph Edit Distance (GED)
To provide a ground-truth measure of similarity for the contrastive task, the authors employed Graph Edit Distance (GED). A "similar" pair consists of an "anchor" molecule and a "mutant" generated by a single graph edit (e.g., adding or replacing an atom). To ensure chemical plausibility, mutants were filtered based on physicochemical properties like molecular weight and solubility using Mahalanobis distance.
Results and Discussion
SALSA was trained on a massive dataset derived from ChEMBL21, consisting of approximately 1.5 million anchors and a novel mutant dataset of over 13.8 million compounds.
Structural and Biological Awareness
The impact of the contrastive objective is immediately apparent in the organization of the latent space. SALSA achieved a Spearman correlation of 0.88 between latent space distance and graph similarity, a substantial improvement over the 0.51 achieved by a naive autoencoder.
Beyond structural alignment, SALSA demonstrated superior biological awareness. On the RDKit Virtual Screening benchmark, SALSA achieved an AUROC of 0.73, outperforming ChemBERTa (0.68) and traditional ECFP4 fingerprints (0.62). This suggests that by learning structural patterns, the model implicitly captures the underlying biological and chemical properties essential for drug discovery.
Semantic Continuity
In interpolation tasks—finding intermediate molecules between two endpoints—SALSA generated candidates that were significantly more chemically related to the targets than the naive model, which often produced unrelated "noise." This continuity is crucial for navigating the latent space to discover novel drug candidates.
Conclusion
SALSA demonstrates that integrating domain-specific similarity metrics into the training of molecular autoencoders can overcome the limitations of text-based representations. By aligning the latent space with chemical reality, SALSA provides a more reliable and interpretable tool for molecular generation and property prediction. Future work may explore more complex mutation operators or apply this similarity-based contrastive paradigm to other scientific domains where data "meaning" can be mathematically defined.
Summary updated