SALSA: Semantically-Aware Latent Space Autoencoder

Kathryn E. Kirchoff, Travis Maxfield, Alexander Tropsha, Shawn M. GomezAAAI 2024 · DOI

Abstract

SALSA (Semantically-Aware Latent Space Autoencoder) addresses the inherent "brittleness" of SMILES-based molecular representations by enforcing structural similarity within the model's latent space. By augmenting a standard Transformer-based autoencoder with a supervised contrastive loss derived from Graph Edit Distance (GED), SALSA produces a latent representation that is structurally, chemically, and biologically organized. Our results demonstrate that SALSA significantly outperforms naive encoders and traditional fingerprints in tasks such as virtual screening and semantic interpolation, providing a more robust foundation for AI-driven drug discovery.

Introduction: The Challenge of Molecular Representation

In the field of deep learning for chemistry, molecules are typically represented as SMILES (Simplified Molecular Input Line Entry System) strings. While these text-based representations allow for the use of powerful Natural Language Processing (NLP) architectures like Transformers, they are notoriously brittle. A single character change in a SMILES string can represent a profound structural shift, while two nearly identical molecules may have drastically different string representations.

When standard autoencoders are trained on SMILES, the resulting latent space—the model's internal mathematical map—becomes disorganized. Neighboring points in this space often lack chemical relationship, hindering tasks like property prediction and latent space interpolation. SALSA aims to resolve this by ensuring that structural similarity in the physical world corresponds to proximity in the latent space.

Methods: The SALSA Framework

The authors propose a dual-objective framework that modifies a standard Transformer-based sequence-to-sequence autoencoder to better capture chemical semantics.

The SALSA architecture integrates a supervised contrastive loss (Lc) alongside the standard reconstruction loss (Lr) to align latent representations with structural similarity.

Architecture and Objectives

The model utilizes a Transformer encoder and decoder separated by a bottleneck layer, which compresses the molecule into a fixed-size vector on a unit hypersphere. The training process involves two simultaneous tasks:

  1. SMILES Reconstruction: The model learns to decode the latent vector back into the original SMILES string.
  2. Supervised Contrastive (SupCon) Loss: This "secret sauce" explicitly instructs the model to minimize the distance between latent codes of structurally similar molecules while maximizing the distance between dissimilar ones.

Defining Similarity via Graph Edit Distance (GED)

To provide a ground-truth measure of similarity for the contrastive task, the authors employed Graph Edit Distance (GED). A "similar" pair consists of an "anchor" molecule and a "mutant" generated by a single graph edit (e.g., adding or replacing an atom). To ensure chemical plausibility, mutants were filtered based on physicochemical properties like molecular weight and solubility using Mahalanobis distance.

Results and Discussion

SALSA was trained on a massive dataset derived from ChEMBL21, consisting of approximately 1.5 million anchors and a novel mutant dataset of over 13.8 million compounds.

Structural and Biological Awareness

The impact of the contrastive objective is immediately apparent in the organization of the latent space. SALSA achieved a Spearman correlation of 0.88 between latent space distance and graph similarity, a substantial improvement over the 0.51 achieved by a naive autoencoder.

A comparison of latent space organization shows that SALSA (right) successfully clusters structurally similar molecules, whereas a naive encoder (left) scatters them across the representation space.

Beyond structural alignment, SALSA demonstrated superior biological awareness. On the RDKit Virtual Screening benchmark, SALSA achieved an AUROC of 0.73, outperforming ChemBERTa (0.68) and traditional ECFP4 fingerprints (0.62). This suggests that by learning structural patterns, the model implicitly captures the underlying biological and chemical properties essential for drug discovery.

Semantic Continuity

In interpolation tasks—finding intermediate molecules between two endpoints—SALSA generated candidates that were significantly more chemically related to the targets than the naive model, which often produced unrelated "noise." This continuity is crucial for navigating the latent space to discover novel drug candidates.

Conclusion

SALSA demonstrates that integrating domain-specific similarity metrics into the training of molecular autoencoders can overcome the limitations of text-based representations. By aligning the latent space with chemical reality, SALSA provides a more reliable and interpretable tool for molecular generation and property prediction. Future work may explore more complex mutation operators or apply this similarity-based contrastive paradigm to other scientific domains where data "meaning" can be mathematically defined.

Summary updated

BibTeX
@inproceedings{kirchoff2024salsa,
  author = {Kathryn E. Kirchoff and Travis Maxfield and Alexander Tropsha and Shawn M. Gomez},
  title = {{SALSA: Semantically-Aware Latent Space Autoencoder}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i12.29221},
}
Figure