Are VAEs Bad at Reconstructing Molecular Graphs?

Hagen Münkler, Hubert Misztela, Michal Pikusa, Marwin H. S. Segler, Nadine Schneider, Krzysztof MaziarzarXiv 2023 · DOI · arXiv

Abstract

Variational Auto-Encoders (VAEs) are a cornerstone of generative chemistry, mapping complex molecular graphs into continuous latent spaces for drug discovery. However, this research reveals a surprising truth: state-of-the-art graph VAEs are remarkably poor at exact reconstruction of molecular graphs, particularly on large, diverse datasets. Crucially, the authors demonstrate that high reconstruction accuracy is not a prerequisite for success in downstream optimization tasks. Even when a VAE fails to rebuild a molecule exactly, it captures essential chemical "vibes," producing structural isomers or "chemical cousins" rather than random noise.

Introduction

In AI-driven drug discovery, VAEs are tasked with encoding molecules into a continuous latent space and then decoding them back into their original graph structures. A common assumption in the field is that a model's inability to reconstruct its inputs signifies a failure to learn the underlying data distribution. While previous benchmarks using small or simple datasets like ZINC or polymers reported high reconstruction rates, this paper investigates whether these results hold for large-scale, chemically diverse datasets like Guacamol.

Methodology

The study performs a head-to-head comparison of three prominent graph-based VAE architectures: JT-VAE (Junction Tree VAE), HierVAE (Hierarchical VAE), and MoLeR (Learning to extend molecular scaffolds). A key focus was the $β$ coefficient, which regulates the trade-off between reconstruction accuracy and latent space regularity (KL divergence). To diagnose failures, the researchers developed a "distinguishability measure" to check for latent space collapse and a stepwise error analysis to pinpoint where decoders fail during the graph-building process.

Experimental Results

On the diverse Guacamol dataset, reconstruction accuracy was found to be surprisingly low across all models. While SMILES-based models (treating molecules as text) can reach ~95% accuracy, the best graph-based model (HierVAE) peaked at approximately 66%.

Figure 1: Performance metrics across different models and β values. The middle panel shows reconstruction accuracy over epochs, while the right panel illustrates the trade-off between reconstruction and optimization metrics as β varies.

As shown in the figure above, there is a "sweet spot" for $β$. At very low $β$, reconstruction is high but the latent space is fractured and poor for optimization. Conversely, at high $β$, reconstruction vanishes, but the model becomes repetitive. The most effective models for discovery operate in the middle ground, sacrificing exact reconstruction for a searchable, smooth latent space.

The Reconstruction Paradox and "Chemical Cousins"

A central finding is that near-zero reconstruction accuracy does not prevent a model from being a top performer in molecular optimization. The errors made during reconstruction are rarely random; instead, they represent an assembly problem rather than a memory problem. In the MoLeR model, over 90% of errors involved choosing incorrect attachment points or bond types rather than incorrect building blocks (motifs).

Figure 2: Similarity analysis of failed reconstructions. The left panel shows that reconstructed molecules often maintain high Tanimoto similarity to the original, while the right panel indicates significant motif overlap, supporting the "chemical cousins" hypothesis.

Failed reconstructions often result in molecules that are structurally and chemically similar to the input. These "chemical cousins" share properties like solubility and Tanimoto similarity with the original molecule, suggesting that the VAE has successfully captured the relevant chemical features even if the exact graph topology is lost.

Conclusion and Future Work

This study challenges the reliance on reconstruction accuracy as a primary metric for generative molecular models. While exact reconstruction may not be necessary for general property optimization, the authors note it might remain critical for "activity models" where minor structural changes (e.g., moving a single atom) can fundamentally alter a drug's efficacy. Future research may focus on "property-aware" decoders that prioritize the reconstruction of biologically active motifs over exact graph matching.

Summary updated

Abstract

Many contemporary generative models of molecules are variational auto-encoders of molecular graphs. One term in their training loss pertains to reconstructing the input, yet reconstruction capabilities of state-of-the-art models have not yet been thoroughly compared on a large and chemically diverse dataset. In this work, we show that when several state-of-the-art generative models are evaluated under the same conditions, their reconstruction accuracy is surprisingly low, worse than what was previously reported on seemingly harder datasets. However, we show that improving reconstruction does not directly lead to better sampling or optimization performance. Failed reconstructions from the MoLeR model are usually similar to the inputs, assembling the same motifs in a different way, and possess similar chemical properties such as solubility. Finally, we show that the input molecule and its failed reconstruction are usually mapped by the different encoders to statistically distinguishable posterior distributions, hinting that posterior collapse may not fully explain why VAEs are bad at reconstructing molecular graphs.
BibTeX
@misc{munkler2023are,
  author = {Hagen Münkler and Hubert Misztela and Michal Pikusa and Marwin H. S. Segler and Nadine Schneider and Krzysztof Maziarz},
  title = {{Are VAEs Bad at Reconstructing Molecular Graphs?}},
  year = {2023},
  eprint = {2305.03041},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.03041},
}
Figure