In the search for new drugs and materials, the "chemical space" is effectively infinite. While deep learning models excel at navigating high-dimensional spaces, they face a fundamental hurdle in chemistry: molecules are not just arbitrary graphs or strings; they are physical objects governed by rigid rules of valency and geometry.
Standard generative models often produce "hallucinated" structures—atoms with too many bonds or impossible ring configurations—that cannot exist in a laboratory. To solve this, researchers are moving away from unconstrained generation toward architectures that bake chemical laws directly into the model's "grammar" or building blocks.
The Valency Problem
At the heart of molecular design is the concept of valency: the number of chemical bonds an atom can form. A carbon atom typically forms four bonds, while oxygen forms two. When a model like a standard Recurrent Neural Network (RNN) or a generic Graph Neural Network (GNN) generates a molecule, it treats atoms as tokens or nodes. Without explicit constraints, the model might "forget" these rules, leading to invalid structures.
As noted by Daniel Schwalbe‐Koda in Generative Models for Automatic Chemical Design, mapping discrete structures into continuous latent spaces allows for optimization, but ensuring the resulting "decoded" molecule is physically plausible remains a significant hurdle.
Grammar-Based Structural Guarantees
One way to ensure 100% chemical validity is to treat molecule generation as a formal language problem. Instead of predicting atoms one by one, the model follows a set of production rules—a grammar—that only allows for valid connections.
Hiroshi Kajino introduced the Molecular Hypergraph Grammar (MHG), which represents atoms as hyperedges and bonds as nodes. By framing generation as a context-free grammar, the model inherently respects valency constraints. This approach was later extended in MHG-GNN, which combines the encoding power of Graph Isomorphism Networks with a grammar-based decoder to ensure that every point in the latent space corresponds to a valid molecule.
Similarly, Hanjun Dai and colleagues developed Syntax-Directed Variational Autoencoders to design polymers. By enforcing chemical syntax during the "dreaming" process, these models avoid the "spaghetti" structures common in unconstrained AI.
Building with Motifs instead of Atoms
If building a molecule atom-by-atom is like trying to assemble a machine from individual screws and gears, "motif-based" generation is like using pre-assembled components. By using frequently occurring substructures—like benzene rings or functional groups—as the primary building blocks, models can maintain local structural integrity by design.
Wengong Jin pioneered this in Hierarchical Generation of Molecular Graphs using Structural Motifs. By shifting to a coarse-to-fine approach, the model ensures that complex motifs are preserved. More recently, Peizhi Niu introduced DMol, a graph diffusion model that compresses these motifs into "supernodes," significantly speeding up generation while improving the "naturalness" of the resulting drug candidates.
Constrained Denoising and Diffusion
Modern generative chemistry has shifted toward diffusion models, which "denoise" a cloud of atoms into a structured molecule. However, continuous diffusion often struggles with the discrete nature of chemical bonds.
To address this, researchers like Mingda Li developed CrysVCD, which uses a "valence decomposition" framework to guide the diffusion process. By combining an elemental language model that understands valence rules with a geometric diffusion model, the system generates stable materials more efficiently than pure data-driven approaches.
In the 3D space, Can Xu and the team behind GFMDiff integrated chemical valency constraints directly into the training objective. This ensures that as the model places atoms in 3D space, it does so in a way that naturally supports valid bond formation, reducing the need for heuristic post-processing.
Active Intervention and Rejection Sampling
When the model's architecture itself cannot guarantee validity, researchers use "critics" or "checkers" to intervene during the sampling process.
- Valency-Based Rejection: Mariya Popova used this in MolecularRNN, where the model builds a graph atom-by-atom but rejects any step that would violate a valency budget.
- Discriminator Guidance: Filip Ekström Kelvinius introduced Discriminator Guidance for ARDMs, using a separate model to "correct" the generative model's mistakes in real-time.
- Iterative Denoising: Yoann Boget proposed Critical Iterative Denoising (CID), which uses a "Critic" to selectively re-noise and fix suspicious parts of a generated graph that might violate connectivity rules.
By moving from unconstrained "black box" generation to systems that respect the fundamental axioms of chemistry, these models are becoming reliable tools for discovering the next generation of medicines and materials.