When deep learning models process text, a typo might result in a misspelled word that a human can still easily understand. However, when generative models process traditional chemical text representations, a single character error is catastrophic. In SMILES (Simplified Molecular-Input Line-Entry System), the long-standing standard for chemical notation, a missing closing parenthesis or an extra double bond will yield a completely invalid molecular graph—a physical impossibility that cannot be parsed by chemical software.
Because the space of valid SMILES strings is an incredibly sparse, winding maze inside the space of all possible character combinations, generative models must spend significant capacity just learning the basic syntax of chemistry. To solve this, a rapidly growing body of research shifts the burden of structural validity away from the machine learning model and directly into the design of the chemical language itself.
By utilizing grammar-constrained, self-referencing string representations—most notably SELFIES (Self-Referencing Embedded Strings)—researchers have created encodings where every possible sequence of tokens maps to a physically valid molecule.
The Core Intuition: Shifting Validity to the Grammar
To understand why grammar-constrained representations work, consider how a standard string representation is parsed versus a self-referencing one. A SMILES string is a static sequence of characters that relies on global state matching (such as matching numbers for ring closures). If a model mutates a single character, it breaks this global state.
As explained by Pollice et al. and Ai et al., robust representations like SELFIES replace static characters with a formal, context-free grammar composed of self-referencing functions. Instead of reading a string as a list of letters, the parser treats tokens as dynamic instructions for constructing a molecular graph.
The grammar enforces physical constraints—such as the maximum valency of an atom (e.g., carbon can only form four bonds)—at each derivation step:
- State-Dependent Interpretation: If a token instructs the parser to "add a triple bond," the parser first checks the available valency of the current atom. If the atom only has one available slot left, the grammar automatically downscales the instruction to a single bond.
- Bounded Overflows: If an instruction requests a ring closure between two atoms, but the specified index exceeds the number of existing atoms in the chain, the grammar safely caps the index or adapts the instruction so that it remains structurally sound.
Because the rules of chemical valency are embedded directly into the language's derivation rules, it becomes mathematically impossible to write an invalid string. Even a completely random sequence of tokens will decode into a physically plausible chemical structure.
Liberating Machine Learning Models
When a generative model is freed from the need to memorize complex syntactic rules, its architecture can be utilized much more efficiently.
In large language models (LLMs) and cross-modal frameworks, this representational shift eliminates "molecular hallucinations"—instances where a model generates text that looks like a chemical string but fails to correspond to a real molecule. For example, Fang et al. demonstrated with MOLGEN that using self-referencing strings achieves 100% chemical validity in molecular generation, allowing the model to effectively navigate complex natural product domains without wasting computational resources on invalid graphs.
Similarly, the creation of large-scale biomolecular instruction datasets, such as Mol-Instructions developed by Fang et al., relies on these robust representations to ensure that the output space of an LLM is strictly constrained to chemically feasible structures. This allows the underlying neural network to focus its attention entirely on capturing high-level chemical logic, properties, and relationships rather than basic syntax.
This reliability is crucial for cross-modal architectures like BioT5 (Pei et al.) and Enhanced BioT5+ (Pei et al.), which translate between natural language descriptions and molecular structures. By guaranteeing that every generated sequence maps to a valid molecular graph, these models provide a more dependable pipeline for text-based molecular discovery.
Smoothing the Optimization Landscape
The benefits of grammar-constrained encodings become especially apparent during molecular optimization, where an algorithm must iteratively tweak a structure to maximize a specific property (such as drug likeness or binding affinity).
In traditional string spaces, the optimization landscape is highly discontinuous: moving a single step in any direction almost always throws the model off the narrow path of validity into a void of unparseable strings. By contrast, an inherently valid string representation creates a continuous, dense search space.
SMILES Space: [Valid Molecule] ---> (Invalid String) ---> (Invalid String) ---> [Valid Molecule]
SELFIES Space: [Valid Molecule] ---> [Valid Molecule] ---> [Valid Molecule] ---> [Valid Molecule]
This structural continuity fundamentally changes how different search engines behave:
- Discrete Latent Spaces: In frameworks like LIMO, developed by Zhao et al., stacking a property predictor onto a Variational Autoencoder (VAE) allows for rapid gradient-based optimization. Because every point in the latent space decodes into a valid structure, the system completely bypasses the need for complex validity filters or reinforcement learning penalties. Similarly, LOL-BO (Maus et al.) pairs these robust representations with high-capacity sequence models to ensure that points sampled from local optimization trust regions remain structurally valid.
- Evolutionary and Language-Driven Search: In evolutionary algorithms, random mutations and crossovers frequently break traditional strings. By using grammar-constrained strings, Nigam et al. showed that every mutation event yields a physically possible structure, drastically reducing computational waste when designing organic emitters. When scaling this up to use LLMs as genetic operators, the MOLLEO framework (Wang et al.) leverages this property to guide evolutionary search with human-like chemical intuition without suffering from syntax errors.
However, the choice of representation is not a silver bullet on its own. Benchmarks like PMO (Gao et al.) and TARTARUS (Nigam et al.) demonstrate that while robust representations provide more stable optimization paths, their ultimate success depends heavily on how well the underlying search engine interacts with the specific chemical domain being explored.
Broadening Applications Across Chemistry
As grammar-constrained representations mature, their adoption is expanding beyond simple drug-like molecules into diverse facets of chemical machine learning.
In structural biology, Villegas-Morcillo et al. extended this concept to all-atom protein sequence design using discrete diffusion models, ensuring the generation of physically plausible chemical backbones. In drug-target interaction (DTI) modeling, frameworks like SaBAN (Lan et al.) and FusionDTI (Meng et al.) utilize token-level encodings of valid strings to map fine-grained interactions between specific drug atoms and protein amino acids.
Furthermore, the robustness of these encodings has proven vital for cross-modal translation tasks, such as MS-BART (Han et al.), which elucidates molecular structures directly from mass spectrometry data. It has also accelerated the discovery of functional materials, enabling deep generative models to successfully navigate the compressed latent spaces of catalyst candidates for complex chemical reactions (Schilter et al.).
By encoding the immutable laws of chemistry directly into the syntax of the data, grammar-constrained representations allow machine learning models to stop acting as amateur grammarians and start acting as true computational chemists.
Key Foundational Work
Deep molecular dreaming: inverse machine learning for de-novo molecular design and interpretability with surjective representations (2021) — contributed 2 concept elements to this direction.
Beyond generative models: superfast traversal, optimization, novelty, exploration and discovery (STONED) algorithm for molecules using SELFIES (2021) — contributed 2 concept elements to this direction.
Augmenting Genetic Algorithms with Deep Neural Networks for Exploring the Chemical Space (2020) — contributed 2 concept elements to this direction.