Automated explorers of chemical reactions push and pull atoms on a quantum-mechanical energy landscape and record which new molecules appear. They spend most of their compute on transformations nobody cares about. Interactive tools let a chemist prune that search by hand, but each pruning decision costs human attention, and the decisions do not scale to catalytic cycles with thousands of intermediates.
We propose replacing part of that hand-steering with a language model. The model writes candidate reaction mechanisms as short programs: ordered lists of bonds to form and break. A decoder masks every next token that would produce a chemically impossible program. Impossible here means an atom index absent from the current molecule, a bond broken that does not exist, or an atom pushed past its allowed valence by earlier steps in the same mechanism. Each accepted program is then run directly as a sequence of physical elementary-step searches.
The core bet is specific. Guaranteeing that a program parses is already solved and nearly worthless here. The value lies in guaranteeing that a multi-step mechanism stays chemically consistent as it is written, because such mechanisms cannot be cheaply enumerated and ranked.
Core Claim
Exploration engines such as Chemoton 2.0 search for elementary steps with "Newton trajectories," which push molecular fragments together (NT1) or force a chosen atom pair to bond or separate (NT2). On a reference set of 70 reactant sets, NT2 with the semi-empirical GFN2-xTB method recovers 79% of documented reactions. The same engine also finds 44 times more reactions than the literature documents, most of them off-pathway.
Human steering through the Heron interface and the "Steering Wheel" protocol of Steiner & Reiher helps. Steering alternates automatic network expansion with human selection steps. On the Eschenmoser–Claisen rearrangement it cut the run from 19,400 reaction trials (2,220 unique aggregates) to 12,600 trials (2,411 aggregates).
We claim that a decoder enforcing state-dependent chemical constraints can serve as an automatic selection step that proposes sequences of bond changes. It should reach reference products in fewer quantum-chemical trials than three alternatives. The first is unconstrained or schema-only constrained generation. The second is a learned ranker over enumerated single steps. The third is distance-based pruning of enumerated networks. We expect the advantage to concentrate on reactions whose mechanism needs several coordinated bond changes. If no such concentration appears, the claim fails.
Why the obvious version is not enough
The tempting combination writes a JSON schema for Steering Wheel commands and runs XGrammar. XGrammar is a pushdown-automaton engine that caches masks for the over-99% of tokens whose validity does not depend on context. It reports raising syntactic correctness from 62–80% to 100%. That buys executability and nothing chemical: a schema-valid command can still name atom 41 in a 30-atom molecule or form a sixth bond on a carbon.
The known limitation of fast engines sits precisely here. XGrammar and GREATGRAMMA operate on context-free grammars, and constraints like valency or ring closure need an external checker. GREATGRAMMA builds a lexer–tokenizer transducer with zero soundness failures and 5–32 ms per-token overhead, but it preprocesses each grammar offline. That rules out compiling a fresh grammar for every molecule the explorer visits.
Molecular Hypergraph Grammar shows that valency can be made context-free by treating atoms as hyperedges. It reports 100% validity and 94.8% reconstruction on ZINC, but it works on parse trees inside a custom autoencoder and generates static molecules, not reactions. The double-pushout graph grammars of rule composition for reaction patterns supply exactly the right semantics for bond rewriting. They reconstruct the formose cycle in under a second, but they are not tied to a generator or to any energy surface.
Our design borrows the rewriting semantics from the graph-grammar line and the mask-caching architecture from XGrammar. The new piece is a decoder state that is a live molecular graph.
Research Program
The mechanism language. The language has a small fixed skeleton. A mechanism is a sequence of steps. Each step is a set of bond operations, each written as form or break, followed by atom i, atom j and an optional bond order. A step may end with an optional "activation" clause that names a coordinate, a direction and a magnitude, in the style of imposed activation (IACTA).
Atom indices are added to the vocabulary as dedicated single tokens. This sidesteps the subword misalignment problem that GREATGRAMMA's transducers exist to solve. The skeleton's masks are cached once.
The decoder state. The state is the current connectivity graph with formal charges, electron count and per-element valence limits. It is updated after every completed bond operation, following double-pushout semantics so atom identity is preserved. At an atom-index position, the dynamic mask has three rules:
- Allow only atoms present in the current state.
- For a "break," allow only atoms bonded to the first atom.
- For a "form," allow only pairs that keep both atoms within valence. The check counts earlier steps in the same mechanism, and spatially remote pairs can optionally be excluded.
This check costs time linear in atom count and touches only the few hundred atom tokens. It should stay far below GREATGRAMMA's reported 5–32 ms per token. A step closes only when charge and electron count stay consistent, which is a stoichiometry check performed at step boundaries.
Execution. Each decoded mechanism is compiled into ordered trials: each step becomes an NT2 trial with its atom pairs, or an IACTA run when an activation clause is present. These are submitted to Chemoton on GFN2-xTB. Results feed back: discovered intermediates become new decoder states, and failed steps are logged as negative examples.
Steering. A chemist can type intent ("hydride migrations at the metal; ignore solvent") and pin atoms. Pinning replaces the requirement in disconnection-aware retrosynthesis that users tag every reaction-center atom. That work reports that accuracy falls as more atoms are tagged.
We do not place the decoder inside the 1,000 Hz haptic or 60 Hz visual loops. Its target cadence is the Steering Wheel's selection step, which takes seconds. Decoding tens of tokens at the reported overheads fits that window comfortably.
Model and data. We fine-tune an open mid-sized language model on bond-change sequences extracted from explored networks: the Heron Eschenmoser–Claisen data and the Steering Wheel case studies (Wilkinson, Ziegler–Natta, Monsanto), all deposited on Zenodo. Reactions from the Chemoton reference set are held out entirely, including any network containing them, to prevent leakage.
Masking renormalizes probability over the allowed tokens. A model that "wants" an invalid atom may therefore be pushed onto an unlikely valid one. We measure this by logging how much probability mass the mask removes at each step, and by testing whether high-removal mechanisms fail more often in quantum chemistry.
Baselines
The naive reference point is Chemoton 2.0's own unsteered NT2 enumeration with GFN2-xTB. For the Eschenmoser–Claisen network, Heron reports 19,400 trials yielding 2,220 aggregates unsteered. On the 70-reaction set, Chemoton reports 79% recovery. We found no reported result on Lacuna for trials-to-first-recovery per reaction, so we will measure it ourselves under identical settings.
The strongest human-steered competitor is the Steering Wheel with Heron, which reports 12,600 trials and 2,411 aggregates on Eschenmoser–Claisen. Beating it means matching or exceeding 2,411 aggregates, and recovering the reference products, in fewer than 12,600 trials. The proposal makes no human selections beyond one typed intent.
The strongest machine competitor is a learned ranker. It is a classifier with the same backbone, trained on the same data, that scores every enumerated NT2 single-step candidate. The top-k candidates are run in each round. This baseline gets the same trial budget, training data and tuning budget.
We also run the minimal-network pruning of Kim et al. on the reactions where it applies. That method generates intermediates from connectivity matrices and prunes them by chemical distance. It reports 36 DFT transition-state searches for cobalt hydroformylation.
Two ablations isolate the key ingredient. The first is unconstrained generation followed by graph-checker rejection and resampling. The second is schema-only XGrammar decoding, which guarantees syntax without the graph state. Beating these means fewer wasted quantum calls and fewer trials-to-recovery at equal model and data.
Why not just…?
Why not just prompt the model with a JSON schema and use XGrammar? That is the schema-only ablation. Syntax validity is solved (XGrammar reports 100%), but nothing prevents nonexistent atoms or over-valent bonds. Each such program wastes a GFN2-xTB trial, or crashes, without any chemical information returned. If the schema-only variant matches the full decoder on trials-to-recovery, our structure-aware state adds nothing. We would then report that outcome plainly.
Why not just generate freely and reject with a chemistry checker? Rejection is correct but pays for invalid generations. Its cost grows sharply with mechanism length, because one bad token anywhere invalidates a multi-step program. We will report resample counts, and wall-clock time to obtain k valid mechanisms, as a function of step count. For single-step proposals rejection may be competitive. We expect it to diverge only for longer mechanisms.
Why not just rank enumerated candidates with a classifier? This is the real threat. For single bond changes, a constrained decoder over a finite candidate set is mathematically a ranker. Our case rests on multi-step and concerted mechanisms. There, enumeration grows combinatorially, and validity depends on earlier steps in a way a per-candidate scorer cannot see. The stratified evaluation below settles it. If the ranker matches us on multi-bond reactions too, we drop the compositional claim.
Why not keep the human in the GUI or rely on IACTA alone? Heron already shows that human steering helps. IACTA, however, still needs a chemist to choose the activating coordinate. It found SN2, E2, E1, SN1 and other paths from one such choice, and 14 pathways for a water-mediated Michael addition. The user-study arm compares typed intent against GUI selection on the same case studies, measuring human time and trial count.
Proposed Experiments
The primary benchmark is the Chemoton 70-reactant-set reference collection. It includes the Zimmerman set, pericyclic reactions, transition-metal cases and bifurcations. All methods use GFN2-xTB, identical NT2 settings and a fixed trial budget per reactant set. Metrics are recovery rate at that budget, compared with Chemoton's reported 79% unbudgeted, median trials to first recovery, and the fraction of trials yielding any new aggregate.
The decisive analysis stratifies reactions by how many bonds change in the reference mechanism. The prediction is no gain over the ranker on single-bond reactions and a clear gain on reactions with three or more changes. On the Eschenmoser–Claisen network we report aggregates and reference products found against trials, overlaid on Heron's two reported points.
A transfer test applies the held-out model to the Monsanto carbonylation network. There we check whether it reaches the reported degradation mechanism with less human input than the Steering Wheel needed.
Finally, we log the mask-removed probability mass alongside downstream success. This tests whether the constraint helps the model or merely forces it into implausible chemistry.
References
- Chemoton 2.0: Autonomous Exploration of Chemical Reaction Networks
- <scp>Heron:</scp>Visualizing and Controlling Chemical Reaction Explorations and Networks
- A human-machine interface for automatic exploration of chemical reaction networks
- XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models
- Flexible and Efficient Grammar-Constrained Decoding
- Molecular Hypergraph Grammar with Its Application to Molecular Optimization
- Inferring chemical reaction patterns using rule composition in graph grammars
- Guided discovery of chemical reaction pathways with imposed activation
- Human-in-the-loop for a Disconnection Aware Retrosynthesis
- Efficient prediction of reaction paths through molecular graph and reaction network analysis