Learning Functional Group Incompatibility from How Chemists Route Around It

Generative models that explore chemical space — programs that propose new molecules one fragment at a time, guided by reinforcement learning — have gotten good at a strange thing: designing molecules that look wonderful on paper and cannot actually be made. The fix so far has been to put a retrosynthesis planner, a program that works backwards from target to purchasable starting materials, inside the generative loop. Models like Saturn do this well enough to find synthesizable candidates in a thousand planner calls where unconstrained search needed four hundred thousand Directly optimizing for synthesizability in generative molecular design using retrosynthesis models. But these planners check only that each disconnection matches a known reaction template and a catalog building block. They never ask the question that decides whether a reaction actually works at the bench: will the other functional groups already on the molecule — the free amine, the pyridine, the sulfonyl fluoride — poison the catalyst or destroy the reagent before the desired bond forms? The organic literature documents these clashes precisely: anilines and amides shut down nickel-catalyzed migratory couplings by binding the metal Nickel-catalyzed migratory alkyl–alkyl cross-coupling reaction, heterocyclic nitrogens arrest dirhodium desymmetrizations Chiral benzo[2.2.1] α-hydroxyketones from a dirhodium( <scp>ii</scp> )/bisphosphine-catalyzed desymmetrization addition reaction, and standard Lewis-basic borylation conditions hydrolyze sulfonyl fluorides Photoredox-catalyzed stereo- and regioselective vicinal fluorosulfonyl-borylation of unsaturated hydrocarbons. A generator armed only with templates scores such routes as fully synthesizable.

The obvious obstacle to fixing this is that failed reactions are almost never published, so there is no training data of clashes. My proposal turns this obstacle into the method. When a functional group genuinely conflicts with a reaction class, chemists do not abandon the target — they route around it: they switch ligands, crank up catalyst loading, add a protecting group, change solvent, or reach for a specialized reagent such as the integrated imidazolium sulfonate that lets borylation coexist with sulfonyl fluorides Photoredox-catalyzed stereo- and regioselective vicinal fluorosulfonyl-borylation of unsaturated hydrocarbons or the transient activating reagent that functionalizes azines without harsh oxidants A photoinduced transient activating strategy for late-stage chemoselective C(sp<sup>3</sup>)–H trifluoromethylation of azines. These workarounds are recorded in successful reactions, which are exactly the data we have. The core claim of this proposal is that functional group incompatibility can be learned statistically from workaround signatures in large reaction corpora, and that the resulting clash predictor, used as a dense reward penalty inside a generative model, sharply cuts the false-positive rate of machine-proposed syntheses without sacrificing the property scores that motivated the search.

Four lines of work approach this boundary and stop short. Saturn and its successors put retrosynthesis in the loop but the oracle is purely topological; the authors state explicitly that reaction conditions and selectivity are out of scope Directly optimizing for synthesizability in generative molecular design using retrosynthesis models, and the granular-constraint follow-up lets users forbid whole reaction classes but has no model of substrate-level cross-reactivity, so an enforced Suzuki on a substrate bearing an unprotected coordinating amine still passes Generative Molecular Design with Steerable and Granular Synthesizability Control. TANGO supplies dense rewards for retaining building blocks, but its Tanimoto and substructure terms track static graph topology only It Takes Two to Tango: Directly Optimizing for Constrained Synthesizability in Generative Molecular Design. The condition-optimization work couples Saturn to a condition predictor trained on ~5 million Pistachio reactions, but conditions are scored for green-chemistry targets, not for whether the substrate's functional handles survive them Synthesis-constrained molecular design with direct optimization of reaction conditions. ARP makes route quality a first-class quantity via a yield surrogate, but treats yield as an empirical scalar blind to mechanism, and its surrogate inherits the positive-outcome bias of the literature Active Retrosynthetic Planning Aware of Route Quality. On the representation side, FARM builds chemically literate functional-group tokens and a 101-motif knowledge graph but is an encoder for property prediction, never a generator or a reactivity model FARM: Functional Group-Aware Representations for Small Molecules, and MolR's reaction-conservation embeddings assume ideal stoichiometry and reaction success Chemical-Reaction-Aware Molecule Representation Learning. None of these learns, or even represents, the pairwise clash between a spectator group and a reaction's catalytic system.

Research Program

The program has three components. First, mine the clash statistics. Over a large reaction corpus — USPTO-479k (479,000 patent reactions, the corpus on which MolR was trained MolR), plus the Pistachio set used to train condition prediction if accessible Synthesis-constrained molecular design with direct optimization of reaction conditions — I will compute, for each reaction class, the conditional frequency of each spectator functional group among the reactants, and compare it to that group's base frequency in the purchasable building-block pool. A group that is common in the pool but systematically absent from the reactants of a class is flagged as a putative clash; a group that appears only alongside non-modal conditions (unusual ligand, elevated temperature, high catalyst loading, an adjacent protecting-group step) is flagged as a conditional clash — survivable only via a workaround. Second, validate the mined matrix against known mechanistic chemistry: the curated list from the organic literature should recover the documented cases (anilines and amides versus nickel migratory coupling, heterocyclic nitrogen versus dirhodium catalysis, alkaloid nitrogens versus organometallic reagents in the Reformatsky setting Enantioselective Reformatsky Reaction Using Cinchona Alkaloids as Chiral Ligands, sulfonyl fluorides versus standard borylation, and the counter-cases where mild reagents such as trifluoroborate salts tolerate free hydroxyl and nitro groups Organoboron compounds as mild nucleophiles in Lewis acid- and transition metal-catalyzed CC bond-forming reactions). A separate check uses the electronic-lab-notebook corpus of 11,210 Buchwald–Hartwig couplings on which context prediction reached roughly 90% top-3 accuracy Prediction of the Chemical Context for Buchwald‐Hartwig Coupling Reactions: substrates whose recorded context deviates from the modal context for the class should be exactly those the model flags.

Third, integrate the predictor into generation. For each retrosynthetic step $t$ in a proposed route, let $\mathcal{F}_t$ be the multiset of spectator functional groups on the intermediates, $r_t$ the reaction class, and $c_t$ the predicted conditions. The predictor outputs a clash probability $P_{\text{clash}}(\mathcal{F}_t, r_t, c_t)$, and the route-level penalty is its sum over steps:

$$\text{penalty}(\text{route}) = -\lambda \sum_{t=1}^{T} P_{\text{clash}}(\mathcal{F}_t, r_t, c_t)$$

with $\lambda$ a single tunable weight. This penalty is added to the property reward of the existing Saturn Mamba generator (a sequence model that writes molecules token by token), exactly where TANGO's dense similarity reward sits It Takes Two to Tango: Directly Optimizing for Constrained Synthesizability in Generative Molecular Design — a dense, soft signal rather than a hard filter, because the condition-constrained work showed that rigid constraints collapse molecular diversity and property scores Synthesis-constrained molecular design with direct optimization of reaction conditions. The penalty also propagates up the route: a clash at step 2 makes the entire molecule that requires that route less attractive, which is precisely the coupling between generation and execution that topological oracles miss.

Baselines

The naive reference point is unmodified Saturn with its topological retrosynthetic oracle: any molecule whose disconnections match templates and catalog building blocks counts as synthesizable, so its false-positive rate on clash-bearing routes defines the problem's magnitude. The strongest existing competitor is Saturn+TANGO, the same generative core with the dense Tanimoto-and-substructure reward, which on the ClpP protease inhibitor task produced over 200 candidates with docking scores below $-10$, quantitative estimate of drug-likeness around 0.77, and an average route length of 2.35 reaction steps It Takes Two to Tango: Directly Optimizing for Constrained Synthesizability in Generative Molecular Design. It will be run under identical conditions to this proposal: the same pre-trained generator, the same Enamine and ZINC building-block inventory, the same docking oracle and property targets, and the same 1,000-call oracle budget that the Saturn line established as its sample-efficiency standard Directly optimizing for synthesizability in generative molecular design using retrosynthesis models, with the same tuning budget for reward weights. Beating it means matching its property metrics (docking below $-10$, drug-likeness near 0.77, route length near 2.35) while producing routes with a substantially lower clash rate on the probe benchmark below — if the clash penalty costs more than a small margin in docking or diversity, the trade is a failure. A second strong competitor on the route-quality axis is ARP, whose actor–critic planner improved route quality by 6.2% on 178 hard USPTO-50k targets and won a double-blind chemist rating of 9.1/10 versus 6.9/10 for its base planner Active Retrosynthetic Planning Aware of Route Quality; I will run the same chemist-rating protocol on clash-flagged versus unflagged routes. Finally, the key ablation replaces the mined co-occurrence statistics with a hand-written rule list encoding the same documented clashes (aniline plus nickel coupling, pyridine plus dirhodium, and so on). If the hand list performs as well as the mined matrix, the data-mining contribution is empty and the proposal reduces to expert curation; if the mined matrix catches clashes absent from the curated list, the learned ingredient earns its place.

Why not just…?

Why not just filter the generated molecules afterwards with heuristic rules or the SA score? This is the cheapest alternative, and the evidence against it is threefold. The SA score, a classic heuristic for "ease of synthesis," degrades badly outside standard drug-like scaffolds, penalizing buildable structures and rewarding unbuildable ones Directly optimizing for synthesizability in generative molecular design using retrosynthesis models. Post-hoc filtering is also wasteful: the same paper reports that unconstrained generation needed roughly 400,000 oracle evaluations to find synthesizable candidates that in-loop guidance found within 1,000 — a filter discards the whole generation budget spent on candidates that die at the end. And static rules are context-blind: a functional group fatal under thermal metal catalysis can be tolerated under photoredox or electrochemical conditions, as the late-stage functionalization literature shows Late Stage Functionalization with Broad Functional Group Tolerance, so a flat forbidden-pair list both over- and under-blocks. The ablation above is nonetheless the honest test: it is the rule-filter alternative implemented inside the loop, and if it matches the mined matrix I will report that the curation suffices.

Why not just tell the retrosynthesis engine to avoid problematic reaction classes? The granular-constraint version of Saturn already accepts "avoid this reaction" directives Generative Molecular Design with Steerable and Granular Synthesizability Control. But macro-avoidance cannot see the substrate: forbidding, say, nitro reductions says nothing about whether an allowed Suzuki coupling on this particular molecule will be poisoned by its pendant pyridine — the exact failure documented for coordinating groups in nickel catalysis Nickel-catalyzed migratory alkyl–alkyl cross-coupling reaction. Avoiding every class that ever clashes with any group would forbid most of chemistry: the condition-constrained work showed that even mild condition constraints noticeably reduced the number of synthesizable molecules found and depressed property scores Synthesis-constrained molecular design with direct optimization of reaction conditions. I will run this baseline directly — Saturn with conservative avoid-reaction directives — and expect it to show the same diversity collapse while leaving substrate-level clashes untouched.

Why not just validate every proposed step with a high-accuracy forward reaction model? Forward models predict the intended major product with about 90–91% top-1 accuracy on USPTO-MIT Data-driven Chemical Reaction Prediction and Retrosynthesis, so a natural idea is to chain one behind the planner and reject steps that fail. Three problems: these models are trained on reported major products and are structurally blind to unrecorded side products and degraded handles — the "exact match fallacy" A Unified View of Deep Learning for Reaction and Retrosynthesis Prediction: Current Status and Future Challenges; they take only reactant structures as input and ignore catalysts, ligands, solvents, and temperature, which are what actually determine whether a group survives Prediction of the Chemical Context for Buchwald‐Hartwig Coupling Reactions; and their inference cost inside a generative loop with thousands of trajectory nodes is prohibitive — deep-model oracle calls are the stated bottleneck of the Saturn line Directly optimizing for synthesizability in generative molecular design using retrosynthesis models. I will include a scaled-down version of this baseline (forward validation on final routes only) to quantify how many clashes it actually catches; my expectation is that it confirms the intended-product blindness, but the experiment settles it.

Proposed Experiments and Decisive Checks

Experiment one validates the mined clash matrix. The ground truth is a curated benchmark I will assemble from the documented mechanistic chemistry above: pairs of (spectator group, reaction class) with literature-attested catastrophic interference, plus matched negatives where mild reagents tolerate the same groups Organoboron compounds as mild nucleophiles in Lewis acid- and transition metal-catalyzed CC bond-forming reactions. The metric is recall and false-flag rate of the mined matrix against this list, plus the lab-notebook check on the Buchwald–Hartwig corpus (does flagged status predict non-modal recorded contexts?). Experiment two is the generation benchmark: the TANGO-style ClpP inhibitor task under matched budgets, reporting docking score, drug-likeness, route length, diversity, and — the new metric — clash rate, the fraction of proposed routes containing at least one step the predictor flags above threshold, cross-checked against the curated list. Experiment three stress-tests scale on the 142-billion-molecule Chemspace Freedom 4.0 library, where the granular Saturn achieved over 90% exact-match retrieval while assessing 0.00001% of the space Generative Molecular Design with Steerable and Granular Synthesizability Control; the question is whether the clash penalty changes which molecules are retrieved, not just how they are routed. The decisive check is the ARP-style double-blind chemist evaluation, comparing routes from the clash-penalized model against the strongest baseline on the same targets; the bar is the 9.1/10 versus 6.9/10 separation that route-quality awareness achieved in planning Active Retrosynthetic Planning Aware of Route Quality. If chemists cannot distinguish the route sets, or if the clash penalty buys its reduction in false positives at the cost of the property scores, the claim fails — and the fallback contribution, a validated statistical map of which functional groups chemists route around and how, would remain useful on its own.