Beyond Exact Match for Retrosynthesis Evaluation

This direction is split into other directions: exact-match evaluation bias (chemical validity metrics). Its last version stays readable and citable here.

Version 1 of 142 papers64 conceptsfrozen 2026-10-04

Retrosynthesis—the process of working backward from a target molecule to identify simpler precursors—is often framed as a machine translation problem. In this paradigm, a model "translates" a product SMILES string into reactant SMILES strings. For years, the gold standard for success has been Top-K accuracy: does the model’s top prediction exactly match the specific reaction recorded in a historical patent database?

However, a growing body of research argues that this "ground-truth dogma" is fundamentally unchemical. Chemistry is a "one-to-many" problem; a single molecule can often be synthesized through dozens of valid pathways. By penalizing any route that doesn't match a specific patent entry, we are not measuring a model's chemical intelligence, but rather its ability to memorize a biased historical record.

The Failure of Exact Match

The primary tension in modern retrosynthesis is the gap between statistical accuracy and chemical plausibility. As Hongliang Duan, Ling Wang, and Chengyun Zhang demonstrated in Retrosynthesis with Attention-Based NMT Model and Chemical Analysis of the "Wrong" Predictions, many "incorrect" model predictions are actually chemically valid alternative pathways. When experts reviewed these "wrong" answers, the "true" accuracy of their model jumped from 54.1% to 64.6%.

This discrepancy exists because standard metrics treat all prediction errors as equal. They fail to distinguish between a chemically impossible "hallucination" and a brilliant, novel synthesis that simply hasn't been patented yet. Bogdan Zagribelnyy and colleagues address this in When Single Answer Is Not Enough: Rethinking Single-Step Retrosynthesis Benchmarks for LLMs by introducing ChemCensor, a framework that prioritizes chemical plausibility over rigid string matching. Similarly, Junren Li and Lei Fang 0004 proposed Retro-BLEU (Retro-BLEU: Quantifying Chemical Plausibility of Retrosynthesis Routes through Reaction Template Sequence Analysis), which uses statistical analysis of reaction templates to quantify how much a predicted route "makes sense" to a human chemist.

From Feasibility to Practicality

Even if a reaction is chemically plausible, it may not be practical. A model might suggest a route that requires a $5,000 catalyst or a solvent that is highly toxic. Traditional search algorithms like Monte Carlo Tree Search (MCTS) often optimize for a single variable: the probability of the reaction succeeding.

Recent work has shifted toward multi-objective optimization. In From Feasible to Practical: Pareto-Optimal Synthesis Planning, Friedrich Hastedt and Antonio Del rio chanona introduced MORetro*, an algorithm that generates a "Pareto front" of synthesis routes. This allows a chemist to choose between a route that is faster (fewer steps) and one that is greener (less toxic) or cheaper (lower precursor cost).

This shift is echoed by Shiv Prakash in A User-Tunable Machine Learning Framework for Step-Wise Synthesis Planning, which allows users to weight environmental and economic factors. The goal is no longer just to find a path, but to find the best path for a specific laboratory context.

Capturing Chemical Context

A major hurdle in multi-step planning is that reactions do not happen in a vacuum. The success of step four often depends on the reagents used in step one. Traditional models that predict one step at a time often lose this "thread."

To solve this, Songtao Liu, Lu Lin, and Rex Ying developed FusionRetro (FusionRetro: Molecule Representation Fusion via In-Context Learning for Retrosynthetic Planning). By treating the entire synthetic route as a graph and using an attention-based fusion mechanism, the model incorporates contextual information from the whole pathway. This "in-context learning" mimics how a human chemist considers the entire strategy—protecting certain functional groups early on to ensure they survive later transformations.

Furthermore, the field is moving toward reasoning-driven models. Guoqing Liu and Junren Li introduced QFANG (A Scientific Reasoning Model for Organic Synthesis Procedure Generation), which generates not just reactants, but full experimental procedures. By training on nearly a million reaction-procedure pairs, these models provide machine-readable instructions that account for the order of reagent addition and specific conditions—details often ignored by SMILES-to-SMILES models.

Standardizing the "Reality Check"

As models become more complex, the need for rigorous, fair benchmarking has become critical. Krzysztof Maziarz introduced Syntheseus (Re-evaluating Retrosynthesis Algorithms with Syntheseus), an open-source library designed to standardize evaluation. Their work revealed that many "state-of-the-art" gains were actually due to inconsistent data post-processing or ignoring stereochemistry.

By enforcing "fair play"—such as correctly handling 3D molecular orientation and deduplicating outputs—researchers like Philippe Schwaller and Victor Sabanza Gil are exposing the pitfalls of current models (Holistic chemical evaluation reveals pitfalls in reaction prediction models). The consensus is clear: to move from "paper accuracy" to "lab utility," AI retrosynthesis must embrace the messy, multi-objective, and open-ended nature of real-world chemistry.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_beyond_exact_match_for_retrosynthesis_ev_c2c8228d,
  title = {Beyond Exact Match for Retrosynthesis Evaluation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/beyond-exact-match-for-retrosynthesis-evaluation/rdr_c2c8228d669543a4953fe8d0af73b0f7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_beyond_exact_match_for_retrosynthesis_ev_c2c8228d_v1,
  title = {Beyond Exact Match for Retrosynthesis Evaluation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/beyond-exact-match-for-retrosynthesis-evaluation/rdr_c2c8228d669543a4953fe8d0af73b0f7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}