Abstract
We cast retrosynthesis as a machine translation problem by introducing a special Tensor2Tensor (T2T) model, an entirely attention-based and fully data-driven architecture. Given a dataset comprising approximately 50,000 diverse reactions extracted from USPTO patents, the model significantly outperforms the baseline sequence-to-sequence (seq2seq) model (37.4%) by achieving a top-1 accuracy of 54.1%. To optimize performance, parameters such as batch size and training time were thoroughly investigated. Additionally, we offer a novel insight into the causes of grammatically invalid SMILES and conduct a test in which experienced chemists analyze "wrong" predictions. We find that many are chemically plausible despite differing from the ground truth, suggesting the model's effectiveness is underestimated and its "true" top-1 accuracy can reach 64.6%.
Introduction
Retrosynthesis—the process of identifying starting materials for a target molecule—is a cornerstone of organic chemistry. Traditionally, this relied on rule-based systems like LHASA, which require manual encoding of thousands of chemical rules by experts. These systems are expensive to maintain and often fail to generalize to novel chemistry.
Recent approaches have treated retrosynthesis as a Neural Machine Translation (NMT) task using SMILES (Simplified Molecular Input Line Entry System) notation. However, earlier Recurrent Neural Network (RNN) based seq2seq models faced two major hurdles: a high rate of grammatically invalid SMILES strings and a rigid evaluation metric that penalizes valid alternative synthetic pathways not present in the training database.
Methodology
This study implements the Tensor2Tensor (T2T) model, a Transformer architecture that replaces recurrence with self-attention mechanisms. This allows the model to weigh the importance of different atoms in a SMILES string simultaneously, capturing long-range structural dependencies more effectively than LSTMs.
Model Configuration
- Encoder-Decoder Stack: Utilizes multi-head attention to process molecular structures.
- Positional Encoding: Employs trigonometric functions to maintain spatial/order information of atoms.
- Optimization: Performance was maximized by using large batch sizes (up to 8192 tokens) and checkpoint averaging of the final 20 saved models.
Human-in-the-Loop Evaluation
To address the limitations of standard accuracy metrics, 10 experienced chemists reviewed predictions labeled as "wrong" by the automated system. They evaluated these predictions for chemical plausibility, checking if the suggested reactants could realistically form the target product via known mechanisms.
Key Results
The T2T model demonstrated a substantial performance leap over traditional RNN baselines. As shown in the accuracy comparison, the Transformer architecture consistently outperformed the baseline across all Top-k metrics.
- Top-1 Accuracy: 54.1% (vs. 37.4% for baseline).
- Top-10 Accuracy: 70.1%.
- Invalidity Rate: Reduced to 3.4% (from >10% in previous models).
Chemical Analysis of "Wrong" Predictions
A core finding of this research is the "Ground Truth Fallacy." Standard evaluation metrics assume there is only one correct way to synthesize a molecule. However, chemical space allows for multiple valid routes.
By analyzing 5,004 test samples, the authors found that 10.5% of the model's "errors" were actually chemically valid alternative pathways (e.g., using different coupling partners or oxidation states). When these plausible reactions are included, the "true" top-1 accuracy rises to 64.6%.
Grammatically invalid SMILES were found to be primarily caused by structural complexity, such as polycyclic or bridged ring systems, where the model lacks sufficient training examples to master the specific string syntax required for closure.
Conclusion
The application of the Transformer-based T2T model to retrosynthesis represents a significant advancement in data-driven chemistry. By moving beyond simple string matching and incorporating chemical expertise into the evaluation, this work demonstrates that NMT models are far more capable of "understanding" chemical logic than previously thought. Future work will focus on expanding the dataset for complex cyclic structures and extending the model to multi-step synthetic planning.
Summary updated