ReactGPT: Understanding of Chemical Reactions via In-Context Tuning

Zhe Chen, Zhe Fang, Wenhao Tian, Zhaoguang Long, Changzhi Sun, Yuefeng Chen, Hao Yuan, Honglin Li, Man LanAAAI 2025 · DOI

TL;DR

This paper introduces ReactGPT, a novel framework that adapts Large Language Models (LLMs) to understand and generate text descriptions of chemical reactions. By leveraging in-context learning and a unique "reaction captioning" task, ReactGPT bridges the gap between chemical reaction representations and natural language, improving both the accuracy and structure of generated descriptions.

What Problem Does This Paper Address?

The field of chemistry and AI is rapidly advancing, with LLMs showing great promise in tasks like molecule captioning (describing individual molecules). However, a significant gap exists in understanding and describing chemical reactions themselves. While reactions are fundamental to chemical discovery, patents, and literature, existing AI models have largely focused on single molecules. Accurately describing a chemical reaction involves detailing reactants, conditions (temperature, pressure, time), solvents, catalysts, and product characteristics (yield, spectroscopic data). This detailed description is crucial for reproducibility and understanding. Current automated methods struggle with the complexity and nuanced language required for these descriptions, and standard text evaluation metrics are insufficient for assessing the quality of reaction descriptions.

Methods

ReactGPT tackles this challenge by adapting LLMs through a three-pronged approach:

  • Fingerprints-based Reaction Retrieval: Instead of random examples, ReactGPT uses a retrieval module that finds similar reaction-caption pairs from a database. This similarity is determined using "reaction fingerprints," which are vector embeddings of chemical reactions generated by pre-trained models (like Rxnfp). This ensures the LLM receives contextually relevant examples.
  • Domain-Specific Prompt Design: A structured prompt template is used to guide the LLM. This template includes a "Task Description" to clarify the goal, "Context Examples" retrieved by the above module, and the "Input" reaction (in SMILES format, with IUPAC names for compounds). This helps the LLM focus on the specific details of chemical reactions.
  • Two-stage In-Context Tuning:
    1. In-Context Reaction Tuning: The LLM is fine-tuned using the constructed prompts and context examples. This teaches the model to align reaction representations with textual descriptions.
  • Structure Preference Alignment: To ensure the generated captions have the correct structure and include essential elements (compounds, conditions, results), a Direct Preference Optimization (DPO) approach is used. This involves training the model to prefer captions that adhere to predefined structural rules, using both positive examples and synthetically generated negative examples.

Datasets

The primary dataset used for training and evaluation is the OpenExp dataset. This dataset comprises 274,439 chemical reactions with corresponding captions and experimental procedures, derived from the USPTO-Applications and ORD databases. The training set is used as a local database for retrieving k-shot context examples for in-context tuning.

Key Insights and Ideas

  • Context is King (for Reactions): Simply providing random examples to an LLM isn't enough for complex tasks like reaction captioning. Retrieving similar reactions and their descriptions significantly boosts performance, highlighting the importance of domain-specific context.
  • Structure Matters: Chemical reaction descriptions have a specific, often complex, structure. Standard text generation metrics don't capture this, necessitating specialized techniques like DPO to enforce structural correctness.
  • Bridging the Gap: ReactGPT demonstrates that LLMs can be effectively adapted to understand and generate detailed, structured text for chemical reactions, moving beyond single-molecule descriptions.
  • New Task: Reaction Captioning: The paper formalizes "reaction captioning" as a distinct task, crucial for advancing AI's role in chemical discovery and literature analysis.

Key Results

ReactGPT achieved state-of-the-art performance on both reaction captioning and experimental procedure prediction tasks.

  • Reaction Captioning: ReactGPT outperformed baselines like MolT5, Galactica, and ChemDFM across multiple metrics (BLEU, ROUGE, METEOR), showing significant improvements, particularly in structure success rate (89.53% vs. ~70-83% for others).
  • Experimental Procedure Prediction: ReactGPT also showed strong results, achieving 100% validity and competitive scores on other metrics, matching or exceeding models like ReactXT.
  • Ablation Studies: Removing the retrieval component or the in-context tuning stage led to performance degradation, underscoring the importance of each module. The optimal number of context examples was found to be 2.

Limitations and Future Work

  • Evaluation Metrics: The authors acknowledge that standard metrics like BLEU and ROUGE are not fully sufficient for evaluating the quality of reaction descriptions due to the diverse ways reactions can be described. The "structure success rate" was introduced to address this.
  • LLM Size: For larger LLMs (over 6B parameters), the authors used LoRA (Low-Rank Adaptation) for efficient fine-tuning.
  • Future Work: The paper suggests that ReactGPT's success in aligning chemical reactions with natural language can facilitate better understanding of chemical reactions and promote research in automated chemical synthesis and retrosynthesis. The framework's ability to improve LLM comprehension of chemical reactions is a key takeaway for future AI applications in chemistry.

Summary updated

BibTeX
@inproceedings{chen2025reactgpt,
  author = {Zhe Chen and Zhe Fang and Wenhao Tian and Zhaoguang Long and Changzhi Sun and Yuefeng Chen and Hao Yuan and Honglin Li and Man Lan},
  title = {{ReactGPT: Understanding of Chemical Reactions via In-Context Tuning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i1.31983},
}
Figure