Enzyme Co-Scientist: Harnessing Large Language Models for Enzyme Kinetic Data Extraction from Literature

Jinling Jiang, Jie Hu, Shuang Xie, Meng-Hao Guo, Yuhang Dong, Shuai Fu, Xianyue Jiang, Zhihao Yue, Junchao Shi, Xiaoyu Zhang, Minghui Song, Guangyong Chen, Hua Lu, Xindong Wu, Pei Guo, D. H. Han, Zeyi Sun, Jiezhong QiubioRxiv 2025 · DOI

TL;DR

This paper introduces an "agentic workflow" that leverages Large Language Models (LLMs) to automatically extract crucial enzyme kinetic data from scientific literature. This approach significantly speeds up data acquisition and improves accuracy compared to traditional manual curation, paving the way for more efficient scientific discovery.

What Problem Does This Paper Address?

Scientific research relies heavily on understanding molecular behavior, and a key aspect of this is enzyme kinetics – how fast enzymes catalyze reactions under various conditions. This information is scattered across vast amounts of scientific literature, often buried in tables and text. Traditionally, extracting this data involves laborious and error-prone manual curation by human experts. This bottleneck slows down research, limits the scale of data-driven studies, and hinders the development of predictive models. The paper aims to overcome this by automating the extraction of quantitative enzyme kinetic data, specifically parameters like Km, kcat, and kcat/Km, from research papers.

Methods

The core of the paper is an "agentic workflow" designed for automated data extraction. Here's a breakdown:

  1. Literature Collection: The process begins by gathering relevant scientific papers, initially in PDF format.
  2. OCR Pre-Processing: PDFs are converted into machine-readable text using Optical Character Recognition (OCR). The authors compared Mathpix and PyMuPDF, selecting Mathpix for its slightly better performance in extracting protein enzyme data.
  3. Prompt Engineering: This is a crucial step where prompts are meticulously crafted to guide LLMs. The prompts include task descriptions, detailed instructions, and format references (using Markdown for easy visualization). This iterative process involves evaluating prompts on annotated data, identifying errors, and refining instructions.
  4. LLM-based Data Extraction: The processed text, along with the engineered prompt, is fed to multiple LLMs. The paper evaluates several popular LLMs, including closed-source (Claude 3.5, GPT-4o) and open-source (Llama 3, Qwen).
  5. Aggregation Agent: To leverage the strengths of different LLMs and improve overall performance, an aggregation agent is developed. This agent consolidates insights from multiple LLMs, aiming for higher confidence results and reducing errors.
  6. Post-Processing: The raw outputs from LLMs are cleaned and standardized into a structured data frame format using the Pandas library. This includes space cleaning, invalid field handling, numerical cleaning (including scientific notation), and unit conversion to a consistent set of units (mM, s⁻¹, s⁻¹mM⁻¹).

Datasets

The paper introduces two key datasets:

  • Annotated Dataset: A manually curated subset of papers (156 for protein enzymes, 164 for ribozymes) serving as a "gold standard" for evaluating the workflow's components and performance.
  • LLM Enzyme Kinetics Archive: A large-scale dataset containing over 91,000 enzyme kinetic entries extracted from approximately 3,500 protein enzyme papers and 2,420 entries from 164 ribozyme papers. This archive is also proposed as a real-world benchmark for evaluating LLMs' long-context capabilities.

Key Insights and Ideas

  • LLMs as "Co-Scientists": The paper demonstrates that LLMs can act as valuable "co-scientists," automating tedious data extraction tasks and accelerating research.
  • The Power of Aggregation: Combining the outputs of multiple LLMs through an aggregation agent significantly boosts performance, outperforming individual models and even human curation in some aspects. This highlights the benefit of ensemble methods for complex information extraction.
  • LLMs Surpass Human Curation (in some aspects): For the evaluated protein enzyme data, the LLM-based approach, particularly the aggregation agent, achieved higher F1 scores than the BRENDA database, which relies on manual extraction. This suggests LLMs can be more consistent and accurate for certain data extraction tasks.
  • Ribozyme Data Gap Addressed: The workflow is successfully applied to the ribozyme domain, where a comprehensive database was previously lacking, demonstrating its versatility and potential for establishing new knowledge bases.
  • Cost-Effectiveness: The ribozyme extraction was achieved for under $90 and in under 3 days, showcasing the economic and time-saving benefits of the LLM-based approach.

Key Results

  • Protein Enzyme Extraction: The workflow achieved an average F1 score above 0.9 on expert-annotated subsets of protein enzymes. The aggregation agent integrating Claude3.5, GPT-4o, Llama3, and Qwen achieved superior performance.
  • Ribozyme Extraction: The workflow extracted five kinetic parameters for ribozymes with a mean F1 score of 0.75 for Claude3.5. An aggregation of Claude3.5 and Llama3 achieved superior performance in this domain.
  • Dataset Size: The final archive contains over 91,000 enzyme kinetic entries from approximately 3,500 papers.
  • Cost and Time: For ribozymes, the entire process (literature collection, OCR, LLM extraction) cost less than $90 and took less than 3 days.

Limitations and Future Work

  • LLM Performance Variability: While Claude3.5 was a top performer, other LLMs showed distinct strengths in handling challenging cases, underscoring the need for aggregation.
  • Aggregation Strategy: The paper suggests the need for "smarter aggregation agents" that can better identify and discard poorly performing LLMs during ensemble.
  • BRENDA Comparison Nuances: When comparing with BRENDA, the authors had to account for different levels of rounding in manual curation, indicating that direct metric comparison can be complex.
  • Future Work: The authors aim to refine the enzyme archive, facilitate new discoveries, and extend the workflow to other research fields, establishing it as a versatile paradigm and a benchmark for LLM capabilities in scientific research.

Summary updated

Abstract

Abstract The extraction of molecular annotations from scientific literature is critical for advancing data-driven research. However, traditional methods, which primarily rely on human curation, are labor-intensive and error-prone. Here, we present an LLM-based agentic workflow that enables automatic and efficient data extraction from literature with high accuracy. As a demonstration, our workflow successfully delivers a dataset containing over 91,000 enzyme kinetics entries from around 3,500 papers. It achieves an average F1 score above 0.9 on expert-annotated subsets of protein enzymes and can be extended to the ribozyme domain in fewer than 3 days at less than $90. This method opens up new avenues for accelerating the pace of scientific research.
BibTeX
@misc{jiang2025enzyme,
  author = {Jinling Jiang and Jie Hu and Shuang Xie and Meng-Hao Guo and Yuhang Dong and Shuai Fu and Xianyue Jiang and Zhihao Yue and Junchao Shi and Xiaoyu Zhang and Minghui Song and Guangyong Chen and Hua Lu and Xindong Wu and Pei Guo and D. H. Han and Zeyi Sun and Jiezhong Qiu},
  title = {{Enzyme Co-Scientist: Harnessing Large Language Models for Enzyme Kinetic Data Extraction from Literature}},
  year = {2025},
  doi = {10.1101/2025.03.03.641178},
  howpublished = {bioRxiv},
}
Figure