Dual-LLM Adversarial Framework for Information Extraction from Research Literature

Zhijing Li, Yunwen Yu, Wenhao Gu, Tiantian Zhu, Hyunmin Song, Wenbin Guo, Yang Xiao, Zexuan ZhubioRxiv 2025 · DOI

Abstract

Information Extraction (IE) is a fundamental task in Natural Language Processing (NLP) that aims to automatically identify relevant information from unstructured or semi-structured data. Information extraction from lengthy research literature, particularly in multi-omics studies, faces significant challenges due to their complex narratives and extensive context. To address this, we present a novel dual–LLM adversarial framework in which one large language model (LLM) performs the extraction and another provides iterative feedback to refine the results. This process systematically reduces errors, enhances consistency across heterogeneous data sources, and converges toward more accurate outputs. We evaluated our approach against manual and single-LLM extraction, using LLMs as evaluators. Experimental results show that our adversarial framework outperforms these baselines, highlighting its effectiveness for extracting structured information from lengthy scientific texts.

Introduction

The exponential growth of scientific literature has surpassed the capacity of human researchers to manually synthesize and extract critical data. Information Extraction (IE) serves as a vital bridge, converting unstructured text into machine-readable structured formats. However, scientific papers in domains like multi-omics—encompassing genomics, proteomics, and metabolomics—present unique challenges. These documents are often lengthy (typically 40+ pages), contain complex technical narratives, and require long-range context to link experimental data with specific results.

Traditional IE methods often struggle with these complexities, requiring extensive labeled datasets. While modern Large Language Models (LLMs) offer improved context handling, a single-pass extraction often suffers from incompleteness, formatting inconsistencies, or missed logical connections. To overcome these limitations, we propose a self-correcting, dual-LLM adversarial framework designed to enhance extraction reliability without human intervention.

The Dual-LLM Adversarial Framework

Our approach utilizes two distinct LLMs in a collaborative yet adversarial relationship, mimicking the peer-review process common in academia.

Overview of the Dual-LLM Adversarial Framework showing the iterative loop between the Extractor and Verifier.

The framework consists of several key components:

  1. The Extractor (LLM A): This model is responsible for reading the literature and generating structured output according to a "tripartite" schema: Data (inputs), Analyses (methodologies), and Results (biomarkers and metrics).
  2. The Verifier (LLM B): Acting as an adversary, this model reviews the Extractor’s output. It identifies omissions, detects inconsistencies with the schema, and provides targeted feedback.
  3. Iterative Refinement: The Extractor receives the Verifier's feedback and generates a revised version. This loop continues until predefined convergence criteria—such as stability, feedback saturation, or schema consistency—are met.
  4. Multi-LLM Evaluation: To objectively measure performance, a separate LLM-based scoring system grades the extractions on content accuracy and format consistency.

Experimental Setup

The framework was evaluated using a corpus of 42 multi-omics research papers. To establish a "gold standard," each paper was manually annotated by two independent bioinformatics researchers, with a third reviewer resolving any discrepancies. The dataset was split into 21 papers for validation (prompt tuning and model selection) and 21 papers for final performance testing.

Results and Analysis

The Dual-LLM framework demonstrated superior performance compared to both single-pass AI extractions and manual human efforts. On the test set, the proposed framework achieved an average score of 63.27, significantly higher than the single-LLM baseline (56.76) and manual human extraction (44.89). The lower human score highlights the difficulty of maintaining perfect consistency and completeness when manually processing extremely long documents.

Evaluation scores for various Extractor and Verifier LLM combinations.

Key findings from our experiments include:

  • The "Two-Iteration" Sweet Spot: Performance typically peaks at the second iteration. Further iterations often lead to "over-correction" and a slight decline in accuracy.
  • Model Selection Matters: The choice of models for the Extractor and Verifier roles significantly impacts the final score. The researchers found that Qwen3-32B was the most robust evaluator, showing the highest stability and "order invariance."
  • Complexity in Analysis Extraction: The framework showed the most significant improvement in extracting "Analyses." While raw data extraction is relatively straightforward, linking specific datasets to statistical methods requires the iterative reasoning provided by the adversarial loop.

Conclusion

The Dual-LLM Adversarial Framework represents a significant step forward in automated information extraction from complex scientific literature. By leveraging the strengths of two models in an iterative feedback loop, the system achieves higher accuracy and consistency than traditional methods. Future work will focus on improving the Verifier's error-detection capabilities, scaling the framework to larger datasets across diverse scientific domains, and mitigating potential evaluator bias by using more diverse LLM ensembles for scoring.

Summary updated

Abstract

Abstract Information Extraction (IE) is a fundamental task in Natural Language Processing (NLP) that aims to automatically identify relevant information from unstructured or semi-structured data. Information extraction from lengthy research literature, particularly in multi-omics studies, faces significant challenges due to their complex narratives and extensive context. To address this, we present a novel dual–LLM adversarial framework in which one large language model (LLM) performs the extraction and another provides iterative feedback to refine the results. This process systematically reduces errors, enhances consistency across heterogeneous data sources, and converges toward more accurate outputs. We evaluated our approach against manual and single-LLM extraction, using LLMs as evaluators. Experimental results show that our adversarial framework outperforms these baselines, highlighting its effectiveness for extracting structured information from lengthy scientific texts.
BibTeX
@misc{li2025dual,
  author = {Zhijing Li and Yunwen Yu and Wenhao Gu and Tiantian Zhu and Hyunmin Song and Wenbin Guo and Yang Xiao and Zexuan Zhu},
  title = {{Dual-LLM Adversarial Framework for Information Extraction from Research Literature}},
  year = {2025},
  doi = {10.1101/2025.09.11.675507},
  howpublished = {bioRxiv},
}
Figure