Agentic induction of common data elements and information extraction from sarcoma case report abstracts

Daniel Reichenpfader, Kwaku Appiagyei, Manovah Nadarajah, Ahmed Reda Bahr, Irina Filchenko, Fabio DennstädtarXiv 2026 · arXiv

This paper presents an end-to-end agentic workflow for automatically generating clinical data schemas and extracting information from medical literature. By applying this to a large corpus of sarcoma case report abstracts, the authors demonstrate that large language models (LLMs) can move beyond simple data extraction to "bottom-up" schema design, though human expert curation remains a critical final step.

TL;DR

The authors developed an LLM-based agentic pipeline that automatically induces a Common Data Element (CDE) schema from ~25,000 sarcoma abstracts and uses it to extract structured data with 94.9% accuracy. While the extraction is highly reliable, the study highlights that agent-generated schemas still require human expert review to resolve redundancies and conceptual errors.

What Problem Does This Paper Address?

In clinical research, Common Data Elements (CDEs) are standardized questions and response formats used to ensure data comparability across studies. Traditionally, these are defined "top-down" by panels of experts—a process that is slow, expensive, and prone to missing emergent concepts found in actual clinical practice, especially for rare diseases like sarcomas.

Sarcomas are highly heterogeneous, and much of our knowledge about them is buried in thousands of unstructured case reports. This paper investigates whether LLM agents can perform "agentic induction"—reading the literature to propose a schema (bottom-up) and then populating that schema at scale.

Methods

The researchers built a multi-stage workflow using the gpt-oss-120b model and the AG2 agent framework. The process, illustrated in Figure 1, consists of two main phases:

  1. Corpus-driven Schema Induction:
    • Proposal: An agent reads each abstract and proposes potential CDEs (Question, Type, and Value).
  2. Consolidation: The system uses embedding-based similarity searches and agentic reasoning to merge thousands of raw proposals into a set of "master" CDEs.
  3. Standardization: A dedicated agent cleans up categorical value sets (e.g., normalizing "Male" and "M").
  4. Fixed-schema Extraction:
    • The final schema (retaining CDEs supported by at least 50 abstracts) is applied to the entire corpus.
  5. Crucially, the model must provide an evidence span (a direct quote from the text) for every extracted value to ensure traceability and reduce hallucinations.
Illustrative overview of the study workflow, showing schema construction (a) and data extraction (b).

Datasets

  • PubMed: The authors retrieved 58,265 records using sarcoma-related MeSH terms.
  • Sarcoma Case Report Corpus: After filtering for single-patient human case reports with abstracts, they settled on a final dataset of 24,577 unique abstracts.
  • Evaluation Set: A random sample of 100 abstracts was used for rigorous human expert validation of the extraction accuracy.

Key Insights and Ideas

  • Agentic vs. Direct Inference: The authors found that agents are best suited for the "creative" task of schema design (planning and tool use), while direct, constrained LLM calls are more efficient for high-volume data extraction.
  • The Frequency vs. Importance Trade-off: The pipeline initially proposed over 72,000 CDEs, but only 53 met the threshold of being mentioned in 50+ abstracts. This highlights a challenge in rare disease research: frequent data points are easy to automate, but "insular criteria" (rare but critical clinical features) might be filtered out by frequency-based thresholds.
  • Traceability as a Safeguard: By requiring the model to link every answer to a specific text span, the authors created a built-in verification mechanism. Errors were often caught when the model couldn't find a matching span for its proposed answer.
  • Schema Quality vs. Extraction Accuracy: Interestingly, human experts were much more critical of the schema (the questions themselves) than the extraction (the answers). This suggests that LLMs are currently better at following a schema than designing a perfect one from scratch.

Key Results

The workflow's progression from raw proposals to a refined schema is detailed in Figure 2.

Flowchart of the corpus construction and CDE schema induction process.
  • Extraction Performance: In the 100-abstract validation set, the LLM achieved 94.9% exact agreement with human experts for non-empty values.
  • Schema Induction: The agent successfully consolidated 72,909 raw proposals into 1,570 master CDEs.
  • Expert Review: Two medical professionals approved 83.0% of the retained CDEs for relevance, but only 56.6% for correctness, citing issues like overlapping fields (e.g., multiple CDEs for patient age).
  • Data Coverage: As shown in Figure 3, coverage varied significantly. "Patient Sex" was extracted in 80.6% of cases, while more specific data like "Tumor Dimensions" appeared in only 10.5%.
Coverage of the 53 retained CDEs across the full corpus of 24,577 abstracts.

Limitations and Future Work

  • Abstract Limitations: Abstracts often omit granular details like specific chemotherapy dosages or long-term follow-up data, which are only available in full-text reports.
  • Semantic Overlap: Despite two consolidation passes, the agent still produced redundant CDEs. Future iterations need better logic for identifying when two differently worded questions ask for the same clinical variable.
  • Complex Relations: The model struggled with anatomical locations and temporal concepts (e.g., distinguishing between the primary tumor location and a metastatic site).
  • Human-in-the-loop: The authors conclude that the most viable path forward is an agent-assisted approach: let the AI draft the schema and perform the bulk extraction, but keep experts in the loop to curate the schema before it is deployed at scale.

Summary updated

Abstract

Common data elements (CDEs) support clinical data comparability but are usually defined manually. We investigated whether large language model agents could derive a CDE schema from sarcoma case report abstracts and apply it at scale. An agent proposed 72,909 candidate CDEs from 24,588 PubMed abstracts; agentic consolidation yielded 1,570 CDEs. We retained 53 supported by at least 50 abstracts and attempted evidence-linked extraction across the corpus. Processing succeeded for all retained CDEs in 79.2% of abstracts, including responses indicating non-mention. Two human reviewers approved 83.0% of retained CDEs for relevance, 67.9% for clarity, and 56.6% for correctness. In 100 sampled abstracts, exact agreement with the value extracted by the LLM was 94.9% among 1,116 pairs with a non-empty answer (abstract-level bootstrap 95% CI, 93.0-96.4%). Errors concentrated in anatomical and temporal fields. These findings support agent-assisted schema drafting followed by expert curation before large-scale extraction.
BibTeX
@misc{reichenpfader2026agentic,
  author = {Daniel Reichenpfader and Kwaku Appiagyei and Manovah Nadarajah and Ahmed Reda Bahr and Irina Filchenko and Fabio Dennstädt},
  title = {{Agentic induction of common data elements and information extraction from sarcoma case report abstracts}},
  year = {2026},
  eprint = {2611.18506},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18506},
}
Figure