Enhancing Study-Level Inference from Clinical Trial Papers via Reinforcement Learning-Based Numeric Reasoning

Massimiliano Pronesti, Michela Lorandi, Paul Flanagan, Oisín Redmond, Anya Belz, Yufang HouEMNLP 2025

TL;DR

This paper tackles the challenge of automatically extracting crucial numerical data from clinical trial papers to synthesize evidence for systematic reviews. They introduce a novel framework that moves beyond simple text retrieval, using reinforcement learning to train models that perform quantitative reasoning, leading to significant improvements in accuracy and interpretability.

What Problem Does This Paper Address?

Systematic reviews are vital for evidence-based medicine, but automating the process of extracting and synthesizing information from numerous clinical trial papers is a major hurdle. A key bottleneck is accurately extracting numerical evidence (like event counts or standard deviations) and then using this data to derive study-level conclusions. Previous approaches often relied on shallow textual cues, failing to capture the complex numerical reasoning that human experts use. This paper aims to bridge this gap by developing a system that directly models this quantitative reasoning process.

Methods

The authors propose a two-component system:

  1. Numeric Data Extraction Model: This model is trained to extract structured numerical evidence from clinical trial papers. They explore various training strategies, including supervised fine-tuning (SFT) and, crucially, Reinforcement Learning (RL). The RL approach uses a novel value-based reward model that incorporates correctness, format adherence, and reasoning trace quality.
  2. Effect Estimate Component: This component takes the extracted numerical data and applies domain knowledge-informed logic to derive outcome-specific conclusions, mimicking how experts would calculate effect sizes and confidence intervals.

The core novelty lies in framing the problem as quantitative reasoning rather than just textual inference and employing RL with sophisticated reward functions to align model behavior with expert principles.

Datasets

  • COCHRANEFORESTEXT: A newly created training dataset consisting of 2,072 examples from 104 systematic reviews, with 25.9 million tokens of full-text biomedical content. This dataset includes study text, outcome type, numerical values, point estimates, confidence intervals, and conclusions.
  • COCHRANEFOREST: Used for evaluation, this dataset contains 725 instances from 48 Cochrane Systematic Reviews.
  • RCTs (Yun et al., 2024): Also used for evaluation, this dataset comprises 413 complete, human-annotated instances from 120 RCTs.

Key Insights and Ideas

  • Quantitative Reasoning is Key: Simply retrieving relevant text snippets isn't enough; models need to understand and perform numerical reasoning to accurately synthesize evidence.
  • Reinforcement Learning for Precision: RL, particularly with carefully designed rewards, significantly enhances a model's ability to extract precise numerical data and align with expert conclusions, outperforming even large pretrained models.
  • Structured Output Matters: The system's ability to output structured numerical data (like YAML) is crucial for downstream meta-analysis and interpretability.
  • Reasoning Traces Improve Performance: Training with intermediate reasoning steps (like Chain-of-Thought) during SFT leads to better performance.
  • Data Modalities are Complementary: While tabular data is highly informative, combining it with textual context yields the best results.

Key Results

  • The best-performing RL-trained model achieved up to a 21% absolute improvement in F1 score compared to retrieval-based systems on the COCHRANEFOREST benchmark.
  • The RL model outperformed general-purpose LLMs (over 400B parameters) by up to 9% on the RCTs dataset.
  • The RL model demonstrated a significantly lower rate of hallucinated reasoning (3.79% vs. 15.15% for SFT), indicating improved factual grounding.
  • Ablation studies showed that models trained only on tables performed best among data ablations, but combining text and tables was superior.
  • The RL training dynamics revealed that structural aspects (format, thought process) are learned quickly, while numerical correctness improves more gradually.

Limitations and Future Work

  • Missing Values and Inconsistent Formats: The current system assumes all relevant numerical evidence is present and cleanly extractable. It doesn't explicitly handle missing values or format inconsistencies, which are common in real-world clinical studies.
  • Limited Outcome Types and Rules: The approach currently focuses on a limited set of outcome types and uses predefined logical rules, which might limit generalization to more complex scenarios.

Future work should focus on training models to detect and report missing values, handling a wider range of outcome types, and developing more robust generalization capabilities for real-world systematic review automation.

Summary updated

BibTeX
@inproceedings{pronesti2025enhancing,
  author = {Massimiliano Pronesti and Michela Lorandi and Paul Flanagan and Oisín Redmond and Anya Belz and Yufang Hou},
  title = {{Enhancing Study-Level Inference from Clinical Trial Papers via Reinforcement Learning-Based Numeric Reasoning}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure