ACTA 2.0: A Modular Architecture for Multi-Layer Argumentative Analysis of Clinical Trials

Benjamin Molinet, Santiago Marro, Elena Cabrio, Serena Villata, Tobias MayerIJCAI 2022 · DOI

TL;DR

This paper introduces ACTA 2.0, an enhanced system for automatically analyzing clinical trial abstracts to extract arguments and their relationships. It goes beyond previous work by not only identifying claims and evidence but also classifying their argumentative function (support/attack) and analyzing the effects of interventions on outcomes, making evidence-based medicine more accessible.

What Problem Does This Paper Address?

Evidence-based medicine relies on synthesizing vast amounts of clinical trial data to inform patient care. However, manually extracting and evaluating the arguments within these trials is time-consuming and complex. Existing tools often focus on specific aspects, like identifying PICO (Patient, Intervention, Comparison, Outcome) elements or retrieving evidence, but lack a comprehensive approach to understanding the argumentative structure and the nuanced effects of interventions. This paper addresses the need for a more sophisticated, automated system that can dissect the argumentative core of clinical trial abstracts and provide actionable insights for clinicians and researchers.

Methods

ACTA 2.0 builds upon its predecessor by introducing several key advancements. At its core, it employs Argument Mining techniques to identify argument components (claims and premises) within clinical trial abstracts. A significant upgrade is the overhauled relation classification module, which now not only detects relationships between argument components but also labels them as either "support" or "attack." This is achieved using a pre-trained SciBERT model fine-tuned with a GRU and CRF for sequence classification.

Another major addition is the Effects on Outcomes module. This module first detects outcome elements within the text and then predicts the effect of an intervention on these outcomes (e.g., "Increased," "Decreased," "Improved"). This is also implemented using a two-step pipeline involving sequence tagging for outcome detection and sequence classification for effect prediction, leveraging SciBERT, GRU, and CRF.

Finally, ACTA 2.0 has been refactored into a modular architecture. This allows individual processing steps (argument component detection, relation classification, PICO and effect prediction) to be accessed independently via a REST API, promoting versatility and enabling integration into other systems.

Datasets

The paper mentions the use of the following datasets:

  • AbstRCT dataset: Used for argumentative analysis and PICO element detection. It contains 500 abstracts of randomized controlled trials on neoplasm treatment, annotated with claims, premises, and their relations.
  • EBM-NLP dataset: Used for PICO element detection.
  • Outcome dataset: Derived from the AbstRCT dataset, containing 3351 sentences annotated with five classes for outcome effects (Improved, Increased, Decreased, NoDifference, NoOccurrence).

Key Insights and Ideas

  • Argumentative Nuance: Moving beyond simply identifying claims and evidence, ACTA 2.0's ability to classify relations as "support" or "attack" provides a richer understanding of how arguments are constructed and contested within scientific literature.
  • Actionable Outcome Analysis: The explicit prediction of intervention effects on outcomes is a crucial step towards making clinical trial data more directly applicable to decision-making, bridging the gap between raw findings and clinical practice.
  • Modularity for Integration: The refactoring into a modular architecture with a REST API is a significant practical advancement, allowing ACTA 2.0's powerful analytical capabilities to be seamlessly integrated into existing research workflows and tools.
  • Bridging the Gap: ACTA 2.0 aims to democratize access to complex clinical trial information by automating the extraction and analysis of argumentative structures and intervention effects, supporting evidence-based medicine.

Key Results

  • Argumentative Analysis: The model achieves an F1-score of 85.2 on the neoplasm test set for argumentative component detection. For relation classification, it achieves macro F1-scores of 0.68, 0.70, and 0.70 for Neoplasm, Glaucoma, and Mixed test sets, respectively.
  • PICO Element Detection: The system achieves an F1-score of 73.4 for PICO element detection.
  • Effects on Outcomes: The combined outcome detection and effect classification tasks reach a macro F1-score of 0.80.

Limitations and Future Work

  • Limitations: The paper acknowledges that the current system focuses on abstracts and does not analyze full clinical trial papers. The relation classification module's performance varies across different test sets.
  • Future Work: The authors suggest several promising directions:
  • Automatic prediction of the link between detected outcomes and related interventions.
  • Analysis of multiple abstracts together to identify relationships between arguments across different studies.
  • Further integration of ACTA 2.0's services into other systems.

Summary updated

BibTeX
@inproceedings{molinet2022acta,
  author = {Benjamin Molinet and Santiago Marro and Elena Cabrio and Serena Villata and Tobias Mayer},
  title = {{ACTA 2.0: A Modular Architecture for Multi-Layer Argumentative Analysis of Clinical Trials}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/859},
}
Figure