Accelerating COVID-19 Research with Graph Mining and Transformer-Based Learning

Ilya Tyagin, Ankit Kulshrestha, Justin Sybrandt, Krish Matta, Michael Shtutman, Ilya SafroAAAI 2022 · DOI

Abstract

The rapid expansion of scientific literature during the COVID-19 pandemic created significant challenges for researchers attempting to synthesize new findings. This paper introduces AGATHA-C and AGATHA-GP, next-generation hypothesis generation (HG) systems that leverage massive semantic graphs and Transformer models to uncover implicit connections across millions of research papers. By mining these relationships, the systems successfully predicted the therapeutic potential of compounds like oxytocin and melatonin for COVID-19 prior to their widespread clinical reporting.

Introduction

The COVID-19 pandemic triggered an "infodemic," with thousands of papers published weekly. This volume makes it nearly impossible for researchers to maintain a comprehensive view of the field, leading to "information silos" where critical discoveries in one domain remain unknown to another. Traditional HG systems, designed to bridge these gaps, often struggle with the scale and domain-specific nuances of COVID-19 data. They are frequently limited by narrow vocabularies and an inability to handle both micro-scale (e.g., drug-protein interactions) and macro-scale (e.g., public health trends) queries simultaneously.

Methods and Data

The researchers developed a multi-stage pipeline to transform raw text into a searchable semantic space.

Semantic Graph Construction

The system builds a heterogeneous graph where nodes represent sentences, entities, "Coded Terms" (from UMLS), n-grams, and predicates (subject-verb-object triplets). To process this data, the team utilized the CORD-19 and MEDLINE datasets, employing standardized medical vocabularies like MeSH for concept tagging.

The AGATHA pipeline for processing biomedical literature and extracting predicates.

Model Architecture

Two specialized models were developed:

  • AGATHA-C (Micro): Optimized for molecular-scale discovery, such as drug-gene relationships.
  • AGATHA-GP (Macro): Utilizes deep-learning-based Open Information Extraction (Open IE) to handle general, macroscopic queries.

Both models use a Transformer encoder to rank the plausibility of connections between terms based on their surrounding predicates. Instead of simple co-occurrence, the system learns high-dimensional embeddings that capture the nature of relationships.

Results

The AGATHA systems demonstrated significant quantitative improvements over baseline models, with Precision-Recall AUC increasing by more than 0.1 in several subdomains.

Proactive Discovery

The system's predictive power was validated using a "cut-off" strategy, training on data available up to a specific date and testing against subsequent discoveries. Notable successes include:

  • Oxytocin: Ranked in the top 1.4% of potential COVID-19 connections; later confirmed to have anti-inflammatory effects.
  • Melatonin: Ranked in the top 5.6%; later confirmed to assist with neurological complications.
  • BST2 (Tetherin) Gene: Ranked in the top 7.6%; later found to restrict viral particle secretion.
Distribution of AGATHA scores for the COVID-19 and Oxytocin relationship, showing high predictive plausibility.

Discussion and Limitations

While AGATHA represents a significant step forward, it faces challenges common in HG systems. The "positive research bias" in literature—where negative results are rarely published—makes it difficult for the model to learn what constitutes an unlikely hypothesis. Additionally, while the system currently focuses on abstracts, future work will explore full-text extraction to capture even richer semantic data, despite the increased computational cost. Improving the interpretability of the "plausibility scores" remains a primary goal for future iterations.

Summary updated

BibTeX
@inproceedings{tyagin2022accelerating,
  author = {Ilya Tyagin and Ankit Kulshrestha and Justin Sybrandt and Krish Matta and Michael Shtutman and Ilya Safro},
  title = {{Accelerating COVID-19 Research with Graph Mining and Transformer-Based Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i11.21543},
}
Figure