ExcavatorCovid: Extracting Events and Relations from Text Corpora for Temporal and Causal Analysis for COVID-19

Bonan Min, Benjamin Rozonoyer, Haoling Qiu, Alexander Zamanian, Jessica MacBride

TL;DR

This paper introduces ExcavatorCovid, a system that extracts COVID-19 related events and their temporal and causal relationships from news and scientific articles. It then constructs a Temporal and Causal Analysis Graph (TCAG) to help policymakers and analysts understand the complex dynamics of the pandemic and its impacts.

What Problem Does This Paper Address?

The COVID-19 pandemic generated an overwhelming volume of information, making it incredibly difficult for policymakers to grasp the full picture of events, their causes, and their consequences in a timely manner. This information overload hinders effective decision-making and response strategies. The paper aims to alleviate this by providing a structured, analytical view of the pandemic's unfolding narrative.

Methods

ExcavatorCovid employs a multi-stage pipeline:

  • Event Taxonomy: A comprehensive taxonomy of 76 COVID-19 related event types was developed through a hybrid approach of manual curation and automated clustering of verb and noun phrases from news articles.
  • Event Extraction: A BERT-based sequence tagging model is used to identify event triggers and classify their types. It also extracts location and time arguments for each event.
  • Relation Extraction: Two approaches are combined: a pattern-based method and a neural network model (also BERT-based) to extract temporal and causal relations between events.
  • TCAG Construction: All extracted events and relations are aggregated into a Temporal and Causal Analysis Graph (TCAG). This graph is visualized in an interactive web interface.
  • Event Popularity Measurement: A time-series analysis of event popularity is performed, normalizing raw counts by the total number of articles published to account for media activity fluctuations.

Datasets

  • Training Data:
    • A portion (10%) of the Aylien Coronavirus News Dataset was used for event extraction training data curation.
  • The ACE 2005 dataset was used for related event-argument annotation (specifically for location and time arguments).
  • Simple and Complex Cause Assertion Annotation datasets and 1.5K temporal relation instances generated by applying LearnIt patterns to sampled Gigaword articles were used for relation extraction training.
  • Corpora for TCAG Construction:
    • Aylien Coronavirus News Dataset (1.2 million articles, focusing on January-May 2020).
  • COVID-19 Open Research Dataset (CORD-19), containing research from PubMed, WHO, bioRxiv, and medRxiv.

Key Insights and Ideas

  • The Power of Structured Information: The paper highlights how transforming unstructured text into a structured graph (TCAG) can reveal complex interdependencies and downstream effects that are otherwise obscured by the sheer volume of information.
  • Bridging News and Research: Combining news articles (rich in real-world events) with research papers (rich in causal relationships) provides a more comprehensive understanding of the pandemic.
  • Beyond Simple Event Tracking: The system goes beyond just identifying events; it actively models their causal and temporal relationships, enabling deeper analysis of how events influence each other.
  • Utility Beyond the Pandemic: The authors emphasize that the Excavator system's ability to build and analyze TCAGs for complex, evolving situations has long-term value for understanding and solving future problems.

Key Results

  • Excavator extracted 6.2 million event mentions of 59 types from the combined corpora.
  • The event extraction models achieved 83% accuracy on a manually reviewed sample.
  • Excavator extracted 226,176 causal and temporal relations.
  • The precision for extracted relations was reported as 0.78 for Causes, 0.87 for Mitigates, and 0.81 for Before.
  • The interactive TCAG visualization allows users to explore events, their causes, and impacts, with examples showing how lockdown policies correlate with economic crises and shortages.

Limitations and Future Work

  • Event Taxonomy Granularity: While the taxonomy is comprehensive, the paper notes that finer-grained distinctions at the sub-type level for relations were difficult to extract accurately and were merged into broader types.
  • Data Coverage: The analysis is limited by the availability and scope of the news and research datasets used.
  • Future Work: The authors suggest that the system's utility extends beyond COVID-19, implying its applicability to other complex, evolving domains. They also mention making the curated dataset publicly available, which could facilitate further research.

Summary updated

Abstract

Timely responses from policy makers to mitigate the impact of the COVID-19 pandemic rely on a comprehensive grasp of events, their causes, and their impacts. These events are reported at such a speed and scale as to be overwhelming. In this paper, we present ExcavatorCovid, a machine reading system that ingests open-source text documents (e.g., news and scientific publications), extracts COVID19 related events and relations between them, and builds a Temporal and Causal Analysis Graph (TCAG). Excavator will help government agencies alleviate the information overload, understand likely downstream effects of political and economic decisions and events related to the pandemic, and respond in a timely manner to mitigate the impact of COVID-19. We expect the utility of Excavator to outlive the COVID-19 pandemic: analysts and decision makers will be empowered by Excavator to better understand and solve complex problems in the future. An interactive TCAG visualization is available at http://afrl402.bbn.com:5050/index.html. We also released a demonstration video at https://vimeo.com/528619007.
BibTeX
@inproceedings{min2021excavatorcovid,
  author = {Bonan Min and Benjamin Rozonoyer and Haoling Qiu and Alexander Zamanian and Jessica MacBride},
  title = {{ExcavatorCovid: Extracting Events and Relations from Text Corpora for Temporal and Causal Analysis for COVID-19}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=JwraeEcmsKs},
}
Figure