Analyzing the vast coronavirus literature with CoronaCentral

Jake Lever, Russ B. Altman

TL;DR

The COVID-19 pandemic has led to an explosion of research, making it impossible for scientists to keep up. CoronaCentral is a new machine learning-powered resource that categorizes and analyzes this vast literature, including papers on SARS-CoV and MERS-CoV, to help researchers navigate the information landscape. It integrates topic and article type classification with Altmetric data to highlight trending research and societal impacts.

What Problem Does This Paper Address?

The sheer volume of scientific publications generated during the COVID-19 pandemic presents a significant challenge for researchers. Keeping abreast of new findings, understanding research trends, and identifying relevant papers is nearly impossible with traditional search methods. Existing tools often have limited scope, fewer categories, or lack the ability to analyze the pace and emphasis of research, especially concerning societal impacts and emerging topics like "long COVID." This paper aims to solve this by providing a comprehensive, machine learning-driven system for organizing and analyzing the coronavirus literature.

Methods

The authors developed CoronaCentral using a supervised machine learning approach. They annotated over 3,200 articles with 32 topics and 8 article types. A BERT-based document multilabel classification model was employed for this task. Beyond this core ML component, they also incorporated simple rule-based methods for identifying certain topics and article types, such as clinical trials and retractions. The system integrates data from PubMed and CORD-19, and also includes papers on SARS-CoV and MERS-CoV. A key innovation is the integration of Altmetric data, which measures online attention and social media impact, to help identify highly discussed papers.

Datasets

The primary datasets used for training and evaluation are the CORD-19 literature dataset and PubMed. The authors also included papers related to SARS-CoV and MERS-CoV to provide a broader historical context. They annotated over 3,200 articles for their supervised learning model. The CoronaCentral resource itself, containing categorized articles and extracted entities, is available on Zenodo.

Key Insights and Ideas

  • The "Long COVID" phenomenon is a rapidly growing area: While initially underrepresented in training data, the system identified papers on "long COVID" and noted a steady increase in publications since April 2020.
  • Preprint servers are crucial but not dominant: While preprint servers like arXiv and medRxiv are important for rapid dissemination, they account for only a small fraction (5.8%) of all SARS-CoV-2 articles.
  • Topic-specific communication strategies: Different scientific topics are communicated through distinct channels. For example, forecasting and modeling papers are predominantly original research, while health policy articles are often commentary or editorials.
  • Altmetric data highlights public interest: The top 100 papers by Altmetric score show a different distribution of topics compared to the overall literature, emphasizing research on treatments and prevention methods.
  • Traditional citation metrics are insufficient: The rapid pace of the pandemic necessitates new methods for identifying important research, as long-term citation counts are too slow to be effective.

Key Results

CoronaCentral covers 128,921 papers as of March 3, 2021. The machine learning model achieved a micro-F1 score of 0.68, with a micro-precision of 0.76 and micro-recall of 0.62 for topic and article type classification. The system identified "Clinical Reports" as the top topic, followed by "Effect on Medical Specialties." It also successfully categorized emerging topics like "long COVID" and "inequality." The analysis revealed that approximately 24.7% of publications are comments or editorials, not original research.

Limitations and Future Work

The authors acknowledge that the performance of their machine learning model varies by topic, with some topics (e.g., "long haul") performing poorly likely due to low representation in the test set. They plan to continue monitoring trending articles to identify new topics and ensure the machine learning quality does not degrade as the pandemic evolves. Future work could involve further refining the topic categorization and expanding the resource to include more nuanced analyses of research trends and societal impacts.

Summary updated

Abstract

The SARS-CoV-2 pandemic has caused a surge in research exploring all aspects of the virus and its effects on human health. The overwhelming publication rate means that researchers are unable to keep abreast of the literature. To ameliorate this, we present the CoronaCentral resource that uses machine learning to process the research literature on SARS-CoV-2 together with SARS-CoV and MERS-CoV. We categorize the literature into useful topics and article types and enable analysis of the contents, pace, and emphasis of research during the crisis with integration of Altmetric data. These topics include therapeutics, disease forecasting, as well as growing areas such as "long COVID" and studies of inequality. This resource, available at https://coronacentral.ai, is updated daily.
BibTeX
@inproceedings{lever2021analyzing,
  author = {Jake Lever and Russ B. Altman},
  title = {{Analyzing the vast coronavirus literature with CoronaCentral}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2021},
  doi = {10.1073/pnas.2100766118},
}
Figure