Quantifying Online News Media Coverage of the COVID-19 Pandemic: Text Mining Study and Resource

Konrad Krawczyk, Tadeusz Chełkowski, Daniel J. Laydon, Swapnil Mishra, Denise Xifara, Benjamin Gibert, Seth Flaxman, Thomas A. Mellan, Veit Schwämmle, Richard Röttger, Johannes Thorling Hadsund, Samir BhattJournal of Medical Internet Research 2021 · DOI

TL;DR

This study quantifies the massive volume of online news coverage of COVID-19 in 2020, revealing it constituted a quarter of all news articles. While overall sentiment wasn't overwhelmingly negative, a significant portion of coverage, particularly around topics like death and crisis, was highly negative, suggesting a potential for information overload and emotional pressure on the public.

What Problem Does This Paper Address?

The COVID-19 pandemic necessitated widespread public adherence to non-pharmaceutical interventions (NPIs) like social distancing and mask-wearing. The effectiveness of these NPIs hinges on public understanding of the risks and guidelines, which is heavily influenced by media coverage. However, the sheer volume and potential emotional tone of this coverage could lead to information overload, hindering effective communication and public response. This paper aims to quantify the extent and sentiment of online news coverage of COVID-19 to provide a data-driven basis for informing future public health communication strategies.

Methods

The researchers employed a text mining approach to analyze a vast dataset of online news articles. Key methods include:

  • Data Collection: They collected approximately 26 million news articles from the front pages of 172 major online news sources across 11 countries, covering the period from January to October 2020. This dataset was made publicly available.
  • Topic Detection: Using a keyword-based approach, they identified articles related to COVID-19 and other reference topics (e.g., politicians, climate, sport) by analyzing article titles and descriptions.
  • Sentiment Analysis: The Vader (Valence Aware Dictionary and sEntiment Reasoner) tool was used to determine the emotional polarity (positive, negative, neutral) of the news articles. A compound score was generated for each article based on its title and description.
  • Relative Sentiment Skew: To contextualize the sentiment of COVID-19 coverage, they calculated a "relative sentiment skew" metric. This metric compares the average sentiment of COVID-19 articles to the average sentiment of all other articles within the same online news source, helping to determine if COVID-19 coverage was more polarized than general news.
  • Subtopic Analysis: Further analysis was conducted on COVID-19 related articles to identify leading subtopics and their associated sentiment polarizations.

Datasets

The primary dataset used in this study consists of 26,077,939 news articles collected from the front pages of 172 major online news sources across 11 countries (United States, United Kingdom, Canada, Australia, New Zealand, Ireland, Germany, France, Italy, Spain, and Russia), plus an international category. The data covers the period from January to October 2020, with pre-2020 articles also collected for fine-tuning. This dataset has been made publicly available via SciRide.

Key Insights and Ideas

  • Information Overload is Real: The sheer volume of COVID-19 news (25.3% of all front-page articles) suggests a significant risk of information overload, potentially making it harder for the public to discern crucial information.
  • Heterogeneous Sentiment: Contrary to initial assumptions, overall COVID-19 coverage wasn't uniformly negative. This heterogeneity suggests a complex interplay of reporting, not just a singular focus on the negative aspects of the pandemic.
  • The Power of "Negative" Subtopics: While overall sentiment was mixed, specific subtopics like "death," "fear," and "crisis" were strongly and negatively polarized. These accounted for a substantial portion (16%) of highly negative coverage, indicating their potential to significantly shape public perception and emotional response.
  • Context Matters for Sentiment: The study highlights the importance of contextualizing sentiment. For instance, the word "positive" in "positive coronavirus test" could be misclassified as positive sentiment by a naive tool, necessitating careful handling of context-specific language.
  • COVID-19 Coverage vs. Other Topics: COVID-19 coverage was significantly more prevalent than that of even major political figures, underscoring its dominant role in the media landscape of 2020.

Key Results

  • Dominant Coverage: COVID-19 accounted for approximately 25.3% of all front-page online news articles between January and October 2020.
  • Sentiment Nuance: Overall COVID-19 coverage was not exclusively negatively polarized, indicating a heterogeneous reporting landscape.
  • Highly Negative Coverage: 16% of COVID-19 news articles (or 4% of all English-language articles) were classified as highly negatively polarized, focusing on themes like death, fear, or crisis.
  • Subtopic Dominance: The top three COVID-19 subtopics by coverage were "case" (9.29%), "lockdown" (8.56%), and "death" (8.08%).
  • Polarization of "Death": Among these top subtopics, only "death" showed a firmly polarized negative sentiment, while "case" and "lockdown" were not significantly skewed.
  • Impact of Negative Subtopics: Removing articles mentioning "fear," "crisis," and "death" from the analysis led to a statistically significant shift towards more positive sentiment in news reporting, highlighting the influence of these specific negative themes.

Limitations and Future Work

  • Limited Scope of Data: The study focused on front-page news articles from major online sources. This excludes content from social media, blogs, and other less prominent online outlets, which could offer a more complete picture of information dissemination.
  • Keyword-Based Topic Identification: While effective for precision, the keyword-based topic identification method may have underestimated the total coverage of COVID-19 by missing more subtle references.
  • Nuance in Sentiment Analysis: While Vader is a robust tool, it may not capture all the complex nuances of human emotion and sentiment in text.
  • Future Work: The authors suggest that future research should explore the extent to which the quantity and quality of information from different communication channels influence public understanding of public health measures. They also emphasize the need to establish how this information impacts public adherence to NPIs and overall pandemic response. The creation of a publicly available dataset is a significant contribution for future research.

Summary updated

Abstract

The goal of COVID-19 public health communication is to increase understanding of distancing rules and to maximize the impact of governmental policy. The extent to which the quantity and quality of information from different communication channels (eg, social media, government pages, and news) influence public understanding of public health measures remains to be established. Here we conclude that a quarter of all reporting in 2020 covered COVID-19, which is indicative of information overload. In this capacity, our data and analysis form a quantitative basis for informing health communication strategies along traditional news media channels to minimize the risks of COVID-19 while vaccination is rolled out.
BibTeX
@article{krawczyk2021quantifying,
  author = {Konrad Krawczyk and Tadeusz Chełkowski and Daniel J. Laydon and Swapnil Mishra and Denise Xifara and Benjamin Gibert and Seth Flaxman and Thomas A. Mellan and Veit Schwämmle and Richard Röttger and Johannes Thorling Hadsund and Samir Bhatt},
  title = {{Quantifying Online News Media Coverage of the COVID-19 Pandemic: Text Mining Study and Resource}},
  journal = {Journal of Medical Internet Research},
  year = {2021},
  doi = {10.2196/28253},
}
Figure