Jennifer for COVID-19: An NLP-Powered Chatbot Built for the People and by the People to Combat Misinformation

Yunyao Li, Tyrone Grandison, Patricia Silveyra, Ali Douraghy, Xinyu Guan, Thomas Kieselbach, Chengkai Li, Haiqi ZhangNLP-COVID @ ACL 2020 · OpenReview

TL;DR

This paper introduces Jennifer, an NLP-powered chatbot designed to combat COVID-19 misinformation by providing accessible information from reputable sources. Built and maintained by a global volunteer community, Jennifer demonstrates a rapid development approach and a commitment to quality assurance, releasing a new dataset of COVID-19 related questions to aid future research.

What Problem Does This Paper Address?

The COVID-19 pandemic brought with it a surge of misinformation, making it difficult for the public to access reliable information. This paper addresses the challenge of combating this spread of harmful content by creating an easily accessible and trustworthy source of information. The authors aim to understand how public information from reputable sources can be effectively organized and disseminated during a crisis, and to identify the public's most pressing questions.

Methods

Jennifer utilizes the Juji (Juji, 2020) base system for dialog management. When a user asks a question, Juji employs a pre-trained machine learning model to identify relevant questions with known answers and provides either an answer or a follow-up question. The core of Jennifer's capabilities lies in its Question-Answer (QA) pairs, which are either manually curated by volunteers or automatically generated using templates. The system prioritizes rapid development, ease of access across platforms, maintainability by a diverse volunteer group, quality assurance through reputable sources, and extensibility for future growth.

Datasets

The paper introduces COQB-19 (COVID-19 Question Bank), a newly released dataset containing 3,924 COVID-19-related questions organized into 944 groups. This dataset was gathered from Jennifer's users and volunteers, aiming to facilitate future research in this domain.

Key Insights and Ideas

  • Volunteer-driven development is a powerful model for rapid response: Jennifer was designed and released within 24 hours, highlighting the efficiency of mobilizing a global volunteer network for urgent information dissemination.
  • Combating misinformation requires a multi-platform approach: Jennifer's availability on the web, Facebook, and embedded in fact-checking systems demonstrates the need to reach users where they are.
  • User-generated questions are a valuable resource for understanding public concerns: The creation of the COQB-19 dataset underscores the importance of capturing and analyzing public queries to tailor information delivery.
  • Quality assurance is paramount for trust: The emphasis on using reputable sources and rigorous processes is crucial for a chatbot intended to combat misinformation.

Key Results

As of June 18, 2020, Jennifer had handled 1,480 questions (excluding menu selections) and successfully answered 1,059 of them, achieving a response rate of 71%. The average engagement duration was 3 minutes and 15 seconds. The system was developed and released within 24 hours of its conception.

Limitations and Future Work

The paper acknowledges that Jennifer is a proof-of-principle system. Future work would likely involve expanding the QA dataset, improving the NLP capabilities for more nuanced understanding and response generation, and potentially integrating more sophisticated methods for identifying and flagging misinformation. The authors also suggest that understanding the public's immediate curiosities is an ongoing area of exploration.

Summary updated

Abstract

Just as SARS-CoV-2, a new form of coronavirus continues to infect a growing number of people around the world, harmful misinformation about the outbreak also continues to spread. With the goal of combating misinformation, we designed and built Jennifer – a chatbot maintained by a global group of volunteers. With Jennifer, we hope to learn whether public information from reputable sources could be more effectively organized and shared in the wake of a crisis as well as to understand issues that the public were most immediately curious about. In this paper, we introduce Jennifer and describe the design of this proof-of-principle system. We also present lessons learned and discuss open challenges. Finally, to facilitate future research, we release COQB-19 (COVID-19 Question Bank, available at https://www.newvoicesnasem.org/data-downloads), a dataset of 3,924 COVID-19-related questions in 944 groups, gathered from our users and volunteers. Jennifer is available at http://bit.ly/jenniferai and on Facebook at http://fb.me/JenniferCOVIDAI.
BibTeX
@inproceedings{li2020jennifer,
  author = {Yunyao Li and Tyrone Grandison and Patricia Silveyra and Ali Douraghy and Xinyu Guan and Thomas Kieselbach and Chengkai Li and Haiqi Zhang},
  title = {{Jennifer for COVID-19: An NLP-Powered Chatbot Built for the People and by the People to Combat Misinformation}},
  booktitle = {NLP-COVID @ ACL},
  year = {2020},
  url = {https://openreview.net/forum?id=HxIZzQZy_0F},
}
Figure