TL;DR
This paper introduces Jennifer, an NLP-powered chatbot designed to combat COVID-19 misinformation by providing accessible information from reputable sources. Built and maintained by a global volunteer community, Jennifer demonstrates a rapid development approach and a commitment to quality assurance, releasing a new dataset of COVID-19 related questions to aid future research.
What Problem Does This Paper Address?
The COVID-19 pandemic brought with it a surge of misinformation, making it difficult for the public to access reliable information. This paper addresses the challenge of combating this spread of harmful content by creating an easily accessible and trustworthy source of information. The authors aim to understand how public information from reputable sources can be effectively organized and disseminated during a crisis, and to identify the public's most pressing questions.
Methods
Jennifer utilizes the Juji (Juji, 2020) base system for dialog management. When a user asks a question, Juji employs a pre-trained machine learning model to identify relevant questions with known answers and provides either an answer or a follow-up question. The core of Jennifer's capabilities lies in its Question-Answer (QA) pairs, which are either manually curated by volunteers or automatically generated using templates. The system prioritizes rapid development, ease of access across platforms, maintainability by a diverse volunteer group, quality assurance through reputable sources, and extensibility for future growth.
Datasets
The paper introduces COQB-19 (COVID-19 Question Bank), a newly released dataset containing 3,924 COVID-19-related questions organized into 944 groups. This dataset was gathered from Jennifer's users and volunteers, aiming to facilitate future research in this domain.
Key Insights and Ideas
- Volunteer-driven development is a powerful model for rapid response: Jennifer was designed and released within 24 hours, highlighting the efficiency of mobilizing a global volunteer network for urgent information dissemination.
- Combating misinformation requires a multi-platform approach: Jennifer's availability on the web, Facebook, and embedded in fact-checking systems demonstrates the need to reach users where they are.
- User-generated questions are a valuable resource for understanding public concerns: The creation of the COQB-19 dataset underscores the importance of capturing and analyzing public queries to tailor information delivery.
- Quality assurance is paramount for trust: The emphasis on using reputable sources and rigorous processes is crucial for a chatbot intended to combat misinformation.
Key Results
As of June 18, 2020, Jennifer had handled 1,480 questions (excluding menu selections) and successfully answered 1,059 of them, achieving a response rate of 71%. The average engagement duration was 3 minutes and 15 seconds. The system was developed and released within 24 hours of its conception.
Limitations and Future Work
The paper acknowledges that Jennifer is a proof-of-principle system. Future work would likely involve expanding the QA dataset, improving the NLP capabilities for more nuanced understanding and response generation, and potentially integrating more sophisticated methods for identifying and flagging misinformation. The authors also suggest that understanding the public's immediate curiosities is an ongoing area of exploration.
Summary updated