Abstract
Since late 2019, COVID-19 has quickly emerged as the newest biomedical domain, resulting in a surge of new information. As with other emergent domains, the discussion surrounding the topic has been rapidly changing, leading to the spread of misinformation. This has created the need for a public space for users to ask questions and receive credible, scientific answers. To fulfill this need, we turn to the task of open-domain question-answering, which we can use to efficiently find answers to free-text questions from a large set of documents. In this work, we present such a system for the emergent domain of COVID-19. Despite the small data size available, we are able to successfully train the system to retrieve answers from a large-scale corpus of published COVID-19 scientific papers. Furthermore, we incorporate effective re-ranking and question-answering techniques, such as document diversity and multiple answer spans. Our open-domain question-answering system can further act as a model for the quick development of similar systems that can be adapted and modified for other developing emergent domains.
The Challenge of Emergent Domains
When a global crisis like COVID-19 hits, the scientific community produces a "torrent" of information—over 100,000 articles in 2020 alone. For the public and researchers alike, finding specific, credible answers in this haystack is difficult, which often facilitates the rapid spread of misinformation.
Standard Open-Domain Question-Answering (ODQA) systems, which find answers to natural language questions from massive document collections, usually require enormous datasets to train effectively. However, "emergent domains" suffer from a "cold start" problem: there isn't enough labeled data to train deep learning models from scratch. This research addresses how to build a high-performing QA system when you have a massive corpus of text (like the CORD-19 dataset) but very few actual question-answer pairs to learn from.
A Hybrid Pipeline for Retrieval
The proposed system utilizes a two-stage pipeline: Retrieval (finding relevant documents) and Reading Comprehension (extracting the specific answer).
Instead of relying solely on "dense retrieval," which understands semantics but can miss specific technical jargon, the researchers implemented a hybrid model. It combines a dense retriever based on PubMedBERT with BM25, a classic keyword-matching algorithm. This ensures the system captures both the general meaning of a query and specific medical terms like "anosmia." This hybrid approach achieved a Fuzzy Matching score of 0.607 at the top 50 documents, significantly outperforming either method used in isolation.
Ensuring Information Diversity
In emergent domains, different papers might offer different theories or findings. To prevent the system from returning redundant information, the researchers introduced a diversity-focused re-ranking strategy using K-Means clustering.
By clustering retrieved documents and selecting the top-ranked papers from each cluster, the system acts as a research assistant that presents a diverse range of scientific perspectives rather than a simple list of similar results.
Reading Comprehension and Multiple Answer Spans
For the extraction phase, the system uses BioBERT, fine-tuned on both general QA data (SQuAD 2.0) and a small set of COVID-specific data (COVID-QA). This "warm-starting" process allows the model to leverage general knowledge before specializing in biomedical terminology.
One of the most impactful features of the reader is its ability to extract multiple non-contiguous answer spans. Medical answers, such as lists of symptoms or treatments, are rarely found in a single string of text. By allowing the model to pick multiple spans, the system's performance jumped from an F1 score of 66.67 to 72.03.
Conclusion
This framework provides a blueprint for the rapid development of QA systems in future emergent domains, whether they be new pandemics or evolving fields like climate science. By combining hybrid retrieval with diversity re-ranking and multi-span extraction, the system successfully bridges the gap between massive document corpora and the need for precise, credible answers.
Summary updated