Vapur: A Search Engine to Find Related Protein-Compound Pairs in COVID-19 Literature

Abdullatif Köksal, Hilal Dönmez, Rıza Özçelik, Elif Ozkirimli, Arzucan ÖzgürarXiv 2020 · arXiv

TL;DR

Vapur is a specialized search engine designed to help researchers navigate the overwhelming sea of COVID-19 literature by focusing on the biochemical relationships between proteins and compounds. By using a BioBERT-powered pipeline to extract and index these specific interactions, Vapur allows scientists to move beyond simple keyword searches and directly explore how biomolecules are related in the latest research.

What Problem Does This Paper Address?

The COVID-19 pandemic triggered an unprecedented surge in scientific publishing, with tens of thousands of papers appearing in just a few months. For medicinal chemists and biologists, finding specific information—such as which compounds inhibit a particular viral protein—is like finding a needle in a haystack.

General-purpose search engines often fail here because they don't understand domain-specific synonyms (e.g., "ACE2" vs. "Angiotensin-converting enzyme 2") or the semantic context of a sentence. A researcher doesn't just want to find papers that mention a protein and a drug; they want to find papers where those two entities actually interact. Vapur addresses this by transforming unstructured text into a structured, relation-oriented index.

Methods

The authors developed a sophisticated BioNLP pipeline to process the CORD-19 dataset, as illustrated in the workflow below:

Figure 1: The workflow of the pipeline behind Vapur. We first split CORD-19 abstracts to sentences and use BERN to detect and normalize the entities in the text. We then identify the biochemical relations with the relation extraction model that we trained and reform the output as an inverted index of relations. Vapur leverages this inverted index to retrieve relevant publications to the query as categorized by related entities.
  1. Named Entity Recognition (NER) & Normalization: The system uses BERN, a state-of-the-art tool that identifies proteins and chemicals and "normalizes" them. This means it maps different names for the same molecule to a single unique ID, ensuring that a search for "IL1B" also finds results for "Interleukin-1b."
  2. Relation Extraction: This is the core innovation. The team fine-tuned BioBERT (a version of BERT trained on biomedical text) to act as a binary classifier. Given a sentence containing a protein and a chemical, the model decides if they are biochemically related.
  3. Relation-Oriented Inverted Index: Instead of indexing words, Vapur indexes triplets: (Entity 1, Entity 2, Publication). This allows the search engine to group results by the specific molecules related to the user's query.
  4. Graph-Based Discovery: Vapur represents its index as a bipartite graph. By applying the SimRank algorithm, the engine can suggest "similar" molecules—not necessarily those that look alike structurally, but those that appear in similar biochemical contexts in the literature.

Datasets

  • CORD-19: The primary dataset for indexing, consisting of over 143,000 unique abstracts related to COVID-19 and SARS-CoV-2.
  • ChemProt: A manually annotated dataset of chemical-protein interactions used to train and evaluate the relation extraction model. It includes 11 categories of relations (e.g., "inhibitor," "substrate"), which the authors simplified into a binary "related vs. not related" task to increase the model's flexibility.

Key Insights and Ideas

  • Binary vs. Multi-class Classification: By training the model to simply identify if a relation exists rather than what type it is, the authors made Vapur more robust. This allows the system to flag potentially important interactions that might not fit neatly into standard categories like "agonist" or "antagonist."
  • Context is King: The use of BioBERT was crucial. Standard BERT, trained on general text like Wikipedia, struggles with the nuances of scientific language. BioBERT’s domain-specific pre-training allowed it to significantly outperform the base model in identifying valid relations.
  • Beyond Search to Discovery: The integration of SimRank allows for serendipitous discovery. A researcher searching for "Favipiravir" might be presented with "Examorelin" as a contextually similar molecule, potentially opening new avenues for drug repurposing research.

Key Results

Vapur demonstrated strong performance both in automated benchmarks and expert reviews:

  • Model Performance: The BioBERT-based relation extraction model achieved an F1-score of 0.777, significantly outperforming the standard BERT model (0.733).
  • Expert Validation: Six domain experts tested the system with 30 unique queries. They found that 90% of the top search results were biochemically relevant to their query.
  • Usability: 73% of the extracted sentences provided by Vapur were deemed useful for research, and experts gave the tool a perfect 5/5 score for ease of use.

The interface provides a clear summary of these relations, as shown in this search scenario:

Figure 2: A search scenario on Vapur. When Favipiravir is entered as the query, Vapur shows the query and its unique ids in 1. It lists contextually similar biomolecules to Favipiravir in 2. It displays related biomolecules found by the relation extraction model in 3. For example, the first result is RNA-dependent RNA polymerase for Favipiravir query. Finally, it shows sentences in related papers to provide information about the relations as in 4.

Limitations and Future Work

While successful, the authors noted that the system's accuracy is heavily dependent on the initial NER step. If BERN misidentifies a protein sequence fragment (like "carboxyl") as a compound, the relation extraction model will produce a "false positive" relation.

Currently, Vapur only indexes abstracts. The authors plan to expand the system to include full-text papers, which will provide a much richer set of relations. They also aim to improve the robustness of the NER and normalization components to reduce errors in entity assignment. Finally, while Vapur is currently focused on COVID-19, the underlying pipeline is designed to be adaptable to any biomedical domain.

Summary updated

Abstract

Coronavirus Disease of 2019 (COVID-19) created dire consequences globally and triggered an intense scientific effort from different domains. The resulting publications created a huge text collection in which finding the studies related to a biomolecule of interest is challenging for general purpose search engines because the publications are rich in domain specific terminology. Here, we present Vapur: an online COVID-19 search engine specifically designed to find related protein - chemical pairs. Vapur is empowered with a relation-oriented inverted index that is able to retrieve and group studies for a query biomolecule with respect to its related entities. The inverted index of Vapur is automatically created with a BioNLP pipeline and integrated with an online user interface. The online interface is designed for the smooth traversal of the current literature by domain researchers and is publicly available at https://tabilab.cmpe.boun.edu.tr/vapur/ .
BibTeX
@misc{koksal2020vapur,
  author = {Abdullatif Köksal and Hilal Dönmez and Rıza Özçelik and Elif Ozkirimli and Arzucan Özgür},
  title = {{Vapur: A Search Engine to Find Related Protein-Compound Pairs in COVID-19 Literature}},
  year = {2020},
  eprint = {2009.02526},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2009.02526},
}
Figure