Cydex: Neural Search Infrastructure for the Scholarly Literature

Shane Ding, Edwin Zhang, Jimmy Lin

Abstract

Cydex is a platform that provides neural search infrastructure for domain-specific scholarly literature. The platform represents an abstraction of Covidex, our recently developed full-stack open-source search engine for the COVID-19 Open Research Dataset (CORD-19) from AI2. While Covidex takes advantage of the latest best practices for keyword search using the popular Lucene search library as well as state-of-the-art neural ranking models using T5, parts of the system were hard coded to only work with CORD-19. This paper describes our efforts to generalize Covidex into Cydex, which can be applied to scholarly literature in different domains. By decoupling corpus-specific configurations from the frontend implementation, we are able to demonstrate the generality of Cydex on two very different corpora: the ACL Anthology and a collection of hydrology abstracts. Our platform is entirely open source and available at cydex.ai.
BibTeX
@inproceedings{ding2020cydex,
  author = {Shane Ding and Edwin Zhang and Jimmy Lin},
  title = {{Cydex: Neural Search Infrastructure for the Scholarly Literature}},
  booktitle = {SDP @ EMNLP},
  year = {2020},
  doi = {10.18653/v1/2020.sdp-1.19},
}
Figure