PubMed and Beyond: Biomedical Literature Search in the Age of Artificial Intelligence

Qiao Jin, Robert Leaman, Zhiyong LuarXiv 2023 · DOI · arXiv

TL;DR

With over one million papers added to PubMed annually, traditional keyword searching is no longer enough to keep up with biomedical literature. This paper surveys 36 specialized, publicly available search tools categorized into five distinct scenarios—such as precision medicine and literature mining—and provides a forward-looking perspective on how Large Language Models (LLMs) are transforming literature retrieval through retrieval-augmented generation.

What Problem Does This Paper Address?

Biomedical research is experiencing an exponential growth phase; PubMed currently hosts nearly 36 million articles. However, a major bottleneck exists: a typical query retrieves hundreds or thousands of results, but users rarely look past the top 20.

Standard keyword-based search engines struggle with specialized information needs. For instance, clinicians practicing evidence-based medicine need to filter results by the quality of clinical evidence, while genomic researchers face massive synonymy challenges (e.g., a single genetic variant can be written in multiple ways like "V600E", "Val600Glu", or "rs113488022"). This paper addresses the awareness and functionality gaps by mapping out specialized search tools designed to handle these complex, domain-specific tasks.

Methods

The authors introduce a scenario-oriented classification system to organize 36 web-based, freely available, and regularly maintained tools. They break down biomedical literature search into five specialized scenarios:

  1. Evidence-based Medicine (EBM): Tools that accept structured "PICO" (Population, Intervention, Comparison, Outcome) queries and prioritize high-quality evidence like systematic reviews over individual case reports.
  2. Precision Medicine & Genomics: Engines that use advanced text-mining to automatically normalize variant synonyms and map gene-disease relationships.
  3. Semantic Search: Systems that leverage vector embeddings to search by conceptual meaning rather than exact word matches, allowing users to ask questions in natural language.
  4. Literature Recommendation: Tools that use machine learning or citation graphs to suggest papers similar to a "seed" list of articles.
  5. Literature Mining: Platforms that apply Natural Language Processing (NLP) for Named Entity Recognition (NER) and Relation Extraction (RE) to construct interactive knowledge graphs from text.
Overview of the five specialized biomedical search scenarios

The paper also reviews the emerging integration of LLMs (like ChatGPT) with traditional search engines, specifically focusing on Retrieval-Augmented Generation (RAG), where a search engine finds relevant documents and an LLM synthesizes them into a direct answer.

Datasets

As a comprehensive survey, this paper does not introduce a new dataset. Instead, it highlights the primary data sources and benchmarks utilized across the 36 tools:

  • Core Corpora: PubMed, PubMed Central (PMC) for full-text search, and Europe PMC.
  • Genomic Databases: Manually curated databases such as UniProt, dbSNP, ClinVar, and Gene Ontology.
  • Specialized Datasets: CORD-19 (COVID-19 Open Research Dataset) for pandemic-specific tools, and the BioASQ challenge datasets used to benchmark biomedical question-answering systems.

Key Insights and Ideas

  • The Five-Word Limit: Traditional keyword search efficiency drops significantly when queries exceed five words, underscoring the necessity of semantic search for natural language questions.
  • Structuring Queries via PICO: For clinical questions, explicitly separating terms into PICO elements yields vastly superior precision compared to dumping keywords into a single search box.
The architecture of an Evidence-Based Medicine (EBM) search engine
  • Automated Synonym Normalization: In genomics, relying on keywords means missing crucial papers due to variant naming variations. Specialized tools like LitVar use background text-mining tools (tmVar) to catch all synonyms automatically.
  • RAG is Powerful but Risky: Experimental tools like Elicit and Consensus show that LLM-driven retrieval augmentation can directly answer research questions, but users must remain vigilant against precision and recall errors (hallucinations).

Key Results

  • LitCovid Superiority: The survey highlights that LitCovid, a specialized topic-based recommendation hub, identifies 30% more relevant PubMed articles regarding COVID-19 than a complex, manually formulated Boolean query.
  • User Behavior Constraints: The authors note that fewer than 20% of articles past the top 20 results are ever reviewed by users, validating why PubMed shifted its default sorting from reverse-chronological order to the machine-learning-based "Best Match" relevance ranking.

Limitations and Future Work

  • The "Black Box" of Recommendations: A major shortcoming of current article-based recommendation systems is their inability to explain why a specific paper is being recommended—a critical issue in high-stakes medical domains.
  • Fragmented Ecosystem: The tools are currently scattered across different web portals. The authors emphasize the need for a unified portal that can automatically triage a user's query to the correct specialized search backend.
  • LLM Evaluation: While LLMs excel at synthesis, they still do not outperform smaller, task-specific models (like fine-tuned BERT variants) on core information extraction tasks like Named Entity Recognition. Future work must focus on improving LLM verifiability before they can be trusted for rigorous tasks like systematic review screening.

Summary updated

Abstract

Biomedical research yields a wealth of information, much of which is only accessible through the literature. Consequently, literature search is an essential tool for building on prior knowledge in clinical and biomedical research. Although recent improvements in artificial intelligence have expanded functionality beyond keyword-based search, these advances may be unfamiliar to clinicians and researchers. In response, we present a survey of literature search tools tailored to both general and specific information needs in biomedicine, with the objective of helping readers efficiently fulfill their information needs. We first examine the widely used PubMed search engine, discussing recent improvements and continued challenges. We then describe literature search tools catering to five specific information needs: 1. Identifying high-quality clinical research for evidence-based medicine. 2. Retrieving gene-related information for precision medicine and genomics. 3. Searching by meaning, including natural language questions. 4. Locating related articles with literature recommendation. 5. Mining literature to discover associations between concepts such as diseases and genetic variants. Additionally, we cover practical considerations and best practices for choosing and using these tools. Finally, we provide a perspective on the future of literature search engines, considering recent breakthroughs in large language models such as ChatGPT. In summary, our survey provides a comprehensive view of biomedical literature search functionalities with 36 publicly available tools.
BibTeX
@misc{jin2023pubmed,
  author = {Qiao Jin and Robert Leaman and Zhiyong Lu},
  title = {{PubMed and Beyond: Biomedical Literature Search in the Age of Artificial Intelligence}},
  year = {2023},
  eprint = {2307.09683},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2307.09683},
}
Figure