A Survey on Biomedical Text Summarization with Pre-trained Language Model

Qianqian Xie, Zheheng Luo, Benyou Wang, Sophia AnaniadouarXiv 2023 · DOI · arXiv

Abstract

Biomedical Text Summarization (BTS) is a critical task for managing the exponential growth of scientific literature and clinical documentation. This survey explores the transition from traditional neural networks to state-of-the-art Pre-trained Language Models (PLMs) and Large Language Models (LLMs) in the biomedical domain. We categorize existing methodologies, identify key datasets across literature and clinical records, and discuss the unique challenges of factual consistency and domain-specific knowledge in medical AI.

Introduction

The biomedical field faces an "information overload" crisis. With the rapid expansion of scientific databases like PubMed and the increasing complexity of Electronic Health Records (EHRs), clinicians and researchers require automated tools to distill vast amounts of data into actionable insights. Biomedical Text Summarization (BTS) addresses this by generating concise, informative summaries of lengthy documents.

Overview of the survey structure and key components of Biomedical Text Summarization.

Unlike general-domain summarization, BTS must navigate specialized jargon, maintain extreme factual accuracy, and handle sensitive patient data. Summarization typically follows two paradigms: extractive (selecting key sentences) and abstractive (generating new text).

Comparison of extractive and abstractive summarization applied to a radiology report.

Methodology: PLMs and LLMs

Modern BTS strategies are primarily driven by two classes of models:

Pre-trained Language Models (PLMs)

PLMs like BERT, BART, and T5 have revolutionized NLP through self-supervised learning on massive corpora. In the biomedical domain, these models are often adapted through:

  • Domain-Specific Pre-training: Models like BioBERT and PubMedBERT are trained directly on medical text to better capture specialized vocabulary.
  • Fine-tuning: Adapting general-purpose models (e.g., BART) to specific summarization tasks using labeled biomedical datasets.

Large Language Models (LLMs)

The emergence of LLMs like GPT-4 has introduced new capabilities:

  • Zero-shot/Few-shot Learning: LLMs can summarize medical text via prompting without task-specific training.
  • Instruction Tuning: Models like ChatDoctor are fine-tuned to follow complex medical instructions.
  • Synthetic Data Generation: LLMs can create high-quality training data to augment smaller, more efficient models.

Datasets and Evaluation

The survey identifies four primary categories of datasets used in the field:

  1. Biomedical Literature: Long-form articles from PubMed, S2ORC, and CORD-19.
  2. Electronic Health Records (EHRs): Summarizing radiology findings (e.g., MIMIC-CXR, OpenI).
  3. Medical Conversations: Doctor-patient interactions (e.g., HET-MC).
  4. Consumer Health Questions: Distilling patient queries into concise questions (e.g., MeQSum).

Evaluation remains a challenge. While ROUGE is standard, it often fails to capture medical factuality. Newer metrics like CheXbert are specifically designed to assess clinical accuracy.

Key Insights and Results

  • Domain Adaptation: Models pre-trained on biomedical text consistently outperform general models. For instance, KeBioSum achieved a ROUGE-1 of 36.39 on PubMed-long.
  • LLM Competitiveness: In radiology summarization, zero-shot LLMs are approaching the performance of supervised models. ImpressionGPT (ChatGPT-based) achieved a ROUGE-1 of 54.45 on MIMIC-CXR, nearly matching the SOTA FactReranker (55.94).
  • The Factuality Gap: A single missing word (e.g., "no") can change a medical summary's meaning entirely. Ensuring factual consistency is the primary hurdle for clinical adoption.

Future Directions

The field is moving toward Multimodal BTS, integrating text with medical imaging (X-rays, MRIs) and lab results. Additionally, improving the interpretability and controllable generation of these models is essential to mitigate hallucinations and build clinician trust.

Summary updated

Abstract

The exponential growth of biomedical texts such as biomedical literature and electronic health records (EHRs), poses a significant challenge for clinicians and researchers to access clinical information efficiently. To tackle this challenge, biomedical text summarization (BTS) has been proposed as a solution to support clinical information retrieval and management. BTS aims at generating concise summaries that distill key information from single or multiple biomedical documents. In recent years, the rapid advancement of fundamental natural language processing (NLP) techniques, from pre-trained language models (PLMs) to large language models (LLMs), has greatly facilitated the progress of BTS. This growth has led to numerous proposed summarization methods, datasets, and evaluation metrics, raising the need for a comprehensive and up-to-date survey for BTS. In this paper, we present a systematic review of recent advancements in BTS, leveraging cutting-edge NLP techniques from PLMs to LLMs, to help understand the latest progress, challenges, and future directions. We begin by introducing the foundational concepts of BTS, PLMs and LLMs, followed by an in-depth review of available datasets, recent approaches, and evaluation metrics in BTS. We finally discuss existing challenges and promising future directions in the era of LLMs. To facilitate the research community, we line up open resources including available datasets, recent approaches, codes, evaluation metrics, and the leaderboard in a public project: https://github.com/KenZLuo/Biomedical-Text-Summarization-Survey/tree/master. We believe that this survey will be a useful resource to researchers, allowing them to quickly track recent advancements and provide guidelines for future BTS research within the research community.
BibTeX
@misc{xie2023survey,
  author = {Qianqian Xie and Zheheng Luo and Benyou Wang and Sophia Ananiadou},
  title = {{A Survey on Biomedical Text Summarization with Pre-trained Language Model}},
  year = {2023},
  eprint = {2304.08763},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2304.08763},
}
Figure