A Hybrid Approach to Generation of Missing Abstracts in Biomedical Literature

Suchet Chachra, Asma Ben Abacha, Sonya Shooshan, Laritza Rodriguez, Dina Demner-FushmanCOLING 2016

TL;DR

This paper tackles the challenge of missing abstracts in biomedical literature by proposing a novel hybrid summarization technique. By combining health outcome detection, keyphrase extraction, and textual entailment, the approach generates extractive summaries for articles lacking author-written abstracts, significantly improving recall and F1 scores compared to single-method baselines.

What Problem Does This Paper Address?

Biomedical literature is a vast and rapidly growing field, with millions of scientific articles published annually. Abstracts serve as crucial entry points for researchers to quickly assess the relevance of an article. However, a significant number of articles, particularly within large repositories like PubMed Central (PMC), are missing these author-generated abstracts. This forces researchers to rely solely on titles, which can be unreliable, or to read the full text, which is time-consuming. The paper addresses this gap by developing an automated method to generate informative extractive summaries for these abstract-less articles, thereby improving information retrieval and accessibility.

Methods

The authors propose a hybrid approach that combines three main components:

  • Health Outcome Identification (HO): This method leverages an existing identifier to extract sentences likely to contain health outcomes. It uses an ensemble of classifiers (rule-based, Naïve Bayes, n-gram, positional, document-length, and semantic) to compute a likelihood score for each sentence.
  • Keyphrase Extraction (KEA): This component identifies salient sentences based on the presence of keywords and multi-word keyphrases. Two methods are explored:
  • KEA (Keyphrase Extraction Algorithm): A well-established algorithm using a Naïve Bayes classifier and a discretization scheme to identify keyphrases.
  • Microsoft Text Analytics (MSTA): A cloud-based service for extracting keyphrases.
  • A normalization step is applied to keywords to remove redundant or overlapping phrases.
  • Textual Entailment Recognition (RTE): This is a core innovation. The system uses a feature-based classifier (SimSet) trained on the SNLI corpus to determine if one sentence entails another. This allows the system to understand the logical relationships between sentences.
  • Hybrid Approach: The system first identifies candidate sentences using the HO and KEA methods. Then, it uses textual entailment to traverse a graph of sentence relationships. This traversal aims to identify sentences that are directly or indirectly entailed by the initial candidate sentences, prioritizing those that are more informative or shorter, depending on the specific traversal strategy (Hybrid MinLength, Hybrid MaxLength, Hybrid MaxScore). This process enriches the initial summary by incorporating logically connected but potentially missed sentences.

Datasets

  • SNLI Corpus: Used for training the Textual Entailment Recognition (RTE) classifier. This corpus contains approximately 570,000 sentence pairs annotated with entailment, contradiction, or neutral labels.
  • Evaluation Dataset: A newly created dataset of 295 biomedical articles from the open-access subset of PMC that lack author-generated abstracts. For these articles, the authors manually generated "golden summaries" (reference extractive summaries) by having two domain experts (a clinician and a medical librarian) select key sentences. This dataset is made publicly available.

Key Insights and Ideas

  • The Power of Combination: The most significant insight is that combining multiple, diverse summarization techniques (health outcome detection, keyphrase extraction, and textual entailment) yields superior results compared to any single method. This highlights the complexity of capturing the essence of a scientific article and the need for multi-faceted approaches.
  • Textual Entailment as a Bridge: Recognizing the logical flow and inferential relationships between sentences through textual entailment is crucial for enriching summaries. It allows the system to go beyond simple keyword or topic matching and identify sentences that logically follow from or support key points.
  • Manual Annotation is Laborious but Essential: The creation of the "golden summaries" underscores the difficulty and time-consuming nature of manual summarization, justifying the need for automated solutions. The availability of this dataset is a valuable contribution to the field.

Key Results

  • The hybrid approach demonstrated substantial improvements over baseline methods.
  • Recall: The hybrid approach achieved a 13% improvement in recall.
  • F1 Score: The hybrid approach showed a 4% improvement in F1 score.
  • Qualitative evaluation by human annotators indicated that the hybrid summaries were generally more informative and captured the bottom-line better than individual baseline systems.
  • The paper also noted a low overlap between summaries generated by the HO and KEA systems, suggesting that a hybrid approach would indeed be more comprehensive.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the hybrid approach generally produces longer summaries than baseline systems, even though it improves recall.
  • The textual entailment classifier was trained on a general-purpose corpus (SNLI) and then adapted for biomedical text by collapsing "contradiction" and "neutral" labels. While effective, it might not capture all nuances specific to biomedical language.
  • Future Work:
    • Named Entity Recognition (NER): The authors plan to incorporate NER to identify and extract named entities (e.g., genes, proteins, diseases) and use them as additional keywords to improve precision.
  • Multi-Document Summarization: They aim to apply a similar hybrid approach to multi-document summarization, which is a more complex task.

Summary updated

BibTeX
@inproceedings{chachra2016hybrid,
  author = {Suchet Chachra and Asma Ben Abacha and Sonya Shooshan and Laritza Rodriguez and Dina Demner-Fushman},
  title = {{A Hybrid Approach to Generation of Missing Abstracts in Biomedical Literature}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2016},
}
Figure