Keyphrase Extraction from Scientific Articles via Extractive Summarization

Chrysovalantis Giorgos Kontoulis, Eirini Papagiannopoulou, Grigorios TsoumakasNAACL 2021

TL;DR

This paper explores how to improve automatic keyphrase extraction from scientific articles by using extractive summarization on the full text. They found that summaries can offer a better balance of keyphrase richness and noise reduction compared to just using titles/abstracts or the entire full text, leading to improved performance for unsupervised keyphrase extraction methods.

What Problem Does This Paper Address?

Extracting keyphrases from scientific articles is crucial for tasks like information retrieval and document classification, as keyphrases provide a concise representation of the content. However, a key challenge is deciding which part of a scientific article to feed into these extraction methods. Titles and abstracts are dense with keyphrases but might miss important nuances. Full texts are richer but also much noisier. This paper investigates whether extractive summarization of the full text can bridge this gap, offering a more balanced input for keyphrase extraction.

Methods

The authors propose using extractive summarization models to generate summaries of scientific articles. Specifically, they leverage a pre-trained distilled RoBERTa model from the TransformerSum library for this summarization task. They then use these generated summaries as an input to various keyphrase extraction methods.

Beyond extractive summaries, they also explore other input types:

  • Title + Abstract (TA): The standard input for many deep learning models.
  • Full-text (F): The entire document.
  • First 3 Paragraphs (3P220, 3P400): Varying lengths of the initial sections of the paper.
  • Extractive Summary (TS): The generated summary.
  • Abstract + Extractive Summary (AS): An ensemble approach combining the abstract with the generated summary.

They evaluate three keyphrase extraction methods:

  • TfIdf: A baseline statistical method.
  • MultipartiteRank (MR): A graph-based method.
  • Bi-LSTM-CRF (BLC): A strong neural model.

Datasets

The study uses three well-known datasets containing full-text scientific articles from the computer science domain:

  • SemEval: 244 documents.
  • NUS: 211 documents.
  • ACM: 2304 documents.

The train and test sets of the SemEval dataset were merged for their experiments.

Key Insights and Ideas

  • The "Sweet Spot" of Summaries: Extractive summaries appear to strike a good balance. They contain more keyphrases than titles/abstracts but are less noisy than the full text, suggesting they can provide a richer yet cleaner input for keyphrase extraction.
  • Unsupervised Methods Benefit Most: Unsupervised keyphrase extraction methods (TfIdf and MR) showed significant improvements when using extractive summaries (TS) as input, outperforming the conventional TA and F inputs.
  • Deep Learning Models' Challenges: While extractive summaries showed promise, the Bi-LSTM-CRF (BLC) model didn't see significant improvement when using summaries compared to titles and abstracts. This might be due to the training data (abstracts) differing significantly in structure from the summaries. However, the "Abstract U Extractive Summary" (AS) approach did show promise for BLC, suggesting that combining information can be beneficial.
  • Sentence-Level Processing for RNNs: For RNN-based models like BLC, processing abstracts split into sentences (ABSE) yielded better results than the entire abstract, indicating limitations in retaining information from longer sequences.
  • Early Document Sections are Informative but Noisy: The first few paragraphs (3P220, 3P400) contain many keyphrases but are also noisy, which can hinder performance for some methods.

Key Results

  • Unsupervised Gains: For TfIdf and MR, using extractive summaries (TS) generally led to higher F1 scores (both exact and partial matches) across all three datasets compared to using only titles/abstracts (TA) or the full text (F). For example, on the SemEval dataset, TfIdf achieved an F1 score of 0.162*+ with TS, compared to 0.143 with TA and 0.135 with F.
  • BLC Performance: The BLC model showed mixed results. While TS didn't consistently improve performance over TA, the AS (Abstract U Extractive Summary) approach showed better results than TA for BLC on some datasets.
  • Extractive Summary Generation Time: Generating extractive summaries took an average of 2.13 to 2.34 seconds per document, which is considered acceptable for offline applications.

Limitations and Future Work

  • BLC Training Data Mismatch: The BLC model was trained on abstracts, which might not generalize perfectly to extractive summaries, potentially masking the full benefit of the summarization approach for this specific model.
  • Moderate Exact Match Gains: While improvements were observed, the gains in exact match F1 scores were described as moderate. The authors note that this is not necessarily a small performance gain, as exact match scores are inherently low due to their strictness, and a moderate increase can represent a significant percentage improvement.
  • Future Directions:
    • Experiment with a wider range of summarization techniques, including abstractive methods and combinations of abstractive and extractive approaches.
  • Explore more recent and state-of-the-art keyphrase extraction methods, particularly those leveraging contextual embeddings.
  • Investigate multi-task learning approaches that combine summarization and keyphrase extraction.

Summary updated

BibTeX
@inproceedings{kontoulis2021keyphrase,
  author = {Chrysovalantis Giorgos Kontoulis and Eirini Papagiannopoulou and Grigorios Tsoumakas},
  title = {{Keyphrase Extraction from Scientific Articles via Extractive Summarization}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure