Keywords-Guided Abstractive Sentence Summarization

Haoran Li, Junnan Zhu, Jiajun Zhang, Chengqing Zong, Xiaodong HeAAAI 2020 · DOI

Introduction

Abstractive sentence summarization aims to generate a concise version of a source sentence while potentially introducing new vocabulary. A significant challenge for standard sequence-to-sequence (seq2seq) models is distinguishing the core "gist" of a sentence from secondary details. Humans typically approach this task by identifying key terms first and then constructing a grammatical summary around them. This paper proposes a framework that explicitly incorporates keyword guidance into both the encoding and decoding phases to bridge this gap.

Methodology

The proposed architecture utilizes a multi-task learning framework to jointly perform keyword extraction and abstractive summarization. It introduces three primary innovations:

  1. Keywords-Guided Selective Encoding: A "co-selective gate" filters the input sentence based on identified keywords, allowing the encoder to focus on relevant semantic fragments and suppress "noise" before the information reaches the decoder.
  2. Dual-Attention Generator: The decoder employs two separate attention mechanisms—one for the original source sentence and another for the extracted keywords. These are integrated using a Hierarchical Fusion strategy to create a final context vector.
  3. Dual-Copy Mechanism: An extension of the pointer-generator network that allows the model to copy words from either the source sentence or the keyword list, or generate them from the vocabulary.
Architecture of the Keywords-Guided Abstractive Summarization model, featuring the Keyword Extractor, Co-Selective Gate, and Dual-Attention/Copy mechanisms.

To facilitate training without manual keyword labels, the authors use a heuristic where non-stop-words appearing in both the source and reference summary are treated as ground-truth keywords.

Experimental Results

The model was evaluated on the English Gigaword dataset, a benchmark containing approximately 3.8 million training pairs. The results demonstrate that the proposed Co-Selective+Hier+DualPG model achieves state-of-the-art performance, reaching a ROUGE-1 score of 47.14 and a ROUGE-2 score of 25.06.

Impact of Multi-Task Learning and Fusion Strategies

A key finding is the synergy between keyword extraction and summarization. As shown in the performance comparison, Multi-Task Learning (MTL) consistently outperforms Task-Specific Learning (TSL) across various fusion methods (Concatenation, Gating, and Hierarchical Fusion).

Comparison of ROUGE-1 and ROUGE-2 scores across different fusion strategies (Concat, Gate, Hier) and learning paradigms (TSL vs. MTL).

The hierarchical fusion strategy (Hier) provides the best results, suggesting that a structured approach to combining source and keyword information is superior to simple concatenation or gating. Furthermore, an "oracle" test using ground-truth keywords during inference showed a ~20% improvement in ROUGE-2, highlighting that the primary bottleneck lies in the accuracy of keyword identification rather than the generation logic itself.

Conclusion

By mimicking the human process of identifying key information before generation, the keywords-guided framework significantly improves abstractive summarization quality. While the model currently relies on word-overlap heuristics for keyword identification, future work may focus on refining extraction through more advanced techniques like reinforcement learning to better handle synonyms and paraphrasing.

Summary updated

BibTeX
@inproceedings{li2020keywords,
  author = {Haoran Li and Junnan Zhu and Jiajun Zhang and Chengqing Zong and Xiaodong He},
  title = {{Keywords-Guided Abstractive Sentence Summarization}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i05.6333},
}
Figure