Using External Resources and Joint Learning for Bigram Weighting in ILP-Based Multi-Document Summarization

Chen Li, Yang Liu, Lin ZhaoNAACL 2015 · DOI

Abstract

Traditional extractive summarization often relies on simple word frequency to determine concept importance. This paper improves the Integer Linear Programming (ILP) framework by incorporating syntactic filtering, leveraging diverse external knowledge sources (Wikipedia, word embeddings, etc.), and employing a joint learning strategy. Evaluated on TAC benchmarks from 2008 to 2011, the proposed system achieves state-of-the-art performance by optimizing bigram weights directly for the final summary quality.

Introduction

Extractive multi-document summarization aims to select a representative subset of sentences from a collection of documents. A widely adopted method uses Integer Linear Programming (ILP) to maximize the coverage of important concepts—typically bigrams—subject to a length constraint. However, standard baselines like the ICSI system weight these bigrams based solely on document frequency. This approach is limited: frequency does not always equate to semantic importance, and the decoupled nature of bigram weighting and sentence selection often leads to sub-optimal summaries.

Proposed Methodology

The authors introduce three significant enhancements to the ILP-based summarization pipeline:

1. Syntactic Bigram Selection

Rather than considering all bigrams that meet a frequency threshold, the system employs Part-of-Speech (POS) tagging and constituent parsing. It prioritizes bigrams from base noun phrases or those containing verbs and adjectives. This filtering process effectively removes "noise" bigrams that lack substantive meaning.

2. External Feature Engineering

To better estimate bigram importance, the model incorporates features from various external resources:

  • Language Models: Bigram distributions are compared against human-authored summaries in the NYT Annotated Corpus.
  • Word Embeddings: word2vec models measure semantic similarity between bigrams and the user's query.
  • Knowledge Bases: Wikipedia (tf-idf), DBpedia (entity descriptions), and WordNet (synonyms) provide broader context.
  • Sentiment Analysis: SentiWordNet captures the emotional charge of the text.

3. Joint Learning with Structured Perceptron

Instead of a traditional pipeline where weights are pre-calculated, the authors use a Structured Perceptron. This allows the model to update feature weights based on the quality of the final summary produced by the ILP solver. By training through the decoder, the weights are optimized for the end goal of summarization rather than an intermediate proxy.

Experimental Results

The system was evaluated on the NIST Text Analysis Conference (TAC) datasets from 2008 through 2011. The task involves generating a 100-word summary from 10 documents based on a specific topic.

Percentage of correct bigrams in the candidate pool for the ICSI baseline vs. the proposed system.

As shown in Figure 1, the syntactic filtering strategy significantly improves the "signal-to-noise" ratio. While it may reduce the total count of bigrams, it increases the percentage of relevant bigrams in the candidate pool, making the ILP optimization more effective.

The final results demonstrate consistent improvements over previous state-of-the-art systems. On the TAC 2011 dataset, the proposed method achieved a ROUGE-2 score of 13.97, outperforming the ICSI baseline (12.71) and other top-ranked systems.

Comparison of ROUGE-2 scores across TAC datasets (2008-2011) for the proposed method against state-of-the-art baselines.

Discussion and Key Insights

  • Semantic vs. Frequency Weighting: Word embeddings emerged as a powerful feature, enabling the system to identify query-relevant concepts even when exact word matches were absent.
  • The Value of Joint Training: The joint model outperformed a standard regression pipeline by 0.2–0.3 ROUGE points. This confirms that learning weights in the context of the final decoding step is crucial for performance.
  • Ablation Trends: While individual resources like DBpedia or SentiWordNet showed modest gains alone, they provided valuable complementary signals when integrated into the full feature set.

Conclusion

By moving beyond simple frequency counts and integrating external semantic knowledge with a joint learning framework, this work sets a new benchmark for ILP-based extractive summarization. Future work may explore deeper semantic structures and document-level understanding to further enhance summary coherence and relevance.

Summary updated

BibTeX
@inproceedings{li2015using,
  author = {Chen Li and Yang Liu and Lin Zhao},
  title = {{Using External Resources and Joint Learning for Bigram Weighting in ILP-Based Multi-Document Summarization}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2015},
  doi = {10.3115/v1/n15-1079},
}
Figure