Abstractive Compression of Captions with Attentive Recurrent Neural Networks

Sander Wubben, Emiel Krahmer, Antal van den Bosch, Suzan Verberne

Introduction

Sentence compression is the task of shortening a sentence while retaining its core meaning and grammatical correctness. Traditionally, research has focused on extractive methods, which simplify sentences by deleting non-essential words. However, human summarization is naturally abstractive: we reorder words, substitute synonyms, and rewrite phrases to create concise, fluent summaries.

This paper addresses the challenges of abstractive compression, specifically the scarcity of large-scale parallel datasets and the complexity of modeling flexible reordering. The authors propose using image captions as a proxy for summarization data and introduce an attentive neural architecture that significantly outperforms traditional translation-based baselines.

Methodology

The authors propose an Attentive Recurrent Neural Network (aRNN) for the compression task. The model utilizes a sequence-to-sequence framework with a bidirectional Long Short-Term Memory (bLSTM) encoder and decoder.

The aRNN architecture featuring a multi-layer bidirectional LSTM encoder-decoder with attention mechanisms.

Unlike standard RNNs that compress an entire input into a single fixed-length vector, the attention mechanism allows the decoder to dynamically "look back" at specific parts of the source sentence while generating each word of the summary. This ensures that important details are preserved even in longer sentences. The model is compared against a Phrase-Based Machine Translation (PBMT) baseline using the Moses toolkit, which treats compression as a translation task from "long" to "short" English.

Dataset Construction

To overcome data scarcity, the researchers mined the MSCOCO and FLICKR30K computer vision datasets. These datasets contain multiple human-written captions for the same image. By pairing long descriptions with shorter ones for the same visual scene (requiring at least a 10% reduction in length), they created a massive parallel corpus of 1,161,056 aligned sentence pairs. This dataset provides a natural source of semantic equivalents with varying structural complexity.

Results and Discussion

Evaluation was conducted using both automatic metrics and human judgment on a 7-point Likert scale for Fluency and Importance.

Qualitative Performance and Robustness

The aRNN demonstrated a remarkable ability to "abstract away" errors in source text. For instance, when presented with ungrammatical source captions, the aRNN often produced clean, fluent summaries, whereas the PBMT model tended to copy the errors.

Qualitative comparison of compression results across different models and human references.

However, the model also exhibited a "hallucination" risk. Because it learns from statistical co-occurrences, it occasionally adds details not present in the source sentence—such as adding "a glass of wine" to a table scene—simply because those elements frequently appear together in the training data.

Quantitative Findings

Human judges rated the aRNN's fluency at 5.62, nearly identical to human-generated summaries (5.61) and far exceeding the PBMT baseline (3.75). In terms of information retention (Importance), the aRNN (4.34) slightly outperformed both the baseline and the human reference in the test set.

A critical finding was the metric mismatch: automatic scores like BLEU and ROUGE showed little correlation with human quality ratings. While the models had similar BLEU scores, human evaluators perceived a vast difference in quality, highlighting the need for better metrics in abstractive generation tasks.

Conclusion

By leveraging image captions and attentive neural networks, this work demonstrates that abstractive compression can reach human-level fluency. While the model currently struggles with repetitive lists and is specific to the captioning domain, it paves the way for future research into sentence fusion and paragraph-level abstractive summarization.

Summary updated

BibTeX
@inproceedings{wubben2016abstractive,
  author = {Sander Wubben and Emiel Krahmer and Antal van den Bosch and Suzan Verberne},
  title = {{Abstractive Compression of Captions with Attentive Recurrent Neural Networks}},
  booktitle = {INLG},
  year = {2016},
  doi = {10.18653/v1/w16-6608},
}
Figure