Indicative Tweet Generation: An Extractive Summarization Problem?

Priya Sidhaye, Jackie Chi Kit CheungEMNLP 2015 · OpenReview

Abstract

Social media such as Twitter have become an important method of communication, with potential opportunities for NLG to facilitate the generation of social media content. We focus on the generation of indicative tweets that contain a link to an external web page. While it is natural and tempting to view the linked web page as the source text from which the tweet is generated in an extractive summarization setting, it is unclear to what extent actual indicative tweets behave like extractive summaries. We collect a corpus of indicative tweets with their associated articles and investigate to what extent they can be derived from the articles using extractive methods. We also consider the impact of the formality and genre of the article. Our results demonstrate the limits of viewing indicative tweet generation as extractive summarization, and point to the need for the development of a methodology for tweet generation that is sensitive to genre-specific issues.

Introduction

With hundreds of millions of tweets produced daily, Natural Language Generation (NLG) offers significant potential for assisting users and organizations in drafting social media content. A particularly valuable category is the indicative tweet: a post containing a URL designed to help readers decide whether to click through to the full article.

Conventionally, researchers have approached this as an "extractive summarization" task, assuming that systems can simply select the most salient sentences or phrases from the source article. However, it remains unclear whether human-written tweets actually follow this pattern. If they do not, existing extractive algorithms may be fundamentally misaligned with the goal of generating natural, human-like social media content.

Methodology

To evaluate the "extractiveness" of human-written tweets, the authors conducted a rigorous analytical study rather than proposing a new model. The analysis focused on three primary areas:

  1. Overlap Analysis: Using metrics inspired by ROUGE, the study calculated the percentage of unigrams (single words), bigrams (two-word phrases), and Longest Common Subsequences (LCS) shared between a tweet and its linked article.
  2. Locality Testing: The researchers investigated whether tweets were extracted from specific "windows" of text (e.g., a three-sentence block) to determine if they functioned as compressed versions of specific segments.
  3. Formality Correlation: Using a specialized formality lexicon, the study calculated formality scores for articles to see if genre influences how content is summarized for social media.

Dataset Characteristics

The study introduced a new, diverse dataset of 2,471 unique tweet-article pairs. Data was collected using 51 trending hashtags across six categories: Politics, Science & Technology, Events, Films/Pop Culture, International, and Sports. To ensure sufficient source material, the authors used the newspaper library to extract clean text and filtered out articles with fewer than 150 words.

Analysis of Extractiveness

The results indicate that tweets are rarely simple "cut and paste" operations. Out of nearly 2,500 pairs, a tweet matched an exact substring of the article only 23 times—and 9 of those instances were merely the article's title.

Lexical Overlap

Even using the most permissive measure, unigram overlap, the mean match was only 29.53%. This suggests that over 70% of the informative words in a typical indicative tweet are absent from the source article.

Unigram overlap across the dataset, showing a mean match of approximately 30%.

When examining bigram overlap, the mean dropped significantly to 10.73%. This highlights that even when individual words are shared, the specific phrasing is almost always original to the tweet.

Bigram overlap across the dataset, indicating that phrasing is rarely copied directly from the source.

The Role of Formality and Intent

The study found a significant positive correlation (0.41) between article formality and the extractiveness of the resulting tweet. Tweets about formal topics (e.g., political summits) were much more likely to use language from the source text than those about pop culture.

Ultimately, these findings suggest that a tweet is not merely a summary but a social act. Users inject style, opinion, or a "hook" to engage their audience—elements that purely extractive models cannot replicate.

Conclusion and Future Directions

The analysis demonstrates the clear limitations of extractive methods for indicative tweet generation. Future research should pivot toward abstractive summarization models capable of generating novel sentences.

Furthermore, the authors advocate for intent-based generation. Systems must move beyond asking "What is this article about?" to "Why is the user sharing this?" (e.g., for promotion, humor, or opinion). Identifying these underlying intents and implementing "tweet normalization" to handle social media-specific linguistic features will be essential for creating systems that generate truly human-like social media content.

Summary updated

Abstract

Social media such as Twitter have become an important method of communication, with potential opportunities for NLG to facilitate the generation of social media content. We focus on the generation of indicative tweets that contain a link to an external web page. While it is natural and tempting to view the linked web page as the source text from which the tweet is generated in an extractive summarization setting, it is unclear to what extent actual indicative tweets behave like extractive summaries. We collect a corpus of indicative tweets with their associated articles and investigate to what extent they can be derived from the articles using extractive methods. We also consider the impact of the formality and genre of the article. Our results demonstrate the limits of viewing indicative tweet generation as extractive summarization, and point to the need for the development of a methodology for tweet generation that is sensitive to genre-specific issues.
BibTeX
@inproceedings{sidhaye2015indicative,
  author = {Priya Sidhaye and Jackie Chi Kit Cheung},
  title = {{Indicative Tweet Generation: An Extractive Summarization Problem?}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2015},
  url = {https://openreview.net/forum?id=rJWQmGMdZS},
}

Loading related content...

Figure