TL;DR
This paper tackles the challenge of automatically summarizing the highlights of pre-scheduled events, like TV shows and sports games, from real-time Twitter streams. They introduce a novel method that identifies key event moments by detecting spikes in tweet volume and then summarizes these moments by focusing on the most discussed characters and their contexts.
What Problem Does This Paper Address?
Pre-scheduled events, such as popular TV series or major sporting events, generate massive amounts of real-time discussion on platforms like Twitter. While these tweets offer a rich, immediate perspective on the event, manually sifting through the sheer volume of data to identify the most significant moments and discussions is incredibly time-consuming and inefficient. Existing summarization techniques often struggle with the noisy, informal, and rapidly evolving nature of social media data. This paper aims to bridge this gap by developing an automated system to distill the essence of these live event discussions.
Methods
The authors propose a two-stage approach:
- Highlight Identification: They identify "highlights" by analyzing the temporal distribution of tweets. Specifically, they plot the number of tweets published per minute and look for significant spikes in volume. To smooth out short-term fluctuations, they use a 3-minute moving average. Peaks exceeding a threshold, determined by the mean and standard deviation of tweet volume across the entire event, are considered highlights.
- Character-Centric Summarization: Once a highlight is identified, the system focuses on the characters discussed most frequently within that specific time window. They leverage a pre-defined list of character names and aliases (derived from Wikipedia) to count mentions. The core of the summarization then involves understanding the context in which these characters are discussed. They use
context2vec, a neural model that generates sentence embeddings, to represent the contextual meaning of tweets mentioning the most popular character. These embeddings are then clustered using K-Means. The tweets closest to the cluster centroids are selected as the summary for that highlight, effectively capturing the most common discussions around the key character during that event moment.
Datasets
The primary dataset used in this paper consists of tweets collected around seven episodes of Game of Thrones Season 7 (GOTS7). The tweets were gathered in real-time using the Twitter streaming API, filtered by the hashtag "#gots7". The authors note that this filtering method omits tweets using other related hashtags or no hashtags at all. They plan to make this dataset available to the research community.
Key Insights and Ideas
- Tweet Volume as a Proxy for Event Highlights: The paper demonstrates that spikes in tweet volume are a strong indicator of significant or exciting moments within a pre-scheduled event. This is an intuitive but powerful observation for automated event analysis.
- Character Centrality in Event Discourse: During event highlights, specific characters often become the focal point of discussions. Identifying the most mentioned character in a highlight provides a direct link to the core of that sub-event.
- Context Matters for Summarization: Simply identifying keywords or frequent mentions isn't enough. Understanding the context in which characters are discussed is crucial for generating meaningful summaries. The use of
context2vecfor capturing nuanced contextual embeddings is a key methodological contribution. - Character Aliases are Essential: Social media discussions often use nicknames or abbreviations. The paper highlights the importance of creating comprehensive lists of character aliases to accurately track mentions.
Key Results
The authors evaluated their algorithm on GOTS7 episodes 3 and 4, comparing their generated summaries against plot summaries from IMDB using the ROUGE metric. Their model outperformed several baseline summarization algorithms, including Luhn, Most Retweeted, TextRank, LexRank, and LSA, on both episodes. For instance, in Episode 3, their model achieved ROUGE-F1 scores of 31.5, significantly higher than the next best baseline (Luhn at 12.3). Table 1 provides examples of highlight summaries generated by their model for specific scenes.
Limitations and Future Work
The authors acknowledge that their hashtag-based filtering might miss some relevant tweets. They also note that in some cases, their generated summaries could be similar, suggesting a need for improved diversity in the output.
Future work includes testing their model on tweets collected around other types of events, such as presidential debates, and further improving the diversity of the generated summaries.
Summary updated