A Participant-based Approach for Event Summarization Using Twitter Streams

Chao Shen, Fei Liu, Fuliang Weng, Tao LiNAACL 2013

TL;DR

This paper introduces a novel approach to summarize live events from Twitter by focusing on individual participants rather than the entire stream. By "zooming in" on participant-specific conversations, their method effectively captures important sub-events that might otherwise be lost in the noise, leading to more comprehensive and coherent event summaries.

What Problem Does This Paper Address?

Twitter has become a crucial source for real-time information, with users sharing live updates on a vast array of events, from natural disasters to sports games and conferences. While this offers unprecedented access to event happenings, summarizing these massive, noisy, and short-form text streams presents a significant challenge. Existing summarization techniques, often developed for more structured news articles, struggle with the brevity, non-standard language, and temporal dynamics of tweets. A key issue is that dominant participants or sub-events can overshadow less prominent but still important aspects of an event, leading to summaries that lack comprehensive coverage. This paper aims to address this by developing a method that can generate representative and concise textual descriptions of scheduled events from Twitter, ensuring better coverage of all significant moments.

Methods

The core of this paper's contribution lies in its participant-based event summarization approach, which consists of three main components:

  1. Participant Detection: This module dynamically identifies key entities (people, organizations, etc.) involved in an event. It uses a hierarchical agglomerative clustering framework that groups proper nouns (mentions) based on both lexical resemblance (e.g., abbreviations, character overlap) and contextual similarity within tweet segments. This allows the system to create individual "participant streams" from the overall event stream.
  2. Sub-event Detection: This is the most novel part. The authors propose a time-content mixture model to identify important sub-events within each participant stream. This model uniquely combines two key properties of event tweets:
  3. Temporal Burstiness: The tendency for tweets related to a sub-event to appear in concentrated bursts over time.
  4. Lexical Cohesiveness: The shared vocabulary and themes within tweets discussing a specific sub-event. The model uses a probabilistic approach to capture both the temporal distribution (Gaussian) and the word distribution (multinomial) of sub-events. This approach is applied to participant streams to detect "participant sub-events," which are then merged to form "global sub-events" representing the overall event timeline.

  5. Summary Tweet Extraction: Once global sub-events are identified, the system extracts a representative tweet for each sub-event. These selected tweets are then concatenated to form the final event summary, aiming for a comprehensive overview of the event's progression.

Datasets

The authors evaluated their approach on six NBA basketball games and one conference event: the Apple CEO's keynote speech at WWDC 2012. They collected tweets in real-time using pre-defined keyword sets for each event. For NBA games, keywords included team names and player/coach names. For WWDC, terms like "wwdc," "apple," and "mac" were used. After initial filtering (removing URLs, non-English tweets, and retweets), over 1.8 million tweets were used for experiments.

For evaluation, they created goldstandard datasets by:

  • Manually grouping proper noun mentions into participant clusters.
  • Manually identifying representative tweets for global sub-events, often by filtering play-by-play descriptions from sources like ESPN and MacRumors to align with tweet spikes and human judgment.

Key Insights and Ideas

  • Participant-centricity is key: Focusing on individual participants, rather than the entire event stream, is crucial for uncovering nuanced sub-events that might be drowned out by more dominant discussions.
  • Burstiness alone isn't enough: While spike detection (burstiness) is a common approach, it misses the "lexical cohesiveness" of sub-events. The proposed mixture model, by combining both temporal and content features, offers a more robust way to identify sub-events.
  • Shadowed sub-events are real: The "long-tail" of less popular participants or sub-events often gets ignored in traditional summarization. The participant-based approach effectively brings these to light.
  • Heterogeneous event types benefit: The approach shows promise across different event types (sports vs. conferences), suggesting its generalizability.

Key Results

  • Participant Detection: The hierarchical clustering approach achieved good precision and recall at both the participant and mention levels, with the "local context + lexical" similarity measure performing well.
  • Sub-event Detection: The proposed "MM" (Mixture Model) approach consistently outperformed the "Spike" (burstiness-only) method in both participant and global sub-event detection, particularly in terms of precision. The "Participant + MM" approach yielded the best results, demonstrating the advantage of combining participant-level analysis with the mixture model.
  • Summarization: ROUGE-1 scores showed that the "Participant + MM" approach achieved significantly better results than the "Spike" or "Participant + Spike" methods, indicating more overlap with human-generated summaries. Qualitative analysis suggested that participant-based summaries led to better text coherence.

Limitations and Future Work

  • Evaluation Metrics: The authors acknowledge that ROUGE-1, while correlated with human judgment, might not be the most appropriate metric for evaluating Twitter event summaries due to their unique characteristics. They suggest exploring other metrics like ROUGE-2, ROUGE-SU4, and the Pyramid method, as well as conducting more extensive human evaluations.
  • Parameter Tuning: Some threshold values for sub-event merging and topic re-adjustment were empirically set. Future work could explore more principled methods for parameter selection.
  • Online Application: While the current work reports results on entire event streams, the authors note that the proposed approach can be applied in an online fashion.
  • Integration: Further exploration into better ways of integrating sub-event detection and summarization approaches is suggested.

Summary updated

BibTeX
@inproceedings{shen2013participant,
  author = {Chao Shen and Fei Liu and Fuliang Weng and Tao Li},
  title = {{A Participant-based Approach for Event Summarization Using Twitter Streams}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2013},
}

Loading related content...

Figure