Introduction
Following live sports matches on social media platforms like Twitter (X) offers instant excitement and diverse perspectives. However, users are often overwhelmed by a "vast amount of diverse live tweets." While some posts are highly informative (e.g., "Goal by Doan!"), many are emotive ("Yesssssss!!") or irrelevant. Existing automated summarization methods struggle with this noise. Extractive systems triggered by volume spikes often miss quiet but critical events like tactical substitutions, while fixed-interval systems frequently produce redundant updates as users continue to tweet about a major event long after it occurred. This research addresses the challenge of building a system that determines when to generate an update, what information to include, and how to avoid repetition.
Methodology
The proposed system utilizes the Text-to-Text Transfer Transformer (T5) for abstractive summarization. Unlike extractive methods that select existing tweets, this approach generates new, concise text based on input microblog data. The researchers evaluated four model variations to address filtering and redundancy:
- $\mathcal{M}_{base}$: A baseline model that processes tweets from a specific minute to generate either an update or a "NaN" (indicating no significant event).
- $\mathcal{M}_{+clf}$ (The Gatekeeper): Incorporates a separate binary classifier to decide if the current tweet stream warrants an update before the generator runs.
- $\mathcal{M}_{+cxt}$ (The Memory): Provides the generator with tweets plus the updates from the previous four minutes. This context allows the model to recognize previously reported events and reduce redundancy.
- $\mathcal{M}_{+clf+cxt}$: A hybrid model combining both the gatekeeper classifier and the contextual memory.
To evaluate these models, the authors employed aligned n-gram overlap. Using dynamic programming, they matched generated updates to reference updates, accounting for the typical one-minute time drift between real-world events and social media reactions.
Experimental Setup
The study utilized data from the J-League (Japanese professional soccer), covering 83 matches from the 2021 and 2022 seasons. The dataset comprised 324,835 tweets and 8,502 official updates. To ensure data density, only matches with more than 3,200 tweets were included. Official text updates from the J-League website served as the ground truth for evaluation.
Results and Discussion
The experimental results demonstrate that the context-aware model ($\mathcal{M}_{+cxt}$) provides the most balanced and effective performance.
Key Findings:
- Context Reduces Redundancy: Feeding previous updates back into the model ($\mathcal{M}_{+cxt}$) significantly improved performance. This "short-term memory" allowed the system to recognize that persistent "GOAL!" tweets referred to an already-reported event.
- Abstractive Superiority: T5-based models outperformed "Oracle" extractive models. Individual tweets are often too informal or fragmented to serve as professional-grade updates; abstractive generation synthesizes these into clearer reports.
- Performance Metrics: The $\mathcal{M}_{+cxt}$ model achieved a unigram F1-score of 0.305, a substantial improvement over the baseline (0.182) and the best extractive oracle (0.241). While the baseline had high precision (0.401), its recall was very low (0.118). The context-aware model more than doubled this recall (0.251) while maintaining high precision (0.389).
- Human Evaluation: Manual reviews confirmed that context-aware models were superior at avoiding redundant reports of the same event across consecutive minutes.
Conclusion
This research demonstrates that a T5-based abstractive summarization system, when augmented with contextual memory of previous updates, can effectively transform chaotic microblog streams into coherent live sports updates. While the system significantly reduces redundancy and improves recall over baseline and extractive methods, future work is needed to incorporate specific entity information (like player rosters) to enhance detail and further minimize occasional repetitions in prolonged "tweet storms."
Note: This study was conducted on Japanese soccer matches; its generalizability to other languages and sports remains a subject for future investigation.
Summary updated