Transformer-based Live Update Generation for Soccer Matches from Microblog Posts

Masashi Oshika, Kosuke Yamada, Ryohei Sasano, Koichi TakedaEMNLP 2023 · DOI

Introduction

Following live sports matches on social media platforms like Twitter (X) offers instant excitement and diverse perspectives. However, users are often overwhelmed by a "vast amount of diverse live tweets." While some posts are highly informative (e.g., "Goal by Doan!"), many are emotive ("Yesssssss!!") or irrelevant. Existing automated summarization methods struggle with this noise. Extractive systems triggered by volume spikes often miss quiet but critical events like tactical substitutions, while fixed-interval systems frequently produce redundant updates as users continue to tweet about a major event long after it occurred. This research addresses the challenge of building a system that determines when to generate an update, what information to include, and how to avoid repetition.

Methodology

The proposed system utilizes the Text-to-Text Transfer Transformer (T5) for abstractive summarization. Unlike extractive methods that select existing tweets, this approach generates new, concise text based on input microblog data. The researchers evaluated four model variations to address filtering and redundancy:

  1. $\mathcal{M}_{base}$: A baseline model that processes tweets from a specific minute to generate either an update or a "NaN" (indicating no significant event).
  2. $\mathcal{M}_{+clf}$ (The Gatekeeper): Incorporates a separate binary classifier to decide if the current tweet stream warrants an update before the generator runs.
  3. $\mathcal{M}_{+cxt}$ (The Memory): Provides the generator with tweets plus the updates from the previous four minutes. This context allows the model to recognize previously reported events and reduce redundancy.
  4. $\mathcal{M}_{+clf+cxt}$: A hybrid model combining both the gatekeeper classifier and the contextual memory.
Architectures of the four proposed model variations: baseline, classifier-gated, context-aware, and the combined hybrid model.

To evaluate these models, the authors employed aligned n-gram overlap. Using dynamic programming, they matched generated updates to reference updates, accounting for the typical one-minute time drift between real-world events and social media reactions.

Experimental Setup

The study utilized data from the J-League (Japanese professional soccer), covering 83 matches from the 2021 and 2022 seasons. The dataset comprised 324,835 tweets and 8,502 official updates. To ensure data density, only matches with more than 3,200 tweets were included. Official text updates from the J-League website served as the ground truth for evaluation.

Results and Discussion

The experimental results demonstrate that the context-aware model ($\mathcal{M}_{+cxt}$) provides the most balanced and effective performance.

Performance comparison of the proposed models against extractive oracles using aligned unigram metrics.

Key Findings:

  • Context Reduces Redundancy: Feeding previous updates back into the model ($\mathcal{M}_{+cxt}$) significantly improved performance. This "short-term memory" allowed the system to recognize that persistent "GOAL!" tweets referred to an already-reported event.
  • Abstractive Superiority: T5-based models outperformed "Oracle" extractive models. Individual tweets are often too informal or fragmented to serve as professional-grade updates; abstractive generation synthesizes these into clearer reports.
  • Performance Metrics: The $\mathcal{M}_{+cxt}$ model achieved a unigram F1-score of 0.305, a substantial improvement over the baseline (0.182) and the best extractive oracle (0.241). While the baseline had high precision (0.401), its recall was very low (0.118). The context-aware model more than doubled this recall (0.251) while maintaining high precision (0.389).
  • Human Evaluation: Manual reviews confirmed that context-aware models were superior at avoiding redundant reports of the same event across consecutive minutes.

Conclusion

This research demonstrates that a T5-based abstractive summarization system, when augmented with contextual memory of previous updates, can effectively transform chaotic microblog streams into coherent live sports updates. While the system significantly reduces redundancy and improves recall over baseline and extractive methods, future work is needed to incorporate specific entity information (like player rosters) to enhance detail and further minimize occasional repetitions in prolonged "tweet storms."

Note: This study was conducted on Japanese soccer matches; its generalizability to other languages and sports remains a subject for future investigation.

Summary updated

BibTeX
@inproceedings{oshika2023transformer,
  author = {Masashi Oshika and Kosuke Yamada and Ryohei Sasano and Koichi Takeda},
  title = {{Transformer-based Live Update Generation for Soccer Matches from Microblog Posts}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.emnlp-main.624},
}
Figure