StreamSoccer: Event-Driven Memory for Streaming Soccer Commentary

Chenxi Shao, Bozhong Wang, Jiaxin Huang, Zhao Liu, Sunwei Zhu, Tianxin Hang, Gaoqi He, Yang Li, Changbo WangarXiv 2026 · arXiv

TL;DR

StreamSoccer introduces an event-driven memory system designed for real-time, streaming soccer commentary across multiple temporal scales (current, recent, and historical). By organizing video history into discrete, bounded "event lifecycles" rather than raw frames or token caches, it achieves state-of-the-art commentary quality while maintaining a constant, faster-than-real-time computational footprint over a full 90-minute match.

What Problem Does This Paper Address?

Generating live sports commentary requires an AI system to process a continuous video stream causally—meaning it can only use information available up to the current moment—while simultaneously handling diverse temporal contexts. A human commentator naturally switches between describing a foul that just happened (current event), summarizing the flow of play over the last few minutes (recent window), or recalling a specific corner kick from early in the first half (historical memory).

Standard streaming Video-Language Models (VLMs) compress history using task-agnostic frame or token caches. As the video progresses, these caches either overflow their context windows or suffer from massive computational slowdowns, making them impractical for long broadcasts. StreamSoccer addresses this gap by structuring video history around semantic event units that evolve, close upon completion, and persist as retrievable records.

Methods

StreamSoccer utilizes a multi-layered, event-driven architecture to manage the lifecycle of match events:

  • Streaming Event Memory Encoder: Maps incoming video clips to a fixed budget of clip-level event tokens. An Operational Transition Head compares new clips against the active memory to predict event boundaries (e.g., identifying when play stops or a set piece occurs). A 24-second maximum duration safeguard ensures the active state remains bounded.
  • Memory-to-Record Consolidation: When an event closes, its compact latent state is stored in a local buffer for immediate use. Concurrently, a projector and language model asynchronously convert the closed event into a textual Historical Event Record containing a generated caption and structured metadata.
  • Multi-Context Commentary Generator: A deterministic, rule-assisted scheduler decides whether the system should speak or remain silent based on cooldowns and event proximity. If it speaks, a unified generator combines the active event memory, recent latent buffers, and retrieved text records into a single language model prompt using LoRA adaptation.
Overview of the StreamSoccer architecture

Datasets

The authors constructed a new three-track streaming soccer commentary dataset by aligning human-annotated SoccerNet action timestamps with MatchTime commentary descriptions. The dataset contains 27,639 samples (15,189 current-event, 7,127 recent-window, and 5,323 historical-memory examples) across 422 matches. To ensure rigorous evaluation, the dataset uses strict match-disjoint splits (70% train, 15% validation, 15% test) and automated checks to prevent future data leakage.

Key Insights and Ideas

  • Events over Frames: Shifting the core memory unit from raw video frames to semantic "event lifecycles" allows the model to naturally compress long histories without losing critical context.
  • Hybrid Latent-Text Memory: Keeping recent history as rich, compact latent vectors while converting older history into retrievable text records provides the best of both worlds: high-fidelity local context and an infinite, computationally cheap long-term memory.
  • Constant Workload Scaling: Because historical events are stored as text records and retrieved via a top-$k$ query, the system completely avoids the computational "history tax" that bogs down traditional streaming VLMs over long sequences.

Key Results

  • Commentary Quality: StreamSoccer achieved the highest reported CIDEr scores on the current-event (38.62) and historical-memory (17.39) tracks, outperforming proprietary MLLMs (like GPT-4) and general streaming VLMs. It ranked second on the recent-window track (23.96), just behind UniSoccer.
  • Computational Efficiency: As shown in Figure 3, StreamSoccer maintains a 95th-percentile Real-Time Factor (RTF) of approximately 0.10–0.22 throughout a 90-minute match. This means it processes video 5x to 10x faster than real-time with zero computational growth over time, whereas baseline models like VideoLLM-online slow down drastically as match history accumulates.
Figure 3: Raw per-minute RTF p95 over 58 source-clean matches and 174 runs (lower is better). TimeChat-Online stops at its context limit.

Limitations and Future Work

  • Recent-Window Aggregation: The system's second-place finish on the recent-window track indicates a minor weakness in synthesizing multiple rapid-fire or overlapping recent events into a single summary.
  • Rule-Based Eventization: The event boundaries rely on deterministic football rules and a fixed 24-second safeguard rather than fully learned, unconstrained semantic discovery.
  • Generalization: The model was developed and evaluated entirely within a single soccer data ecosystem; its performance on other sports or varying broadcast styles (e.g., different languages or leagues) remains an open question for future research.

Summary updated

Abstract

Streaming video understanding requires models to causally update state as video arrives and organize growing history into semantic units that can evolve, persist, and be recalled under bounded computation and memory. This challenge is pronounced in live soccer commentary, where a system must describe completed events, summarize recent play, recall earlier events, or remain silent using only information available before each utterance. We present StreamSoccer, an event-driven system that uses event memory as its intermediate representation. A fixed-budget active memory integrates the stream; completed event states are retained locally and consolidated into retrievable historical records. A unified generator uses current, recent, and historical context to produce three commentary modes, while a rule-assisted scheduler selects a mode or silence. Unlike streaming video-language models organized around frames, visual tokens, or caches, and soccer-commentary methods based on predefined clips or output timestamps, StreamSoccer explicitly models event lifecycles. We construct a three-track streaming soccer commentary dataset and a layered evaluation protocol. At common reference anchors, StreamSoccer obtains CIDEr scores of 38.62, 23.96, and 17.39 for current-event, recent-window, and historical-memory commentary, ranking first on the current-event and historical-memory tracks and second on recent-window. Controlled ablations show that local completed events improve all tracks and that the full system performs best on all three. Across 174 raw-video runs on 58 matches, per-minute RTF p95 ranges from 0.10 to 0.22 without sustained growth with match history. These results indicate that event memory supports streaming soccer commentary across temporal scopes while controlling long-history computation.
BibTeX
@misc{shao2026streamsoccer,
  author = {Chenxi Shao and Bozhong Wang and Jiaxin Huang and Zhao Liu and Sunwei Zhu and Tianxin Hang and Gaoqi He and Yang Li and Changbo Wang},
  title = {{StreamSoccer: Event-Driven Memory for Streaming Soccer Commentary}},
  year = {2026},
  eprint = {2608.19723},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.19723},
}
Figure