TL;DR
This paper quantifies how "surprise"—defined as fluctuations in audience beliefs about who will win—drives the perceived quality of live-streamed esports. The researchers discovered that the timing of surprise is critical: twists occurring near the end of a match are far more predictive of high viewer ratings than the total amount of surprise throughout the game.
What Problem Does This Paper Address?
The live-streaming industry, particularly esports like League of Legends (LoL), is booming, yet what makes one match "better" than another remains difficult to quantify. While previous research hypothesized that "information flow" (how a story or game reveals its outcome) influences enjoyment, these studies often relied on theoretical models or indirect metrics like audience size.
The authors set out to bridge this gap by directly eliciting real-time human beliefs during live events. They aimed to answer a fundamental question for content creators and game designers: Is it enough to just have a "surprising" game, or does the schedule of those surprises dictate how much we actually enjoy the show?
Methods
The researchers developed a custom Information Flow Elicitation Platform to collect data during the 2020 League of Legends World Championship (LOL S10).
- Real-Time Belief Elicitation: Subjects watched live matches and updated their belief (0-100%) of the "blue team's" winning probability as often as they liked.
- Incentive Compatibility: To ensure honest reporting, subjects were paid based on a quadratic scoring rule, where more accurate predictions (relative to the final outcome) resulted in higher monetary rewards.
- Quantifying Surprise: The authors defined "surprise" as the sum of absolute changes in the median belief curve of all subjects. This median approach filters out "irrational" outliers who might report extreme beliefs regardless of game state.
Datasets
The study focused on a single, high-stakes dataset:
- Event: 2020 League of Legends World Championship (LOL S10).
- Scope: 76 individual games.
- Participants: 107 subjects recruited from top Chinese universities (mostly STEM majors).
- Observations: 4,566 total data points, including real-time belief updates and post-game quality ratings on a 1–9 Likert scale.
Key Insights and Ideas
- The Recency Bias of Excitement: The most striking finding is that surprise in the second half of a game is a much stronger predictor of quality than surprise in the first half. In fact, surprise in the first half was negatively correlated with final ratings in some models, suggesting that early volatility might be perceived as low-quality play or lack of control.
- The "Peak-End" Effect: The results strongly support the psychological "peak-end rule," which suggests people judge experiences based on their most intense point and their conclusion. The "end surprise" (generated in the last 2.5 minutes) explained more variance in ratings than the surprise of the entire second half.
- Rubbish Time is a Rating Killer: The authors quantified "rubbish time"—the period after the winner becomes predictable but before the game actually ends. They found that games with high rubbish time (low end-of-game surprise) consistently received lower ratings.
- Strategic Information Revelation: For game designers and broadcasters, this implies that "everything else being equal, it is better for twists to be more likely to happen toward the end of a show rather than uniformly throughout."
Key Results
- Predictive Power: The coefficient for second-half surprise (1.743) was significantly higher than for overall surprise (1.214) in predicting game ratings.
- Correlation Flip: Surprise in the first half had a negative correlation with ratings, while the second half was positive.
- Model Fit: Using "end surprise" (the final 2.5 minutes) yielded an adjusted $R^2$ of 0.232, outperforming the model using the entire second half ($R^2 = 0.222$).
- Robustness: These results held true even when controlling for whether the subjects' favorite team won or lost, though a win for a preferred team naturally gave a baseline boost to the rating.
Limitations and Future Work
- Suspense vs. Surprise: The paper focuses on surprise (retrospective change in belief). It does not explicitly measure "suspense," which is the expected variance of future beliefs. The authors suggest that future work could use this data to learn models for suspense.
- Subject Demographics: The subjects were primarily STEM students from top universities, which may not perfectly represent the global, diverse esports audience.
- Qualitative Data: The authors suggest that incorporating "bullet comments" (live chat text) could provide deeper context into why certain moments were perceived as surprising or high-quality.
- Causality: While the correlation is strong, the study doesn't strictly prove that scheduling surprise causes higher ratings, though the regression analysis makes a compelling case.
Summary updated