I Would Not Plant Apple Trees If the World Will Be Wiped: Analyzing Hundreds of Millions of Behavioral Records of Players During an MMORPG Beta Test

Ah Reum Kang, Jeremy Blackburn, Haewoon Kwak, Huy Kang KimWWW 2017 · OpenReview

TL;DR

This paper analyzes player behavior in the final days of an MMORPG beta test, treating the impending data wipe as an "end times" scenario. Surprisingly, they found no widespread "pandemic" behavior changes, but rather a significant abandonment of character progression and a rise in anti-social behavior among those who voluntarily left the game.

What Problem Does This Paper Address?

Philosophers have long pondered how humans would behave if their actions had no lasting consequences – the "end times" scenario. This paper uses the closed beta test (CBT) of the MMORPG ArcheAge as a unique "living laboratory" to investigate this question. The key is that at the end of a CBT, all player data is wiped, meaning any in-game actions or achievements become meaningless in the real world. This provides a compelling proxy for a situation where consequences are removed. While previous studies on game betas focused on market testing or performance, this work specifically examines behavioral shifts in anticipation of a complete data wipe, bridging the gap between virtual behavior and real-world implications.

Methods

The researchers analyzed a massive dataset of 270 million player actions from ArcheAge's 4th CBT. They employed a multi-level analysis:

  • System-wide analysis: Examining aggregate player behavior over time to detect widespread "pandemic" changes.
  • Individual-level analysis: Focusing on individual players to identify specific behavioral shifts, particularly distinguishing between those who left voluntarily ("churners") and those who stayed until the end of the CBT.
  • Network analysis: Constructing graphs of player actions and their "peak" occurrences (significant deviations from normal behavior) to identify communities of associated actions and compare these communities between voluntary leavers and system-end leavers.
  • Sentiment analysis: Using valence scores on chat logs to gauge the emotional tone of player communication.

Datasets

The primary dataset used is the anonymized, full logs from the 4th closed beta test (CBT) of the MMORPG ArcheAge. This dataset comprises 270 million records and covers the period from December 8th, 2011, to February 20th, 2012 (approximately 11 weeks).

Key Insights and Ideas

  • The "Apple Tree" Adage is Questionable: Contrary to the idea that people would continue their productive activities even in the face of oblivion ("Even if I knew the world would go to pieces tomorrow, I would still plant my apple tree"), players significantly reduced their focus on character progression (quest completion, leveling, ability changes) as the beta test neared its end.
  • No Mass Hysteria, But Outliers Emerge: The study found no evidence of widespread, "pandemic" behavioral changes across the entire player base. However, some individual players did exhibit increased anti-social behavior, such as player killing (PK).
  • Voluntary Leavers are Different: Players who voluntarily left the game before the CBT ended (churners) were more likely to engage in anti-social behavior like PK. This suggests a loss of responsibility or attachment to the game, contrasting with those who stayed.
  • Social Interaction Trends Positively: Surprisingly, chat sentiment analysis revealed a slight positive trend in communication as the CBT concluded, with players exchanging more messages and forming more parties. This suggests that social bonds might strengthen in the face of an impending shared "end."
  • Behavioral Peaks Don't Mix Signs: When analyzing individual behavioral "peaks" (significant deviations from normal behavior), the researchers found that players did not simultaneously exhibit positive and negative peaks for different actions. This implies a tendency to either increase or decrease specific behaviors, but not both at the same time.

Key Results

  • No Systemic Pandemic Behavior: Aggregate analysis showed no significant widespread changes in player actions as the CBT ended.
  • Abandonment of Progression: A drastic decrease in quest completion, leveling, and ability changes was observed in the final days.
  • Increased Anti-Social Behavior (Outliers): While not pandemic, some players engaged in more player killing (PK) towards the end.
  • Positive Chat Sentiment: Chat messages showed a slight, statistically significant positive bias, indicating increased social interaction.
  • Churner Behavior: Voluntary leavers were significantly more likely to exhibit anti-social behavior (PK) compared to those who stayed.
  • Network Analysis: Communities of related behavioral peaks were identified, with distinct patterns between voluntary leavers and system-end leavers.

Limitations and Future Work

  • Virtual vs. Real World: The authors acknowledge that ArcheAge is a game, and the consequences of the CBT ending are virtual. While virtual property can have real-world monetary value, it's not a perfect one-to-one mapping with real-world "end times" scenarios.
  • Prior CBT Experience: Players might have had prior experience with CBTs and data wipes, potentially influencing their behavior. However, the authors argue that the immediate impact of the avatar's end still holds relevance.
  • Future Work: The study suggests that sentiment analysis on chat messages can be a useful tool for identifying outlier players exhibiting extreme behavior. Further research could explore how to leverage these findings for game retention strategies and better CBT design. The authors also highlight the potential for this research to inform studies on human behavior in situations where consequences are removed.

Summary updated

Abstract

In this work, we use player behavior during the closed beta test of the MMORPG ArcheAge as a proxy for an extreme situation: at the end of the closed beta test, all user data is deleted, and thus, the outcome (or penalty) of players' in-game behaviors in the last few days loses its meaning. We analyzed 270 million records of player behavior in the 4th closed beta test of ArcheAge. Our findings show that there are no apparent pandemic behavior changes, but some outliers were more likely to exhibit anti-social behavior (e.g., player killing). We also found that contrary to the reassuring adage that "Even if I knew the world would go to pieces tomorrow, I would still plant my apple tree," players abandoned character progression, showing a drastic decrease in quest completion, leveling, and ability changes at the end of the beta test.
BibTeX
@inproceedings{kang2017i,
  author = {Ah Reum Kang and Jeremy Blackburn and Haewoon Kwak and Huy Kang Kim},
  title = {{I Would Not Plant Apple Trees If the World Will Be Wiped: Analyzing Hundreds of Millions of Behavioral Records of Players During an MMORPG Beta Test}},
  booktitle = {The Web Conference},
  year = {2017},
  url = {https://openreview.net/forum?id=UazPY78586FA},
}
Figure