Unexpected Relevance: An Empirical Study of Serendipity in Retweets

Tao Sun, Ming Zhang, Qiaozhu Mei

Abstract

Serendipity is a beneficial discovery that happens in an unexpected way. It has been found spectacularly valuable in various contexts, including scientific discoveries, acquisition of business, and recommender systems. Although never formally proved with large-scale behavioral analysis, it is believed by scientists and practitioners that serendipity is an important factor of positive user experience and increased user engagement. In this paper, we take the initiative to study the ubiquitous occurrence of serendipitous information diffusion and its effect in the context of microblogging communities. We refer to serendipity as unexpected relevance, then propose a principled statistical method to test the unexpectedness and the relevance of information received by a microblogging user, which identifies a serendipitous diffusion of information to the user. Our findings based on large-scale behavioral analysis reveal that there is a surprisingly strong presence of serendipitous information diffusion in retweeting, which accounts for more than 25% of retweets in both Twitter and Weibo. Upon the identification of serendipity, we are able to conduct observational analysis that reveals the benefit of serendipity to microblogging users. Results show that both the discovery and provision of serendipity increase the level of user activities and social interactions, while the provision of serendipitous information also increases the influence of Twitter users.

Introduction

Serendipity—the "Aha!" moment of stumbling upon something unexpectedly useful—is widely regarded as a cornerstone of positive user experience in digital environments. However, quantifying this elusive concept has historically been a challenge. Previous research often relied on small-scale surveys or manual labeling, which lack the scalability required for modern social media platforms.

In the context of microblogging, serendipity is not merely novelty. A tweet from a known source about a predictable topic is novel but not surprising. Conversely, a tweet about an obscure topic is surprising but only serendipitous if it is relevant to the receiver. This paper addresses this gap by formalizing serendipity as unexpected relevance and providing a scalable statistical framework to detect it within the massive stream of social media data.

Methodology

The researchers developed a formal statistical framework to identify serendipitous information diffusion by modeling users and contexts as unigram language models.

Conceptual models of information diffusion, culminating in serendipitous diffusion characterized by unexpectedness and relevance.

The identification process utilizes the Likelihood Ratio Test (LRT) across two dimensions:

  1. Test of Unexpectedness: This test determines if a tweet is surprising given the source. The null hypothesis ($H_0$) posits the tweet was generated from the source's typical topics. The alternative ($H_1$) suggests a mixture of the source, current trends, and general background. A significant shift toward $H_1$ marks the tweet as "unexpected."
  2. Test of Relevance: This test assesses if the tweet aligns with the receiver's interests. The null hypothesis assumes the tweet is explained by general trends, while the alternative incorporates the receiver’s specific interest model.

A retweet is classified as serendipitous only if it passes both tests at a 95% confidence level ($\alpha = 0.05$). This rigorous approach avoids arbitrary similarity thresholds and accounts for the inherent sparsity of microblogging data.

Empirical Analysis and Results

The framework was applied to two large-scale datasets: a 10% random sample of Twitter retweets from 2011 (approx. 587,000 RTs) and a dataset from Weibo (approx. 67,000 RTs).

Prevalence of Serendipity

The study found that serendipity is a ubiquitous phenomenon in social networks. It accounts for 27% of retweets on Twitter and 30% on Weibo. This suggests that nearly a third of social sharing is driven by "happy accidents"—information that is both surprising from its source and highly relevant to the sharer.

Impact on User Engagement and Influence

Using a "difference-in-differences" econometric analysis, the researchers quantified the effects of serendipity on user behavior:

  • Engagement Boost: Exposure to serendipitous content correlates with a statistically significant increase in overall user activity, including higher rates of tweeting, replying, and retweeting.
  • Social Influence: Users who consistently provide serendipitous information to their followers see a rise in social status. This is reflected in increased "listed" counts and higher retweet rates for their original content, indicating they are perceived as valuable information brokers.

Discussion and Limitations

While the results demonstrate a strong correlation between serendipity and engagement, several avenues for future work remain. The current study relies on retweets as a proxy for serendipity, which may exclude instances where a user finds content serendipitous but does not share it. Future research could integrate click-through or dwell-time data for a more granular view. Additionally, while unigram models provide a robust baseline, more advanced semantic models (such as LDA or embeddings) could further refine the detection of relevance.

Ultimately, this work provides the first large-scale empirical evidence that serendipity is not just a pleasant coincidence but a measurable and vital driver of social media dynamics and user growth.

Summary updated

Abstract

Serendipity is a beneficial discovery that happens in an unexpected way. It has been found spectacularly valuable in various contexts, including scientific discoveries, acquisition of business, and recommender systems. Although never formally proved with large-scale behavioral analysis, it is believed by scientists and practitioners that serendipity is an important factor of positive user experience and increased user engagement. In this paper, we take the initiative to study the ubiquitous occurrence of serendipitious information diffusion and its effect in the context of microblogging communities. We refer to serendipity as unexpected relevance, then propose a principled statistical method to test the unexpectedness and the relevance of information received by a microblogging user, which identifies a serendipitous diffusion of information to the user. Our findings based on large-scale behavioral analysis reveal that there is a surprisingly strong presence of serendipitous information diffusion in retweeting, which accounts for more than 25% of retweets in both Twitter and Weibo. Upon the identification of serendipity, we are able to conduct observational analysis that reveals the benefit of serendipity to microblogging users. Results show that both the discovery and provision of serendipity increase the level of user activities and social interactions, while the provision of serendipitous information also increases the influence of Twitter users.
BibTeX
@inproceedings{sun2021unexpected,
  author = {Tao Sun and Ming Zhang and Qiaozhu Mei},
  title = {{Unexpected Relevance: An Empirical Study of Serendipity in Retweets}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/icwsm.v7i1.14435},
}
Figure