Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads

Arkaitz Zubiaga, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, Peter TolmiearXiv 2015 · DOI · arXiv

TL;DR

This paper investigates the lifecycle of social media rumors by analyzing conversational threads on Twitter across nine major breaking news events. The authors reveal that while users blindly support unverified rumors early on, true rumors are verified seven times faster than false ones are debunked. Furthermore, high-reputation accounts like news organizations often inadvertently fuel false rumors by posting unverified information with high certainty and structural authority.

What Problem Does This Paper Address?

During breaking news, social media is a double-edged sword: it provides real-time updates but acts as a breeding ground for unverified rumors. While previous research often treated rumors simply as "false information," this paper adopts a more nuanced definition: a rumor is a story of questionable veracity that circulates before its truth value is resolved.

The core challenge the authors address is understanding the dynamic lifecycle of these rumors. Specifically, how do social media users—ranging from ordinary citizens to mainstream media outlets—react to, spread, support, or deny rumors before and after their veracity is officially established? Understanding these behavioral patterns is crucial for developing automated, real-time rumor-verification systems.

Methods

The authors developed a comprehensive methodology to capture and analyze rumor dynamics through the lens of conversational threads:

  1. Data Collection & Sampling: Using Twitter’s Streaming API, they tracked keywords for breaking events. Journalists on the research team identified "source tweets" that introduced rumors and tracked down the specific "resolving tweets" that later confirmed or debunked them.
  2. Thread Scraping: Instead of looking at isolated tweets, the authors scraped complete, nested reply trees sparked by the rumor-introducing tweets.
  3. Three-Dimensional Annotation Scheme: Using crowdsourcing (validated by an expert panel), they annotated a subset of the dataset across three dimensions:
  4. Support/Response Type: Whether a tweet agreed, disagreed, appealed for more information, or just commented.
  5. Certainty: The author's confidence level (Certain, Somewhat certain, Uncertain).
  6. Evidentiality: The type of evidence presented (e.g., first-hand experience, URL, picture, or no evidence).
  7. User Reputation Metric: They defined a follow ratio ($\log_{10}(\text{\#followers} / \text{\#following})$) to standardize and measure user reputation.

Datasets

The study introduced a newly curated dataset drawn from 9 newsworthy events, split into breaking news (e.g., the Ferguson unrest, Ottawa shooting, Sydney siege, Charlie Hebdo shooting, Germanwings crash) and specific rumors known a priori (e.g., Putin missing).

From an initial pool of 2,695 rumorous source tweets, they randomly sampled a subset for deep annotation:

  • 330 rumorous conversation threads (comprising 4,842 total tweets).
  • Languages: 4,560 tweets in English, 282 in German.
  • Veracity Breakdown: 159 true stories, 68 false stories, and 103 that remained unverified.

Key Insights and Ideas

  • The Asymmetry of Truth and Falsehood: True rumors are corroborated quickly (median time: ~2 hours), whereas false rumors take significantly longer to be debunked (median time: >14 hours). Proving a negative or providing definitive counter-evidence is fundamentally harder and slower.
  • The "Blind Support" Tendency: Before a rumor's veracity is resolved, the overwhelming tendency of the crowd is to support it, regardless of whether it eventually turns out to be true or false. "Collective intelligence" fails during the unverified stage; aggregation cannot determine truth when everyone is guessing.
  • The Immediacy Trap for Reputable Users: Users with a high follow ratio (predominantly news organizations) are highly certain and prone to attaching external evidence to their tweets. However, due to the professional pressure of digital journalism ("publish first, correct later"), they frequently end up mainstreaming unverified rumors that turn out to be entirely false.
  • The Post-Resolution Shift: Once a rumor is resolved, the dynamics flip. For false rumors, denial messages spike dramatically. For true rumors, surprisingly, skepticism and denial also slightly increase because supporters stop tweeting (the issue is settled), leaving only the vocal skeptics in the thread.

Key Results

  • Resolution Speed: True rumors have a median resolution time of 2 hours, compared to over 14 hours for false rumors.
  • Retweet Bursts: Unverified rumors spark a massive, immediate burst of retweets within the first 20 minutes of being posted, outperforming post-resolution tweets.
  • Crowd Behavior: The only scenario where denying tweets numerically outnumber supporting tweets is after a false rumor has been explicitly debunked. Prior to that, support dominates.
  • Annotation Agreement: The crowdsourced annotations achieved an overall inter-annotator agreement rate of 63.7% for Support, 61.1% for Certainty, and 60.8% for Evidentiality.

Limitations and Future Work

The primary limitation acknowledged by the authors is the sample size; due to time and budget constraints, only 330 out of the 2,695 collected conversation threads were fully annotated via crowdsourcing.

For future work, the authors plan to leverage this annotated dataset to train machine learning models capable of predicting rumor veracity in real-time. They also aim to investigate whether the confident, rumor-supporting behavior observed in high-reputation accounts generalizes beyond news organizations to other types of influencers.

Summary updated

Abstract

As breaking news unfolds people increasingly rely on social media to stay abreast of the latest updates. The use of social media in such situations comes with the caveat that new information being released piecemeal may encourage rumours, many of which remain unverified long after their point of release. Little is known, however, about the dynamics of the life cycle of a social media rumour. In this paper we present a methodology that has enabled us to collect, identify and annotate a dataset of 330 rumour threads (4,842 tweets) associated with 9 newsworthy events. We analyse this dataset to understand how users spread, support, or deny rumours that are later proven true or false, by distinguishing two levels of status in a rumour life cycle i.e., before and after its veracity status is resolved. The identification of rumours associated with each event, as well as the tweet that resolved each rumour as true or false, was performed by a team of journalists who tracked the events in real time. Our study shows that rumours that are ultimately proven true tend to be resolved faster than those that turn out to be false. Whilst one can readily see users denying rumours once they have been debunked, users appear to be less capable of distinguishing true from false rumours when their veracity remains in question. In fact, we show that the prevalent tendency for users is to support every unverified rumour. We also analyse the role of different types of users, finding that highly reputable users such as news organisations endeavour to post well-grounded statements, which appear to be certain and accompanied by evidence. Nevertheless, these often prove to be unverified pieces of information that give rise to false rumours. Our study reinforces the need for developing robust machine learning techniques that can provide assistance for assessing the veracity of rumours.
BibTeX
@misc{zubiaga2015analysing,
  author = {Arkaitz Zubiaga and Maria Liakata and Rob Procter and Geraldine Wong Sak Hoi and Peter Tolmie},
  title = {{Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads}},
  year = {2015},
  eprint = {1511.07487},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1511.07487},
}
Figure