Call Attention to Rumors: Deep Attention Based Recurrent Neural Networks for Early Rumor Detection

Tong Chen, Lin Wu, Xue Li, Jun Zhang, Hongzhi Yin, Yang Wang

TL;DR

The researchers introduce CallAtRumors, a deep learning framework that uses Recurrent Neural Networks (RNNs) combined with a soft-attention mechanism to detect rumors on social media in their infancy. By automatically focusing on the most informative textual cues within a stream of posts, the model outperforms traditional methods and can identify rumors significantly faster than human fact-checkers.

What Problem Does This Paper Address?

Rumors spread like wildfire on social media, often causing real-world harm—such as the 2013 incident where a hacked Associated Press tweet about an explosion at the White House wiped $136.5 billion off the stock market in minutes.

Detecting these rumors early is critical, but it’s a massive technical challenge for three main reasons:

  1. Manual Labor: Traditional models rely on "hand-crafted" features (like sentiment analysis or user profile data), which are labor-intensive to build and don't scale well.
  2. High Duplication: Social media is full of retweets and repetitive phrases. As shown in Figure 1, a few keywords like "Donald Trump" and "Disqualified" appear constantly, creating a "noise" problem that makes it hard to find distinctive patterns.
  3. Temporal Evolution: The way people talk about a rumor changes over time—moving from surprise to belief, and eventually to skepticism or debunking. Models need to capture this shifting context.
Figure 1: Textual duplication in social media posts regarding a specific event.

Methods

The authors propose CallAtRumors, a model built on Long Short-Term Memory (LSTM) networks—a type of RNN designed to handle long-range dependencies in sequential data.

The core innovation is the integration of a deterministic soft-attention mechanism directly into the recurrence. Here is how the pipeline works:

  • Preprocessing: Instead of treating every tweet individually, the model groups posts into batches (time intervals) to ensure a consistent "word density." These are converted into TF-IDF vectors of the top 10,000 most frequent words.
  • Selective Focus: The attention mechanism calculates a weight for each word in the input matrix. It essentially asks, "Which words in this batch are the most relevant for identifying a rumor?"
  • Temporal Learning: The LSTM layers process these weighted inputs over time, capturing how the conversation evolves.
  • Classification: A final sigmoid layer outputs the probability of the event being a rumor.
Figure 2: Schematic overview of our framework. In regards to each event, posts in sequence are collected and transformed into their tf-idf vectors. Ten, deep recurrent neural networks augmented with sof-attention mechanism are deployed to derive temporal latent representations by capturing long-term dependency among post series and selectively focusing on important relevance. Additional layer is topped upon learned representations to determine the event to be rumor/non-rumor.

Datasets

The model was evaluated on two large-scale, real-world datasets:

  • Twitter: 996 events (498 rumors, 498 non-rumors) collected via Snopes.
  • Weibo: 4,702 events (2,351 rumors, 2,351 non-rumors) collected from the Sina Community Management Center.

These datasets are particularly challenging because they contain roughly 80% duplicate content, providing a rigorous test for the model's attention mechanism.

Key Insights and Ideas

  • Attention as a Denoising Filter: The "aha" moment is that attention isn't just for finding important words; it’s for ignoring the massive amount of repetitive "noise" in social media streams. By weighting distinctive tokens higher, the model overcomes the scarcity of unique textual features.
  • Capturing the "Skepticism" Phase: The model learns to focus on expressions of doubt or inquiry (e.g., "really?", "sure?", "how?") rather than just the core event keywords. Figure 6 visualizes this, showing higher attention degrees on words expressing user anger or questioning.
  • End-to-End Learning: Unlike previous models that required complex feature engineering, CallAtRumors learns everything from raw TF-IDF inputs, making it much more flexible for different types of rumors.
Figure 6: Visualization of attention degrees on a detected rumor.

Key Results

CallAtRumors consistently outperformed state-of-the-art baselines (like SVM-TS and ML-GRU) across both datasets:

  • Accuracy: On Twitter, it achieved an F-measure of 0.8715, significantly higher than the 0.8191 achieved by the standard multi-layer GRU.
  • Earliness: This is the model's strongest suit. It can detect rumors with high precision using only the first 10-20% of the posts in an event's lifecycle.
  • Human vs. AI: The model identifies rumors with an average time lag of about 20.5 hours. For comparison, human fact-checking services like Snopes and Sina take an average of 54 to 72 hours to debunk the same rumors.

Limitations and Future Work

While highly effective, the authors note a few areas for improvement:

  • Propagation Structure: The current model focuses primarily on textual content. Future iterations could incorporate the "social graph"—how the rumor spreads from user to user—as a sequential input to further boost accuracy.
  • Efficiency: Dealing with massive, multi-level feature spaces is computationally expensive. The authors suggest investigating hashing techniques to make the model even faster for real-time deployment.
  • Feature Complexity: The authors aim to explore combining more complex features (like user metadata or image data) with the deep attention model.

Summary updated

Abstract

The proliferation of social media in communication and information dissemination has made it an ideal platform for spreading rumors. Automatically debunking rumors at their stage of diffusion is known as \textit{early rumor detection}, which refers to dealing with sequential posts regarding disputed factual claims with certain variations and highly textual duplication over time. Thus, identifying trending rumors demands an efficient yet flexible model that is able to capture long-range dependencies among postings and produce distinct representations for the accurate early detection. However, it is a challenging task to apply conventional classification algorithms to rumor detection in earliness since they rely on hand-crafted features which require intensive manual efforts in the case of large amount of posts. This paper presents a deep attention model on the basis of recurrent neural networks (RNN) to learn \textit{selectively} temporal hidden representations of sequential posts for identifying rumors. The proposed model delves soft-attention into the recurrence to simultaneously pool out distinct features with particular focus and produce hidden representations that capture contextual variations of relevant posts over time. Extensive experiments on real datasets collected from social media websites demonstrate that (1) the deep attention based RNN model outperforms state-of-the-arts that rely on hand-crafted features; (2) the introduction of soft attention mechanism can effectively distill relevant parts to rumors from original posts in advance; (3) the proposed method detects rumors more quickly and accurately than competitors.
BibTeX
@inproceedings{chen2018call,
  author = {Tong Chen and Lin Wu and Xue Li and Jun Zhang and Hongzhi Yin and Yang Wang},
  title = {{Call Attention to Rumors: Deep Attention Based Recurrent Neural Networks for Early Rumor Detection}},
  booktitle = {PAKDD (Workshops)},
  year = {2018},
  doi = {10.1007/978-3-030-04503-6\_4},
}
Figure