Twitter Crossfire: Terror Attack Detection via Probabilistic Classifiers

Herman Wandabwa, Zhifang Liao, Korir SammyInternational Journal of Database Theory and Application 2015 · DOI

Abstract

The advent of social computing brought with it different social networking platforms. The idea of surfers socializing with people of different backgrounds as well as geographical regions is quite fascinating. In our approach, we delved deeper in disaster discovery whereby we extracted panic related attributes and trained them with real data in three disaster scenarios in different parts of the world. Fine tuning of the final attributes led to accuracies above 91% proving the fact that with proper attribute selection and handling of sparse data balance, it's possible to detect related disasters as soon as related tweets appear. We believe that we are the first to use probabilistic classifiers approach as well as NLP in specifically human induced terror attacks detection as there is no known system currently that solely caters for these.

Introduction

While Twitter has long been recognized as a goldmine for real-time information (often called "citizen journalism"), the sheer volume of "noise"—chatter, ads, and irrelevant updates—makes it difficult to extract actionable intelligence during a crisis.

Previous research successfully used Twitter to detect natural disasters like earthquakes by tracking volumetric spikes in specific keywords. However, human-induced terror attacks are more complex to identify via simple keyword counts. This research addresses a critical gap: the lack of a dedicated system designed to filter the "Twitter sphere" for the unique semantic signatures of terror attacks, which often manifest as specific patterns of panic and distress across different geographical regions.

Methodology

The researchers proposed a multi-stage pipeline to transform raw tweets into accurate classifications:

  1. Data Acquisition: Using Twitter’s Streaming API to collect data during known crisis windows.
  2. Preprocessing: Cleaning the data by removing symbols, human errors, and irrelevant metadata.
  3. Feature Extraction: Identifying 1,602 initial attributes, with a heavy focus on "panic" related semantics.
  4. Classification: The study compared three distinct machine learning approaches: Multinomial Naïve Bayes (MNB), Sequential Minimal Optimization (SMO), and J48 Decision Trees.
  5. Optimization: To handle inherent class imbalance, a Resample filter was used. Information Gain (InfoGain) and a Ranker search method identified the top 100 most influential attributes, effectively discarding noise.

Datasets

The study utilized three real-world datasets collected during major terror events in 2014. These datasets provided a diverse geographical and cultural context for testing the classifiers.

Summary of the three terror event datasets used for model training and evaluation.

From these events, 4,927 instances were sampled for training and evaluation, categorized into "NORMAL" and "DISASTER" strata.

Key Insights

  • The "Social Sensor" Concept: Every Twitter user acts as a mobile sensor. Collective changes in text volume and sentiment serve as triggers for detection.
  • Panic as a Metric: Unlike natural disasters identified by nouns (e.g., "earthquake"), terror attacks are best identified through the semantics of panic. Measuring the "panic" attribute significantly augmented detection accuracy.
  • The Sparsity Challenge: Standard accuracy metrics can be misleading on raw, imbalanced Twitter data. True performance emerges after resampling classes to a uniform distribution.
  • Feature Pruning: Only the top 100 of over 1,600 attributes were paramount for prediction. Ranking features by Information Gain was the key to achieving accuracies over 90%.

Results and Discussion

The fine-tuned models achieved impressive results, with accuracies exceeding 91%. While Multinomial Naïve Bayes performed better on raw, unbalanced data, SMO (SVM) emerged as the superior classifier once the dataset was balanced and features were ranked.

ROC curves comparing the performance of MNB, SMO, and J48 classifiers.

The Area Under the Curve (AUC) for the optimized models showed a significant "lift" compared to baseline classifiers, demonstrating that proper attribute selection is more critical than the specific algorithm choice.

Conclusion and Future Work

Achieving 100% accuracy remains a challenge due to the evolving nature of social media language, including slang and code-switching. Future research will explore ensemble methods like boosting and bagging, more sophisticated feature combination techniques to capture context, and improved methods for handling class imbalance in real-time streaming environments.

Summary updated

Abstract

The advent of social computing brought with it different social networking platforms. The idea of surfers socializing with people of different backgrounds as well as geographical regions is quite fascinating. In our approach, we delved deeper in disaster discovery whereby we extracted panic related attributes and trained them with real data in three disaster scenarios in different parts of the world. Fine tuning of the final attributes led to accuracies above 91% proving the fact that with proper attribute selection and handling of sparse data balance, it's possible to detect related disasters as soon as related tweets appear. We believe that we are the first to use probabilistic classifiers approach as well as NLP in specifically human induced terror attacks detection as there is no known system currently that solely caters for these.
BibTeX
@article{wandabwa2015twitter,
  author = {Herman Wandabwa and Zhifang Liao and Korir Sammy},
  title = {{Twitter Crossfire: Terror Attack Detection via Probabilistic Classifiers}},
  journal = {International Journal of Database Theory and Application},
  year = {2015},
  doi = {10.14257/ijdta.2015.8.4.05},
}
Figure