Graph Based Semi-supervised Learning with Convolution Neural Networks to Classify Crisis Related Tweets

Firoj Alam, Shafiq R. Joty, Muhammad Imran

Abstract

During time-critical situations such as natural disasters, rapid classification of data posted on social networks by affected people is useful for humanitarian organizations to gain situational awareness and to plan response efforts. However, the scarcity of labeled data in the early hours of a crisis hinders machine learning tasks thus delays crisis response. In this work, we propose to use an inductive semi-supervised technique to utilize unlabeled data, which is often abundant at the onset of a crisis event, along with fewer labeled data. Specifically, we adopt a graph-based deep learning framework to learn an inductive semi-supervised model. We use two real-world crisis datasets from Twitter to evaluate the proposed approach. Our results show significant improvements using unlabeled data as compared to only using labeled data.

Introduction

In the wake of natural or man-made disasters, social media platforms like Twitter become vital channels for real-time information. Affected individuals post critical updates regarding injuries, infrastructure damage, and urgent needs. For humanitarian organizations, the ability to automatically filter these massive streams into actionable categories is essential.

While supervised learning models are typically used for this task, they require substantial labeled datasets to achieve high accuracy. In the immediate aftermath of a crisis, labeled data is extremely scarce, creating a bottleneck for deployment. Conversely, unlabeled data is instantly available in high volumes. This research addresses the challenge of leveraging this abundant unlabeled data to train accurate classifiers during the critical early hours of an emergency.

Methodology

The proposed approach utilizes an inductive semi-supervised learning framework that integrates a standard text-classification Convolutional Neural Network (CNN) with a graph-based regularization component.

Architecture

The core model is a dual-branch CNN. As shown in the architecture diagram below, the process begins with pre-trained word embeddings (trained on 364 million historical crisis tweets). These embeddings pass through convolution and max-pooling layers to extract high-level features.

Architecture of the proposed graph-based semi-supervised CNN model, featuring dual branches for classification and graph context prediction.
  1. Supervised Branch: This branch uses the extracted features to predict the class label (e.g., relevant vs. irrelevant) via a Softmax layer.
  2. Semi-supervised Branch: This branch utilizes the same features to predict a tweet's "context nodes"—its neighbors in a k-nearest neighbor (k-NN) similarity graph constructed from the embeddings.

By optimizing both branches simultaneously, the model uses the graph context as a regularizer, forcing the network to learn representations that respect the underlying structure of the data distribution. Crucially, the model is inductive, meaning it can process new, streaming tweets in real-time without recalculating the entire similarity graph.

Experimental Setup

The model was evaluated on two real-world crisis datasets with crowdsourced binary labels (relevant vs. irrelevant):

  • 2015 Nepal Earthquake: ~11.6k labeled tweets and ~50k unlabeled tweets.
  • 2013 Queensland Floods: ~10k labeled tweets and ~21k unlabeled tweets.

Results and Discussion

The semi-supervised approach consistently outperformed the supervised-only baseline across various data volumes. The inclusion of unlabeled data led to absolute F-measure improvements ranging from 5% to 26%.

F-measure performance comparison between supervised (L) and semi-supervised (L+U) models across different labeled data sizes.

As illustrated in the results table, the model is particularly effective in low-resource settings. For the Queensland Flood dataset, using only 500 labeled tweets, the F-measure increased from 60.14% (supervised) to 85.54% (semi-supervised). When all available labeled data was used, the model reached peak F-measures of 66.63% for Nepal and 93.54% for Queensland.

Key Insights

  • Graph Regularization: Predicting graph neighbors helps the model learn a more robust feature space, effectively utilizing the structure of unlabeled data to improve classification.
  • Streaming Readiness: The inductive nature of the CNN architecture is vital for disaster response, where data arrives in high-velocity streams and must be classified instantly.

Conclusion

This work demonstrates that graph-based semi-supervised learning can significantly bridge the data gap during the early stages of a crisis. By effectively utilizing unlabeled data, humanitarian organizations can deploy accurate classifiers much faster than with supervised methods alone. Future work will explore domain adaptation and zero-shot learning to further minimize the need for manual labeling in the initial hours of a disaster.

Summary updated

Abstract

During time-critical situations such as natural disasters, rapid classification of data posted on social networks by affected people is useful for humanitarian organizations to gain situational awareness and to plan response efforts. However, the scarcity of labeled data in the early hours of a crisis hinders machine learning tasks thus delays crisis response. In this work, we propose to use an inductive semi-supervised technique to utilize unlabeled data, which is often abundant at the onset of a crisis event, along with fewer labeled data. Specif- ically, we adopt a graph-based deep learning framework to learn an inductive semi-supervised model. We use two real-world crisis datasets from Twitter to evaluate the proposed approach. Our results show significant improvements using unlabeled data as compared to only using labeled data.
BibTeX
@inproceedings{alam2018graph,
  author = {Firoj Alam and Shafiq R. Joty and Muhammad Imran},
  title = {{Graph Based Semi-supervised Learning with Convolution Neural Networks to Classify Crisis Related Tweets}},
  booktitle = {ICWSM},
  year = {2018},
  url = {https://openreview.net/forum?id=KhtJSg4LFw},
}
Figure