Graph Based Semi-Supervised Learning with Convolution Neural Networks to Classify Crisis Related Tweets

Firoj Alam, Shafiq R. Joty, Muhammad Imran

Abstract

During natural or man-made disasters, humanitarian organizations rely on social media for situational awareness. However, the scarcity of labeled data in the initial hours of a crisis hinders the deployment of effective supervised learning models. This paper presents a semi-supervised learning framework that combines Convolutional Neural Networks (CNNs) with graph-based similarity to classify crisis-related tweets. By leveraging abundant unlabeled data through a similarity graph, the proposed inductive model achieves significant performance gains—up to 26%—over standard supervised methods. This approach enables rapid and accurate classification of critical information, such as infrastructure damage or urgent medical needs, even when labeled data is extremely limited.

Introduction

The rapid identification of relevant information on social media is critical for disaster response. While thousands of tweets are generated every minute during a crisis, only a small fraction are labeled for training purposes. Traditional supervised models struggle in these low-data regimes. This research addresses the challenge of utilizing the vast pool of unlabeled data to "boost" model performance. Unlike transductive graph-based methods, the proposed approach is inductive, allowing for the real-time classification of new, unseen tweets without re-constructing the similarity graph.

Methodology

The authors adopt a graph-based deep learning framework that integrates a CNN with a graph-based loss function. The architecture consists of two main components:

  1. Graph Construction: A similarity graph is constructed where each tweet is a node. Edges are established between a tweet and its 10 nearest neighbors ($k=10$) based on the Euclidean distance of their averaged word embeddings.
  2. Model Architecture: The model features a CNN branch for feature extraction from tweet text and a graph branch that predicts a tweet's "context" (its neighbors in the similarity graph).
The proposed semi-supervised architecture integrating a CNN with a graph-based loss function.

The final loss function is a weighted combination of the supervised classification loss and the unsupervised graph context loss. This joint optimization forces the network to learn representations that respect the natural clusters found in the unlabeled data.

Experimental Setup

The framework was evaluated using two real-world crisis datasets: the 2015 Nepal Earthquake and the 2013 Queensland Floods. The study utilized specialized word2vec embeddings trained on 364 million crisis-related tweets. Unlabeled data pools consisted of 50,000 tweets for Nepal and approximately 21,000 for Queensland.

Results and Discussion

The semi-supervised approach consistently outperformed the supervised-only baseline across various labeled data sizes.

F-measure performance comparison between supervised (L) and semi-supervised (L+U) models across different labeled sample sizes.

Key findings include:

  • Performance Gains: Improvements in F-measure ranged from 5% to 26% absolute.
  • Low-Data Efficacy: The impact was most pronounced when labeled data was scarce. For instance, in the Queensland dataset with only 100 labeled instances, the F-measure increased from 58.52% to 75.08% upon adding unlabeled data.
  • Early Response Viability: The results suggest that as few as 500 labeled instances, combined with unlabeled data, can provide a robust model for the onset of a crisis.

Conclusion

This paper demonstrates that integrating graph-based context into a CNN framework significantly enhances tweet classification in data-scarce environments. The inductive nature of the model makes it a practical tool for real-time disaster response. Future work will explore hyperparameter tuning, multi-class classification for more granular needs assessment, and domain adaptation across different types of disasters.

Summary updated

Abstract

During time-critical situations such as natural disasters, rapid classification of data posted on social networks by affected people is useful for humanitarian organizations to gain situational awareness and to plan response efforts. However, the scarcity of labeled data in the early hours of a crisis hinders machine learning tasks thus delays crisis response. In this work, we propose to use an inductive semi-supervised technique to utilize unlabeled data, which is often abundant at the onset of a crisis event, along with fewer labeled data. Specif- ically, we adopt a graph-based deep learning framework to learn an inductive semi-supervised model. We use two real-world crisis datasets from Twitter to evaluate the proposed approach. Our results show significant improvements using unlabeled data as compared to only using labeled data.
BibTeX
@inproceedings{alam2018graph,
  author = {Firoj Alam and Shafiq R. Joty and Muhammad Imran},
  title = {{Graph Based Semi-Supervised Learning with Convolution Neural Networks to Classify Crisis Related Tweets}},
  booktitle = {ICWSM},
  year = {2018},
  url = {https://openreview.net/forum?id=KhtJSg4LFw},
}
Figure