Abstract
During time-critical situations such as natural disasters, rapid classification of data posted on social networks by affected people is useful for humanitarian organizations to gain situational awareness and to plan response efforts. However, the scarcity of labeled data in the early hours of a crisis hinders machine learning tasks thus delays crisis response. In this work, we propose to use an inductive semi-supervised technique to utilize unlabeled data, which is often abundant at the onset of a crisis event, along with fewer labeled data. Specifically, we adopt a graph-based deep learning framework to learn an inductive semi-supervised model. We use two real-world crisis datasets from Twitter to evaluate the proposed approach. Our results show significant improvements using unlabeled data as compared to only using labeled data.
Introduction
In the wake of natural or man-made disasters, social media platforms like Twitter become vital channels for real-time information. Affected individuals post critical updates regarding injuries, infrastructure damage, and urgent needs. For humanitarian organizations, the ability to automatically filter these massive streams into actionable categories is essential.
While supervised learning models are typically used for this task, they require substantial labeled datasets to achieve high accuracy. In the immediate aftermath of a crisis, labeled data is extremely scarce, creating a bottleneck for deployment. Conversely, unlabeled data is instantly available in high volumes. This research addresses the challenge of leveraging this abundant unlabeled data to train accurate classifiers during the critical early hours of an emergency.
Methodology
The proposed approach utilizes an inductive semi-supervised learning framework that integrates a standard text-classification Convolutional Neural Network (CNN) with a graph-based regularization component.
Architecture
The core model is a dual-branch CNN. As shown in the architecture diagram below, the process begins with pre-trained word embeddings (trained on 364 million historical crisis tweets). These embeddings pass through convolution and max-pooling layers to extract high-level features.
- Supervised Branch: This branch uses the extracted features to predict the class label (e.g., relevant vs. irrelevant) via a Softmax layer.
- Semi-supervised Branch: This branch utilizes the same features to predict a tweet's "context nodes"—its neighbors in a k-nearest neighbor (k-NN) similarity graph constructed from the embeddings.
By optimizing both branches simultaneously, the model uses the graph context as a regularizer, forcing the network to learn representations that respect the underlying structure of the data distribution. Crucially, the model is inductive, meaning it can process new, streaming tweets in real-time without recalculating the entire similarity graph.
Experimental Setup
The model was evaluated on two real-world crisis datasets with crowdsourced binary labels (relevant vs. irrelevant):
- 2015 Nepal Earthquake: ~11.6k labeled tweets and ~50k unlabeled tweets.
- 2013 Queensland Floods: ~10k labeled tweets and ~21k unlabeled tweets.
Results and Discussion
The semi-supervised approach consistently outperformed the supervised-only baseline across various data volumes. The inclusion of unlabeled data led to absolute F-measure improvements ranging from 5% to 26%.
As illustrated in the results table, the model is particularly effective in low-resource settings. For the Queensland Flood dataset, using only 500 labeled tweets, the F-measure increased from 60.14% (supervised) to 85.54% (semi-supervised). When all available labeled data was used, the model reached peak F-measures of 66.63% for Nepal and 93.54% for Queensland.
Key Insights
- Graph Regularization: Predicting graph neighbors helps the model learn a more robust feature space, effectively utilizing the structure of unlabeled data to improve classification.
- Streaming Readiness: The inductive nature of the CNN architecture is vital for disaster response, where data arrives in high-velocity streams and must be classified instantly.
Conclusion
This work demonstrates that graph-based semi-supervised learning can significantly bridge the data gap during the early stages of a crisis. By effectively utilizing unlabeled data, humanitarian organizations can deploy accurate classifiers much faster than with supervised methods alone. Future work will explore domain adaptation and zero-shot learning to further minimize the need for manual labeling in the initial hours of a disaster.
Summary updated