On Identifying Hashtags in Disaster Twitter Data

Jishnu Ray Chowdhury, Cornelia Caragea, Doina Caragea

Introduction

During natural disasters, social media platforms like Twitter serve as critical lifelines for affected individuals and first responders. However, a significant "information gap" often hinders emergency response. Much of the data shared is noisy, and critical rescue requests or damage reports are frequently missed. Two primary hurdles exist: approximately half of disaster-related tweets contain no hashtags at all, and when they do, the tags are often either too generic (e.g., #HurricaneIrma) or too idiosyncratic to be useful for filtering specific needs like "power loss" or "blocked street."

To address this, researchers have shifted the focus from traditional "hashtag recommendation"—which selects from a pre-defined list—to hashtag extraction. This approach identifies the most informative terms directly from the tweet text to generate "synthetic" hashtags in real-time.

Methods and Architecture

The proposed system utilizes a sequence labeling approach powered by a Joint-Layer Long Short-Term Memory (LSTM) network within a Multi-Task Learning (MTL) framework. This architecture allows the model to simultaneously learn two tasks: identifying single words that could be part of a hashtag and labeling multi-word phrases of arbitrary length.

The multi-task deep learning architecture incorporating phonetic (IPA) and contextual (ELMo) embeddings.

A key innovation is the model's phonetic awareness. To handle the informal nature of Twitter—including slang, typos, and phonetic spellings—the researchers used tools like Epitran and Panphon to convert text into International Phonetic Alphabet (IPA) representations. These features, processed via a CNN, help the model recognize words based on how they "sound" rather than just how they are spelled. Additionally, ELMo (Embeddings from Language Models) was integrated to capture contextual nuances, such as distinguishing whether "Harvey" refers to a person or a hurricane.

Dataset and Annotation

The study introduces a massive new benchmark dataset comprising 67,288 tweets across 37 different disasters, including hurricanes, floods, and earthquakes. To annotate this corpus, the authors developed a "disaster lexicon" of 2,430 unique phrases covering locations, disaster names, and situational awareness terms. This semi-automated annotation was validated via human judges on Amazon Mechanical Turk, with an 89% approval rating for accuracy.

Examples of disaster tweets and the corresponding hashtags extracted by the model to identify locations and needs.

Key Results

The model variants were evaluated across several test sets, including specific events like the Maria Hurricane and California Fires.

  • High Accuracy: The most advanced model configuration (LSTM-MTL + ELMo + IPA + POS) achieved a peak F1-score of 92.22% on a multi-disaster dataset.
  • Ablation Insights: The inclusion of ELMo and phonetic features consistently improved performance, proving that "noise" in social media text can be converted into a useful signal.
  • Zero-Shot Capability: The model demonstrated strong generalization, maintaining an F1-score of approximately 89% on disasters it had never seen during training (such as the California Fires). This suggests the system is robust enough for deployment during future, unforeseen emergencies.

Conclusion

By treating hashtagging as a keyphrase discovery problem rather than a classification task, this research provides a more flexible and robust tool for emergency response. Future work aims to expand this system to support multiple languages and add hierarchical layers to classify the type of extracted information (e.g., distinguishing a "Location" from a "Rescue Need") to further streamline actionable data for first responders.

Summary updated

Abstract

Tweet hashtags have the potential to improve the search for information during disaster events. However, there is a large number of disaster-related tweets that do not have any user-provided hashtags. Moreover, only a small number of tweets that contain actionable hashtags are useful for disaster response. To facilitate progress on automatic identification (or extraction) of disaster hashtags for Twitter data, we construct a unique dataset of disaster-related tweets annotated with hashtags useful for filtering actionable information. Using this dataset, we further investigate Long Short Term Memory-based models within a Multi-Task Learning framework. The best performing model achieves an F1-score as high as 92.22%. The dataset, code, and other resources are available on Github.
BibTeX
@inproceedings{chowdhury2020identifying,
  author = {Jishnu Ray Chowdhury and Cornelia Caragea and Doina Caragea},
  title = {{On Identifying Hashtags in Disaster Twitter Data}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i01.5387},
}
Figure