Unsupervised Detection of Sub-Events in Large Scale Disasters

Chidubem Arachie, Manas Gaur, Sam Anzaroot, William Groves, Ke Zhang, Alejandro JaimesAAAI 2020 · DOI

Abstract

Social media platforms like Twitter provide critical real-time information during large-scale disasters. However, the volume of data makes manual extraction of specific "sub-events"—localized incidents like infrastructure damage or health crises—impossible. We present an unsupervised framework that combines linguistic dependency parsing with phrase extraction and domain-specific ontologies to identify and cluster these sub-events. By ranking candidate mentions against a crisis-specific ontology and applying spectral clustering, our system significantly reduces noise and improves detection accuracy compared to existing baselines. Evaluation on Hurricane Harvey and Nepal Earthquake datasets demonstrates that incorporating phrase detection and aggressive filtering leads to superior F1-scores and more cohesive event clusters.

Introduction

During crises such as hurricanes or earthquakes, social media becomes a vital tool for situational awareness. Emergency responders and policymakers rely on these updates to identify sub-events: specific, localized incidents within the broader disaster (e.g., a bridge collapse or a hospital supply shortage).

The primary challenge is the "deluge of chatter"—millions of posts that make manual monitoring unfeasible. Traditional automated methods often rely on supervised learning, which fails to generalize to new types of disasters. Existing unsupervised approaches frequently focus on simple "noun-verb" (NV) pairs, which can miss critical context like "contaminated water" or "shelter needs." This paper addresses these gaps with a robust, unsupervised pipeline designed for high-noise environments.

Methodology

The proposed framework utilizes a three-stage unsupervised pipeline to transform raw social media data into actionable insights.

The three-stage unsupervised pipeline for sub-event detection and clustering.
  1. Extraction: The system employs the spaCy dependency parser to identify Noun-Verb (NV) pairs. To capture states or conditions that lack a standard NV structure, it integrates a Gensim phrase model to extract frequent n-gram phrases (e.g., "infectious disease").
  2. Ranking via Ontology: Candidate sub-events are filtered using the Management of Crisis (MOAC) ontology, which contains 62 crisis-related terms. Both candidates and ontology terms are represented as vectors using FastText embeddings trained on a corpus of 53 million crisis tweets. Candidates are then ranked based on their cosine similarity to the MOAC terms.
  3. Clustering: The top-ranked sub-events undergo Spectral Clustering, organizing them into cohesive categories such as "Infrastructure Damage" or "Public Health" for decision-makers.

Experimental Evaluation

The framework was validated using two large-scale disaster datasets:

  • Hurricane Harvey (2017): Comprising 4.6 million unlabeled tweets.
  • 2015 Nepal Earthquake: Comprising 635,150 unlabeled tweets. Human-labeled subsets (4,000 and 4,639 tweets respectively) were used to verify if the system's top-ranked sub-events corresponded to informative content.

Results and Discussion

A key finding of this research is the importance of phrase detection and aggressive filtering. While NV pairs capture actions, phrases like "water contamination" are essential for medical response but are often invisible to NV-only parsers.

F1-score comparison on the Hurricane Harvey dataset, showing the impact of phrase extraction and filtering.

As shown in the performance metrics for Hurricane Harvey, the "MOAC+NV+Phrases filtered" approach significantly outperforms the DEPSUB baseline and non-filtered variants. By pruning sub-events that appeared only once and ranking them against the crisis ontology, the authors reduced candidate sub-events by over 90% (and up to 99.6% for Harvey). This reduction in noise directly translated to a higher F1-score, demonstrating that precision in candidate selection is more valuable than raw recall in disaster scenarios.

Qualitatively, human evaluators via Amazon Mechanical Turk confirmed that the resulting clusters were significantly more cohesive than random samples, providing clear, actionable categories for responders.

Conclusion

This unsupervised framework offers a scalable solution for extracting situational awareness from social media during disasters. By combining dependency parsing with phrase extraction and domain-specific ontology ranking, the system effectively filters the noise of large-scale datasets. Future work will focus on adapting this retrospective analysis into a real-time system capable of tracking evolving sub-events as they happen and expanding support to multi-lingual disaster data.

Summary updated

BibTeX
@inproceedings{arachie2020unsupervised,
  author = {Chidubem Arachie and Manas Gaur and Sam Anzaroot and William Groves and Ke Zhang and Alejandro Jaimes},
  title = {{Unsupervised Detection of Sub-Events in Large Scale Disasters}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i01.5370},
}
Figure