HTCNN-Attn: a fine-grained hierarchical multi-label deep learning model for disaster emergency information intelligent extraction from social media

Shanshan Li, Qingjie Liu, Xiaoling SunPeerJ Comput. Sci 2025 · DOI

TL;DR

This paper introduces HTCNN-Attn, a novel deep learning model that excels at extracting detailed, hierarchical emergency information from noisy social media during disasters. By combining Transformer and CNN architectures with a unique hierarchical attention mechanism and loss function, it achieves state-of-the-art performance in classifying disaster-related content with unprecedented accuracy and efficiency, crucial for rapid emergency response.

What Problem Does This Paper Address?

Disasters generate a massive influx of social media data, which is vital for situational awareness and effective emergency response. However, this data is often noisy, unstructured, and requires fine-grained categorization to be useful for different emergency agencies. Traditional methods struggle with the sheer volume and complexity of this information, often providing only broad classifications. The challenge is to accurately and efficiently extract specific, actionable details from this data, respecting the hierarchical nature of emergency needs (e.g., distinguishing between a general "aid request" and a specific "request for water").

Methods

The core of the proposed solution is the HTCNN-Attn model, which integrates several key components:

  • Hierarchical Labeling Architecture: A three-level tree structure is employed to categorize disaster information, moving from broad disaster identification to specific needs and conditions.
  • Transformer-based Global Feature Extraction: A Transformer encoder is used to capture long-range dependencies and contextual information within the social media text.
  • CNN Layers for Local Pattern Capture: Convolutional Neural Networks (CNNs) are utilized to extract local patterns and n-gram features, complementing the Transformer's global view.
  • Hierarchical Attention Mechanism: This mechanism allows the model to focus on the most relevant parts of the text, guided by the hierarchical structure of the labels, enhancing feature representation.
  • Hierarchical Loss Function: This novel loss function combines weighted loss (giving more importance to lower-level labels) and a consistency penalty to ensure that predictions adhere to the hierarchical label dependencies.

Datasets

The study utilizes two primary datasets:

  • Appen: A dataset containing 26,248 labeled messages from various historical disasters worldwide.
  • HumAID: A dataset with 77,196 annotated messages from 2016-2019, covering 11 categories.

For cross-event generalization testing, the CrisisBench dataset was used, specifically focusing on the 2012 Hurricane Sandy event, which was not part of the training data.

Key Insights and Ideas

  • Hierarchy Matters: Explicitly modeling the hierarchical relationships between disaster information categories significantly improves classification accuracy and logical consistency.
  • Synergy of Transformers and CNNs: Combining Transformer's global context understanding with CNN's local feature extraction provides a more robust representation of social media text.
  • Attention for Granularity: A hierarchical attention mechanism is crucial for focusing on specific details relevant to fine-grained categorization within the disaster context.
  • Loss Function for Consistency: A specialized hierarchical loss function is essential for enforcing label dependencies and ensuring semantically consistent predictions across all levels of the hierarchy.
  • Efficiency for Real-time: The model achieves a remarkable balance between high accuracy and low latency, making it suitable for real-time emergency response applications.

Key Results

  • State-of-the-Art Performance: HTCNN-Attn achieved an accuracy of 0.9725, a Micro-F1 of 0.7402, and a hierarchical consistency (HC) score of 0.821 on the Appen and HumAID datasets, outperforming existing baselines.
  • Robust Generalization: On the CrisisBench dataset, the model demonstrated strong generalization with an HC score of 0.678, even on unseen disaster scenarios.
  • High Efficiency: The model boasts a low latency of 12.0 ms, making it suitable for real-time deployment, and requires significantly less memory and training time compared to other deep learning baselines.
  • Ablation Study Validation: Each component of the HTCNN-Attn model (hierarchical structure, Transformer, CNN, attention) was shown to contribute significantly to its overall performance.

Limitations and Future Work

The authors acknowledge several limitations:

  • Dataset Dependency: The reliance on pre-trained embeddings and large labeled datasets limits applicability in low-resource or multilingual environments.
  • Untested on Non-English Data: The model's performance on platforms like Weibo and in zero-shot scenarios remains untested.
  • Privacy Measures: While basic privacy measures are included, more advanced techniques like federated learning and differential privacy could be integrated.

Future work directions include:

  • Exploring lightweight model architectures and transfer learning for reduced data dependency.
  • Developing multilingual NLP techniques for cross-lingual generalization.
  • Integrating zero-shot learning frameworks for unseen disaster types.
  • Developing hybrid architectures with edge-based federated learning for enhanced privacy.
  • Enhancing PII redaction with dynamic differential privacy mechanisms.
  • Investigating the integration of MTEB models' generic embeddings with explicit hierarchical architectures.

Summary updated

BibTeX
@article{li2025htcnn,
  author = {Shanshan Li and Qingjie Liu and Xiaoling Sun},
  title = {{HTCNN-Attn: a fine-grained hierarchical multi-label deep learning model for disaster emergency information intelligent extraction from social media}},
  journal = {PeerJ Comput. Sci.},
  year = {2025},
  doi = {10.7717/peerj-cs.2992},
}
Figure