Speech acts and Communicative Intentions for Urgency Detection

Enzo Laurenti, Nils Bourgon, Farah Benamara, Alda Mari, Véronique Moriceau, Camille CourgeonNAACL 2022

TL;DR

This paper introduces a novel approach to improve urgency detection in crisis-related tweets by incorporating speech act (SA) analysis. By developing a new annotated dataset and deep learning models, they demonstrate that understanding the communicative intention behind a tweet significantly enhances its classification for urgency and information type, especially in challenging out-of-type scenarios.

What Problem Does This Paper Address?

Identifying urgent messages during crises is crucial for effective disaster response. While existing work focuses on classifying tweets based on their content and urgency, it often overlooks the underlying communicative intent. This paper addresses the gap by investigating how speech acts – the actions performed by language (e.g., asserting, requesting, commanding) – can provide valuable signals for better urgency detection, particularly in the context of social media where brevity and indirectness are common. The authors aim to bridge the understanding of how speakers' intentions, expressed through speech acts, correlate with the urgency and information type of their messages during natural disasters.

Methods

The core of this research lies in integrating speech act (SA) information into deep learning models for urgency detection. The authors propose two main ways to achieve this:

  • SA as Extra-Features: Speech act labels are treated as additional input features to existing language models (FlauBERT). This involves various encoding methods, with one-hot encoding proving most effective.
  • SA as an Auxiliary Task (Multitask Learning): Urgency detection is framed as a multitask learning problem where SA classification is performed alongside urgency-related tasks (relatedness, urgency, information type). This allows the model to jointly learn from both types of information, potentially uncovering deeper correlations.

They experiment with both monotask and multitask architectures, comparing SA-aware models against SA-agnostic baselines. The models are evaluated using two protocols: "out-of-event" (testing on unseen disaster types) and "out-of-type" (testing on a different crisis category than those seen during training).

Datasets

The study introduces a new dataset of 6,669 French tweets collected during ecological crises in metropolitan France. This dataset is annotated for:

  • Urgency: Categorized as URGENT, NON URGENT, and NOT USEFUL.
  • Information Type: Six actionable categories including HUMAN DAMAGES, MATERIAL DAMAGES, WARNING-ADVICE, SUPPORT, CRITICS, and OTHER.
  • Speech Acts (SA): Four categories: JUSSIVES, ASSERTIVES, INTERROGATIVES, and SUBJECTIVES, with an additional OTHERS category for unclassifiable cases.

This dataset is a valuable contribution as it specifically links speech acts to urgency in the context of disaster-related social media communication.

Key Insights and Ideas

  • Speech Acts Matter for Urgency: The fundamental insight is that understanding what a speaker is trying to do with their words (their communicative intention via speech acts) significantly improves the ability to detect urgency in crisis tweets.
  • Multitask Learning is Powerful: Jointly learning speech acts and urgency-related tasks (multitask learning) proves to be a highly effective strategy, often outperforming models that simply treat speech acts as additional features. This suggests a deeper, synergistic relationship between communicative intent and urgency.
  • Generalization is Key: The "out-of-type" evaluation setting, where models are tested on crisis types not seen during training, highlights the true value of SA. SA-aware models show much better generalization capabilities in these challenging scenarios, indicating their robustness.
  • Subjective vs. Assertive in Crises: The distribution of speech acts varies by crisis type. For instance, sudden events like building collapses tend to elicit more SUBJECTIVE (evaluative/emotional) tweets compared to more anticipated events like floods or hurricanes, which see a higher proportion of ASSERTIVE (factual) messages.

Key Results

  • Improved Performance: SA-enhanced models consistently outperform SA-agnostic baselines across both "out-of-event" and "out-of-type" evaluation settings for urgency and information type detection.
  • Out-of-Type Advantage: The benefits of SA are most pronounced in the "out-of-type" configuration, where models demonstrate better generalization. For example, multitask models (ML² and ML⁴) achieve good results, outperforming SA-agnostic baselines.
  • Specific Gains: In the "out-of-type" setting, SA injection improved urgency detection by +1.2% over the best SA-agnostic model. For information type classification, SA-aware models showed improvements, particularly for MATERIAL DAMAGES and WARNING ADVICE in the "out-of-type" scenario.
  • Statistical Significance: The correlation between speech acts and urgency categories is statistically significant, with a Cramer's V value of .28 confirming a moderate statistical correlation.

Limitations and Future Work

  • Limitations:
    • The study acknowledges that while SA detection is an important preliminary step, it is not the primary focus of this paper.
  • The dataset, while novel, is specific to French tweets and ecological crises.
  • The "out-of-event" and "out-of-type" experiments had some limitations due to small training set sizes for certain event groupings.
  • Future Work:
    • The authors plan to explore a finer-grained SA taxonomy for urgency detection.
  • Investigating the impact of SA on other crisis-related NLP tasks is also suggested.
  • Further work on improving SA detection itself is mentioned as a potential area for future research.

Summary updated

BibTeX
@inproceedings{laurenti2022speech,
  author = {Enzo Laurenti and Nils Bourgon and Farah Benamara and Alda Mari and Véronique Moriceau and Camille Courgeon},
  title = {{Speech acts and Communicative Intentions for Urgency Detection}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure