"Our Grief is Unspeakable": Automatically Measuring the Community Impact of a Tragedy

Kimberly Glasgow, Clayton Fink, Jordan L. Boyd-GraberICWSM 2014

Abstract

Social media offer a real-time, unfiltered view of how disasters affect communities. Crisis response, disaster mental health, and — more broadly — public health can benefit from automated analysis of the public’s mental state as exhibited on social media. Our focus is on Twitter data from a community that lost members in a mass shooting and another community—geographically removed from the shooting — that was indirectly exposed. We show that a common approach for understanding emotional response in text: Linguistic Inquiry and Word Count (LIWC) can be substantially improved using machine learning. Starting with tweets flagged by LIWC as containing content related to the issue of death, we devise a categorization scheme for death-related tweets to induce automatic text classification of such content. This improved methodology reveals striking differences in the magnitude and duration of increases in death-related talk between these communities. It also detects subtle shifts in the nature of death-related talk. Our results offer lessons for gauging public response and for developing interventions in the wake of a tragedy.

Introduction

In the immediate aftermath of a disaster, public health officials and crisis responders face the challenge of gauging a community's mental state to deploy resources effectively. While social media provides a real-time window into public sentiment, traditional analytical tools often lack the nuance required to distinguish between genuine trauma and everyday linguistic patterns.

The industry standard, Linguistic Inquiry and Word Count (LIWC), relies on dictionary-based keyword matching. For example, if a tweet contains the word "killing," LIWC flags it as death-related. However, this creates a significant "false positive" problem: a user tweeting "My throat is killing me" is expressing physical discomfort, not processing grief. This research addresses the need for a more sophisticated, automated method to measure the psychological impact of a tragedy by separating literal mentions of death from figurative or metaphorical language.

Methodology

To improve upon dictionary-based methods, the researchers implemented a supervised machine-learning pipeline focused on two distinct populations from December 2012 to mid-January 2013: Fairfield County, CT (directly affected by the Sandy Hook shooting) and Montgomery County, MD (a geographically removed control group with similar demographics).

Taxonomy and Classification

The team categorized death-related tweets into three primary buckets:

  1. Literal Death: Real-world loss of life (e.g., "rest in peace," "fatalities").
  2. Figurative Death: Metaphorical or hyperbolic use (e.g., "killing it," "dead battery").
  3. Proper Names: Entities with death-related names (e.g., the band "The Killers").

Using an active learning framework (DUALIST), they efficiently annotated a subset of a raw dataset containing 820,000 tweets to build a robust training set. They then trained a Support Vector Machine (SVM) classifier using unigrams, bigrams, Twitter-specific features (emoticons, hashtags), and the original LIWC "death" flag as a feature. This hybrid approach allowed the model to correct the "false confidence" of simpler keyword-matching models.

Results and Analysis

The "Sensitivity Shift"

One of the most significant findings was the discovery of a "sensitivity shift" in the directly affected community. In Fairfield County, the proportion of figurative death talk dropped precipitously following the tragedy. As the community became sensitized to real-world loss, everyday metaphors like "this is killing me" felt inappropriate and were largely abandoned for weeks.

The "Sensitivity Shift" in Fairfield County, showing a dramatic increase in literal death talk and a corresponding drop in figurative language.

Magnitude and Duration of Impact

The machine-learning approach revealed that standard LIWC metrics vastly underestimate the scale of community response. On the day of the shooting, the new classifier detected a 130 standard deviation (s.d.) increase in literal death talk in Fairfield County, whereas LIWC only predicted a 17 s.d. increase. Furthermore, literal death talk remained elevated in the affected community well into January, while the control group returned to baseline much faster.

Comparison of literal death talk vs. standard LIWC metrics across Fairfield and Montgomery counties.

Conclusion

The study demonstrates that while keyword-based tools like LIWC are useful for identifying general trends, they are "blunt instruments" that mask the true magnitude and nature of a community's psychological response to tragedy. By distinguishing between literal and figurative language, researchers can better detect the subtle shifts in communication that signal deep-seated community trauma. Future work may involve using these refined metrics to monitor community resilience and identify specific windows for mental health interventions.

Summary updated

BibTeX
@inproceedings{glasgow2014our,
  author = {Kimberly Glasgow and Clayton Fink and Jordan L. Boyd-Graber},
  title = {{"Our Grief is Unspeakable": Automatically Measuring the Community Impact of a Tragedy}},
  booktitle = {ICWSM},
  year = {2014},
}
Figure