The Language of Trauma: Modeling Traumatic Event Descriptions Across Domains with Explainable AI

Miriam Schirmer, Tobias Leemann, Gjergji Kasneci, Jürgen Pfeffer, David Jurgens

Introduction

Psychological trauma remains a critical public health challenge, yet NLP research in this area is often confined to specific, isolated domains—such as military veterans or specific social media platforms. This fragmentation leads to a "transferability gap," where models trained in one context fail to generalize to others. Furthermore, distinguishing concrete traumatic events (defined by the APA as exposure to death, serious injury, or sexual violence) from general distress or aggression is a complex task for automated systems. This research introduces TRACE (Trauma Event Recognition Across Contextual Environments), a cross-domain dataset designed to build models that are both accurate across contexts and interpretable for clinical use.

Methodology

The researchers framed trauma detection as a binary classification task: determining whether a given text contains a traumatic event based on clinical definitions. The study compared traditional baselines (Naive Bayes, Logistic Regression), deep learning models (TF-IDF FCNNs), and state-of-the-art transformers (BERT, RoBERTa), alongside Large Language Models like GPT-4.

Overview of the TRACE framework, from data sources to XAI-driven characteristic identification.

The TRACE dataset aggregates diverse sources:

  • Genocide Transcript Corpus (GTC): Legal witness statements from international tribunals (Rwanda, Yugoslavia, Cambodia).
  • PTSD Subreddit: Personal narratives regarding post-traumatic stress.
  • Counseling Dataset: User queries for mental health advice.
  • Incel Posts: A control group of aggressive, non-traumatic content to test the model's ability to distinguish trauma from general toxicity.

To ensure model transparency, a three-stage Explainable AI (XAI) framework was employed, utilizing SHAP for token contribution, SLALOM for distinguishing token value from importance, and concept-based explanations to identify high-level semantic themes like "physical injury" or "flashbacks."

Results and Generalizability

The fine-tuned RoBERTa model emerged as the top performer, achieving F1-scores of 0.74 on Genocide data and 0.71 on PTSD data, slightly outperforming GPT-4. A key finding was the high degree of generalizability across domains.

Heatmap showing the cross-domain performance (AUC-ROC) of models trained and tested on different datasets within TRACE.

As shown in the performance heatmap, models trained on the high-density trauma of Genocide transcripts (GTC) transferred remarkably well to Reddit (PTSD) data, achieving an AUC-ROC of 0.885. This suggests that extreme trauma provides a clear, learnable signal that transcends specific linguistic contexts. Additionally, the "Incel Test" confirmed that models could successfully differentiate between aggressive, hateful language and actual traumatic events, rather than simply flagging negative sentiment.

Explainability and Clinical Insights

XAI analysis revealed a "universal" language of trauma. Themes of death and sexual violence were the most consistent indicators across all domains. The models also successfully "rediscovered" clinical symptoms; for instance, in the PTSD dataset, the model prioritized terms like "flashbacks" and "nightmares," which are core diagnostic criteria.

However, the study highlighted challenges in classifying mental and emotional abuse. Because these forms of trauma are often less concrete than physical violence, they are more difficult for both models and human annotators to categorize under strict APA definitions.

Conclusion and Future Work

While the TRACE framework demonstrates strong cross-domain performance, limitations remain. Real-world counseling data often presents a "needle in a haystack" problem where traumatic events are rare compared to the high-density trauma found in legal transcripts. Future research will focus on span detection to identify specific traumatic phrases within larger texts, providing more granular support for clinicians. By bridging clinical psychology and machine learning, this work moves toward automated tools that can effectively flag trauma in online communities and facilitate timely mental health interventions.

Summary updated

Abstract

Psychological trauma can manifest following various distressing events and is captured in diverse online contexts. However, studies traditionally focus on a single aspect of trauma, often neglecting the transferability of findings across different scenarios. We address this gap by training various language models with progressing complexity on trauma-related datasets, including genocide-related court data, a Reddit dataset on post-traumatic stress disorder (PTSD), counseling conversations, and Incel forum posts. Our results show that the fine-tuned RoBERTa model excels in predicting traumatic events across domains, slightly outperforming large language models like GPT-4. Additionally, SLALOM-feature scores and conceptual explanations effectively differentiate and cluster trauma-related language, highlighting different trauma aspects and identifying sexual abuse and experiences related to death as a common traumatic event across all datasets. This transferability is crucial as it allows for the development of tools to enhance trauma detection and intervention in diverse populations and settings.
BibTeX
@inproceedings{schirmer2024language,
  author = {Miriam Schirmer and Tobias Leemann and Gjergji Kasneci and Jürgen Pfeffer and David Jurgens},
  title = {{The Language of Trauma: Modeling Traumatic Event Descriptions Across Domains with Explainable AI}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.773},
}
Figure