Abstract
This paper introduces the Grievance Dictionary, a psycholinguistic dictionary which can be used to automatically understand language use in the context of grievance-fuelled violence threat assessment. We describe the development of the dictionary, which was informed by suggestions from experienced threat assessment practitioners. These suggestions and subsequent human and computational word list generation resulted in a dictionary of 20,502 words annotated by 2,318 participants. The dictionary was validated by applying it to texts written by violent and non-violent individuals, showing strong evidence for a difference between populations in several dictionary categories. Further classification tasks showed promising performance, but future improvements are still needed. Finally, we provide instructions and suggestions for the use of the Grievance Dictionary by security professionals and (violence) researchers.
Introduction
In the field of security and threat assessment, practitioners are increasingly tasked with analyzing vast quantities of text—ranging from manifestos to social media posts—to gauge the risk of violence. Traditionally, analysis has relied on general psycholinguistic tools like LIWC (Linguistic Inquiry and Word Count), which measure broad emotional categories but lack the domain-specific granularity required for threat detection. Alternatively, "expert dictionaries" built for specific domains often operate as "black boxes" with opaque development processes. This research addresses the need for a transparent, domain-specific, and validated tool that aligns with modern policing requirements for explainable algorithms.
Methodology and Development
The Grievance Dictionary was developed through a rigorous five-phase process designed to ensure both expert relevance and linguistic breadth:
- Expert Survey: 21 threat assessment professionals identified 22 key psychological and social categories relevant to grievance-fuelled violence.
- Seed Word Generation: Initial word lists were generated for these categories by subject matter experts.
- Computational Expansion: Lists were expanded using WordNet for synonyms and GloVe word embeddings to identify the nearest neighbors in vector space.
- Crowdsourced Validation: 2,318 human annotators rated 24,322 word-category pairs, resulting in a refined final list of 20,502 words.
- Psychometric Evaluation: Internal consistency was calculated across diverse corpora to ensure reliability.
Psychometric Reliability
To ensure the dictionary's categories were robust, the authors calculated Cronbach’s alpha across four diverse corpora, including IMDB reviews, novels, and Reddit posts. The results showed internal consistency comparable to industry-standard tools, providing a foundation for the dictionary's use in varied linguistic contexts.
Key Insights and Validation
The dictionary was validated against several datasets, including lone-actor terrorist manifestos, extremist forum posts (Stormfront), and neutral control groups. Several key findings emerged:
- Granularity of Signal: Unlike general tools that flag broad "negative emotion," the Grievance Dictionary distinguishes between specific states such as frustration, paranoia, and jealousy, allowing for a more nuanced psychological profile.
- Predictive Categories: In classification tasks, the "Soldier" and "Weaponry" categories emerged as the strongest predictors of violent intent.
- Manifestos vs. Extremism: The tool successfully differentiated between actual perpetrators (manifestos) and "keyboard warriors" on extremist forums. Perpetrators utilized significantly more language related to planning, weaponry, and soldiering.
- Performance Metrics: While the tool achieved high accuracy (97%) in distinguishing manifestos from neutral blogs, precision remained low (0.17). This indicates that while the tool is effective at identifying potential threats, it also flags a significant amount of non-threatening material, necessitating human-in-the-loop analysis.
Limitations and Future Directions
While the Grievance Dictionary represents a significant step forward in transparent threat assessment, the authors note several limitations. It remains difficult to distinguish between a propensity for violence and the mere topic of violence (e.g., a fictional story about a soldier). Furthermore, the "bluffer problem"—individuals who use violent language without intent to act—remains a challenge for automated systems. Future iterations may benefit from data-driven seeding using authentic jargon and slang extracted directly from known violent texts to further improve precision.
Summary updated