Automatic Extraction of Mental Health Disorders From Domestic Violence Police Narratives: Text Mining Study

George Karystianis, Armita Adily, Peter W. Schofield, Lee G. Knight, Clara Galdon, David M. Greenberg, Louisa Jorm, Goran Nenadić, Tony ButlerJournal of Medical Internet Research 2018 · DOI

TL;DR

This paper introduces a novel text mining approach to automatically identify mental health disorder mentions within domestic violence (DV) police narratives. By leveraging a knowledge-driven system with custom dictionaries and rules, the study demonstrates high accuracy in extracting this crucial, previously untapped information, paving the way for better public health research into the complex relationship between DV and mental health.

What Problem Does This Paper Address?

Domestic violence (DV) is a significant public health issue with a complex, bidirectional relationship with mental health disorders. Police forces worldwide attend a vast number of DV incidents annually, recording details in databases. While structured data is used for reporting, the unstructured text narratives, which often contain rich contextual information, remain largely unanalyzed. Manually reviewing these narratives to identify mental health mentions is impractical due to their sheer volume and the nuanced language used. This gap prevents a deeper understanding of how mental health issues intersect with DV, hindering effective policy formulation and intervention strategies.

Methods

The authors developed a knowledge-driven text mining approach to automatically extract mentions of mental health disorders from police narratives. This system involved several key steps:

  • Dictionary Creation: A set of 13 custom dictionaries were manually crafted. These dictionaries contained terms related to mental health disorders (including ICD-10 codes, common misspellings, and unofficial terms), negations, and terms identifying the "person of interest" (POI) and "victim."
  • Rule-Based Extraction: Using the General Architecture for Text Engineering (GATE) framework and its Java Annotations Pattern Engine (JAPE), the researchers designed 264 rules. These rules combined lexical patterns (e.g., "defendant suffers from") with dictionary lookups to identify mentions of mental health disorders associated with either the POI or the victim. The rules were designed for high precision to minimize false positives.
  • ICD-10 Mapping: Extracted mentions were mapped to the International Classification of Diseases, Tenth Revision (ICD-10) schema. This involved a multi-level mapping process, including the creation of eight custom categories to accommodate terms not directly covered by the standard ICD-10.
  • Narrative-Level Unification: Duplicate mentions within a single narrative were eliminated to ensure each event was represented at a narrative level.

Datasets

  • Training and Development Sets: 100 randomly selected DV events with identified mental health disorder mentions were used for training, and another 100 for development to optimize the system.
  • Evaluation Set: 100 randomly chosen DV events were used for evaluation.
  • Large-Scale Corpus: The developed system was applied to a large dataset of 492,393 DV events recorded by the New South Wales Police Force (NSWPF) between January 2005 and December 2016.

Key Insights and Ideas

  • Untapped Information in Police Narratives: The study highlights that unstructured police narratives contain a wealth of information about mental health in DV contexts that is currently underutilized.
  • Feasibility of Automated Extraction: The research demonstrates that text mining can effectively and accurately extract specific mental health disorder mentions from these complex, often informal, police reports.
  • High Precision is Crucial: Given the sensitive nature of the data and the goal of informing policy, the emphasis on high precision in the rule design is a key methodological choice, even if it means sacrificing some recall.
  • POI vs. Victim Differences: The study reveals differences in the types and prevalence of mental health disorders mentioned for POIs versus victims, suggesting distinct patterns in their involvement with DV.

Key Results

  • High Performance: The system achieved high performance metrics on the evaluation set, with an F-score of 86.9% for POIs and 80.6% for victims. Precision was particularly high, reaching 97.5% for POIs and 87.1% for victims.
  • Large-Scale Identification: Across nearly half a million DV events, the system identified 77,995 events (15.83%) mentioning mental health disorders.
  • Prevalence: Mental health disorders were more frequently mentioned in relation to POIs (76.96% of identified events) than victims (16.47%).
  • Common Disorders: Depression was the most common mental health disorder mentioned for both victims and POIs. Alcohol abuse was also highly prevalent among POIs, while anxiety disorders were common among victims.
  • Mapping to ICD-10: The mapping to ICD-10 categories provided a standardized way to analyze the extracted information, revealing patterns like "unspecified mental disorder" being common for both POIs and victims.

Limitations and Future Work

  • Training Data Size: The rules were developed based on a relatively small training and development set. The authors suggest that a dataset focused solely on victim mentions might improve performance for this group.
  • Informal Language: The police narratives contain informal language, misspellings, and abbreviations, which can be challenging for text mining.
  • Veracity of Mentions: The extracted mentions are based on police officer observations and victim/POI statements, not formal diagnoses. Future work aims to validate these informal mentions against formal diagnoses from administrative data.
  • Expanding Scope: The authors plan to expand the targeted information to include other characteristics of POIs and victims (e.g., elderly, same-sex relationships, carers) to inform predictive models for DV risk and prevention strategies. They also suggest adapting the rules for other types of police-recorded narratives.

Summary updated

Abstract

The results suggest that text mining can automatically extract targeted information from police-recorded DV events to support further public health research into the nexus between mental health disorders and DV.
BibTeX
@article{karystianis2018automatic,
  author = {George Karystianis and Armita Adily and Peter W. Schofield and Lee G. Knight and Clara Galdon and David M. Greenberg and Louisa Jorm and Goran Nenadić and Tony Butler},
  title = {{Automatic Extraction of Mental Health Disorders From Domestic Violence Police Narratives: Text Mining Study}},
  journal = {Journal of Medical Internet Research},
  year = {2018},
  doi = {10.2196/11548},
}
Figure