Suicidal Risk Detection for Military Personnel

Sungjoon Park, Ki-Woong Park, Jaimeen Ahn, Alice OhEMNLP 2020 · OpenReview

Abstract

Sungjoon Park, Kiwoong Park, Jaimeen Ahn, Alice Oh. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).

2020.

Introduction

Suicide is a leading cause of death in the military, particularly in countries with compulsory service like the Republic of Korea. Soldiers often face unique stressors—isolation from support networks, harsh unit climates, and combat exposure—that differ significantly from the general public.

Existing suicide detection research typically focuses on general populations, but these models do not always translate to the military context. Furthermore, collecting data from this demographic is difficult due to privacy concerns and the "closed" nature of military life. The authors sought to bridge this gap by creating a high-quality, ethically sourced dataset of military-related social media posts and testing whether state-of-the-art Natural Language Processing (NLP) could accurately flag soldiers in distress.

Methods and Dataset

The researchers followed a multi-stage pipeline to develop a specialized detection framework. They scraped 2,791 posts from Naver Knowledge iN (a popular Korean Q&A platform) using 58 military-related keywords. To protect privacy, they manually stripped all personally identifiable information (PII), including unit names and user IDs.

Dual-Perspective Annotation

A novel methodological step in this study was the recruitment of two distinct expert groups for annotation:

  • External Experts: Psychiatrists and psychotherapists.
  • Internal Experts: Military counselors and commanders.

Each post was rated on a scale of 0 (No Risk) to 3 (Imminent Risk). This dual approach allowed the researchers to capture both clinical indicators and domain-specific stressors unique to military life.

The four-level risk scale and descriptions used by annotators to categorize social media posts.

Modeling

The team fine-tuned several transformer-based models, including Multilingual BERT, XLM-R, and KoBERT (a BERT model pre-trained specifically on Korean text). They compared these against a standard Convolutional Neural Network (CNN) baseline.

Key Results and Insights

The transformer-based models significantly outperformed the CNN baseline across all classification tasks. KoBERT, which was trained on Korean-specific corpora, consistently provided the best results, highlighting the importance of linguistic nuance in mental health detection.

Performance metrics (F1 score and Accuracy) for various models across different annotation sets and risk categories.
  • Binary Classification (Flagged vs. Not Flagged): KoBERT achieved an F1 score of 0.88 and an accuracy of 91%.
  • Urgent Risk Detection: For identifying the most critical cases, the models reached an F1 score of 0.80.
  • The "Perspective Gap": A fascinating finding was the disagreement between internal and external experts. Military commanders (internal) were often more sensitive to "low-risk" maladaptation to military life, whereas clinicians (external) were better at identifying "imminent" clinical risks. Models trained on "Internal Expert" labels generally yielded higher F1 scores, suggesting their criteria might be more closely tied to the linguistic markers found in the posts.

Discussion and Future Work

While the study demonstrates the effectiveness of NLP in this domain, it is limited by the dataset size and its specific focus on the South Korean military context. The researchers suggest that future work should incorporate military-specific slang and integrate multi-language data to improve robustness. Crucially, the authors emphasize that such tools should be used as support systems for human intervention programs, rather than standalone diagnostic tools.

Summary updated

BibTeX
@inproceedings{park2020suicidal,
  author = {Sungjoon Park and Ki-Woong Park and Jaimeen Ahn and Alice Oh},
  title = {{Suicidal Risk Detection for Military Personnel}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=YweuuhMyyoW},
}
Figure