Adapting Coreference Resolution for Processing Violent Death Narratives

Ankith Uppunda, Susan D. Cochran, Jacob G. Foster, Alina Arseniev-Koehler, Vickie M. Mays, Kai-Wei Chang

Introduction

Coreference resolution—the task of identifying all mentions that refer to the same entity in a text—is a fundamental component of Natural Language Processing (NLP). While state-of-the-art models perform exceptionally well on standard benchmarks like news articles, they often fail when applied to specialized administrative data. This paper, presented at NAACL 2021 by researchers from UCLA, addresses these failures within the context of violent death narratives (VDN), particularly those involving marginalized populations such as the LGBT community.

The Challenge of Administrative Data

Administrative records, such as police reports or clinical notes, differ significantly from the polished prose of news media. They are often terse, jargon-heavy, and stylistically distinct. Beyond this domain gap, existing models exhibit significant gender bias. Most coreference systems rely on binary gender cues and heteronormative patterns. In narratives involving same-sex couples, these models frequently misidentify entities, as they "expect" a male-female pairing when terms like "partner" are used.

Example of coreference resolution challenges in a violent death narrative involving same-sex partners.

As shown in the example above, identifying that "he" refers to the "primary_victim's partner" rather than the "primary_victim" (who is also male) is a high-stakes task. Failure to accurately process these narratives prevents researchers and policy-makers from correctly tracking and preventing violence in vulnerable populations.

Methodology: Weakly Supervised Augmentation

To address these issues, the researchers adapted the End-to-End Coreference Resolution (E2E-Coref) system using a pipeline powered by Snorkel, a toolkit for weakly supervised data labeling. Instead of relying on expensive manual annotation of thousands of cases, they developed probabilistic data augmentation rules to "neutralize" gendered language and introduce more inclusive examples into the training data:

  • Rule 1 (R1): Gender swapping (replacing gendered terms with the opposite gender).
  • Rule 2 (R2): Relationship neutralization (e.g., changing "boyfriend" to "partner").
  • Rule 3 (R3): Descriptor neutralization.
  • Neopronoun Injection: Replacing binary pronouns with neopronouns (e.g., zie/zir) to improve performance on gender-inclusive datasets.

By training on these augmented datasets, the model learns to prioritize linguistic structure over gender stereotypes.

Experimental Setup

The study evaluated the proposed approach across four primary datasets:

  1. VDN (Violent Death Narratives): A new corpus of 115 expert-annotated cases from the CDC’s National Violent Death Reporting System.
  2. OntoNotes 5.0: The standard general-domain benchmark.
  3. GICoref: A dataset focused on non-binary individuals and LGBTQ periodicals.
  4. MAP: A Wikipedia-based dataset for testing gender bias.

Results and Discussion

The experimental results highlight the brittleness of standard models in this domain. A baseline model trained on OntoNotes achieved a dismal F1 score of 21.8 on the VDN dataset. However, targeted interventions yielded significant improvements.

Performance comparison (Precision, Recall, F1) between baseline, fine-tuned, and augmented models on the VDN dataset.

Fine-tuning the model on a small set of in-domain narratives (E2E-FT) increased the F1 score to 61.4. The addition of Snorkel-based augmentation (E2E-Aug) further boosted performance to 62.8. Notably, on narratives specifically involving LGBT individuals, the augmentation strategy provided a 5% F1 improvement over standard fine-tuning. Furthermore, on the GICoref dataset, the performance gap between binary pronouns and neopronouns was reduced from 12% to just 3%.

Conclusion

This research demonstrates that small amounts of in-domain data, combined with intelligent data augmentation, can significantly improve coreference resolution for marginalized populations. Achieving algorithmic equity is a crucial step toward accurately identifying risks and informing public health policies for vulnerable groups. Future work will focus on expanding these rules to encompass a broader spectrum of social and family structures found in administrative records.

Summary updated

BibTeX
@inproceedings{uppunda2021adapting,
  author = {Ankith Uppunda and Susan D. Cochran and Jacob G. Foster and Alina Arseniev-Koehler and Vickie M. Mays and Kai-Wei Chang},
  title = {{Adapting Coreference Resolution for Processing Violent Death Narratives}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-main.361},
}
Figure