Distinguishing Address vs. Reference Mentions of Personal Names in Text

Vinodkumar Prabhakaran, Aida Mostafazadeh Davani, Melissa Ferguson, Stav AtirACL 2023 · DOI

Abstract

Standard Named Entity Recognition (NER) identifies the presence of individuals in text but typically ignores the pragmatic context of the mention. This research introduces a method to distinguish between addressing a person (vocative use, e.g., "Alice, what do you think?") and referring to them (third-person use, e.g., "Alice thought about it"). By employing a "slot-filling" technique with pre-trained language models like BERT, the authors achieve 85% accuracy in this classification task. Analysis reveals that address mentions are strongly correlated with online toxicity and exhibit significant gendered patterns across social media and public discourse datasets.

Introduction

In many inflected languages, the vocative case explicitly marks when a person is being addressed directly. English lacks such formal markers, making the distinction between address and reference dependent on syntactic and pragmatic context. While seemingly subtle, this distinction is vital for understanding social dynamics. For instance, the difference between "Let's eat, Grandma" and "Let's eat Grandma" hinges entirely on whether "Grandma" is an addressee or an object of reference.

Current NLP pipelines treat all name mentions identically, missing an opportunity to capture how users interact with one another. This paper addresses this gap by developing a tagger that identifies the "how" of a mention, providing a new lens through which to study online abuse and social bias.

Methodology: The Slot-Filling Approach

Rather than training a complex new architecture from scratch, the researchers utilized the latent linguistic knowledge within pre-trained models like BERT and DistilBERT. They proposed a "slot-filling" heuristic:

  1. Masking: The personal name in a sentence is replaced with a [MASK] token.
  2. Substitution Likelihood: The model predicts the most likely word to fill the mask. The researchers compared the likelihood of "address-style" tokens (second-person pronouns like you or greetings like hey) against "reference-style" tokens (third-person pronouns like he/she).
  3. Classification: If the model assigns a higher probability to address-style cues, the mention is tagged as an Address; otherwise, it is a Reference.

This zero-shot or few-shot approach leverages the syntactic understanding already present in Large Language Models (LLMs), bypassing the need for massive labeled datasets.

Datasets and Evaluation

The study utilized several key datasets to validate the approach and analyze social trends:

  • Annotated Facebook Corpus: 774 manually annotated English mentions from Facebook comments on political posts.
  • RtGender: A large-scale dataset of social media responses to politicians and public figures.
  • Jigsaw Toxicity Dataset: Used to evaluate the relationship between address mentions and online abuse.

The best-performing model (BERT with "hey/hi" cues) achieved 85% accuracy. Notably, human annotators reached a high inter-rater agreement ($\kappa = 0.898$), confirming that the distinction is linguistically grounded and clear to native speakers.

Social Bias and Gendered Addressing

The researchers found significant disparities in how different groups are addressed versus referenced. Across Facebook and TED Talk datasets, women are addressed directly more frequently than men, who are more often spoken about in the third person.

Addressed/Referenced ratios across FB Wiki, FB Congress, and Ted Talks datasets show that women are addressed more frequently than men across different platforms.

Furthermore, the study noted a "Political Interaction Bias." The gender disparity in address frequency was most pronounced in comments directed at Republican politicians, suggesting that political identity intersects with gender to shape public interaction styles.

A critical finding of the research is the relationship between direct address and online abuse. Using the Jigsaw dataset, the authors demonstrated that sentences addressing a person directly are significantly more likely to be toxic than those referring to them.

The ratio of address-to-reference mentions is notably higher in toxic comments compared to non-toxic ones, highlighting the aggressive nature of direct address in online discourse.

The statistical significance ($p < .001$) of these findings suggests that "addressing" is a pragmatic marker for targeted toxicity, with address mentions yielding higher toxicity scores (0.088) compared to reference mentions (0.070).

Conclusion and Future Work

This work demonstrates that the distinction between address and reference is both computationally tractable and socially informative. While the current study is limited by its focus on US-centric social media and binary gender categories, it provides a robust framework for future research. Potential applications include improving addressee detection in multi-party chats and refining sentiment analysis by distinguishing between opinions about a person and comments directed at them.

Summary updated

BibTeX
@inproceedings{prabhakaran2023distinguishing,
  author = {Vinodkumar Prabhakaran and Aida Mostafazadeh Davani and Melissa Ferguson and Stav Atir},
  title = {{Distinguishing Address vs. Reference Mentions of Personal Names in Text}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.findings-acl.425},
}
Figure