TL;DR
This paper introduces a novel Multi-task Learning (MTL) framework to improve the detection of both physical and, crucially, precursory emotional forms of intimate partner abuse in text. By training a single model on related tasks simultaneously, the authors demonstrate enhanced accuracy in identifying subtle emotional abuse, paving the way for earlier intervention.
What Problem Does This Paper Address?
Intimate Partner Violence (IPV) is a pervasive issue with devastating consequences. While physical and sexual abuse are often the most visible forms, they are frequently preceded by psychological and emotional abuse, such as belittling or manipulation. Identifying these early warning signs is critical for prevention, but detecting subtle emotional abuse in text is challenging. Existing Natural Language Processing (NLP) approaches have primarily focused on detecting overt forms of violence or abuse, often relying on explicit self-reports. This paper aims to bridge this gap by developing a robust model capable of recognizing a broader spectrum of abusive behaviors, including those that might not yet be recognized as abuse by the user.
Methods
The core of this work is a Multi-task Learning (MTL) framework. MTL involves training a single model on multiple related tasks simultaneously, allowing it to learn shared representations and potentially improve performance on each individual task.
The proposed architecture utilizes a pre-trained DeBERTaV3 large language model (LLM) as a base. This LLM is fine-tuned on two distinct tasks:
- Task 1: Emotional Behavior Classification (Multi-label): This task aims to identify specific types of emotional abuse, such as hostility, antagonism, dismissiveness, condescension, and sarcasm.
- Task 2: Relationship Abuse Detection (Binary): This task classifies text as either an instance of relationship abuse or not.
The LLM's embeddings are then passed through a shared feature extractor (a dropout layer followed by a dense layer). From this shared representation, the model branches into separate task-specific heads for each of the two tasks. The entire network is trained by optimizing a weighted sum of the losses from both tasks, controlled by a hyperparameter $\tau$.
The authors also explored different weighting strategies for the two tasks ($\tau=0.4$ and $\tau=0.6$) and a version trained on a higher-quality subset of the emotional abuse dataset (MTL Confident). For comparison, they also trained two single-task learning (STL) models.
Datasets
- Unhealthy Comments Corpus (UCC): Used for Task 1 (emotional behavior classification). This dataset contains over 44,000 online comments annotated by crowdworkers, labeled with categories like hostile, antagonizing, dismissive, condescending, and sarcastic. A subset of this data, with higher confidence scores from crowdworkers, was used for the "MTL Confident" model.
- Reddit Abuse Dataset: Used for Task 2 (relationship abuse detection). This dataset comprises 1,336 anonymous submissions from Reddit detailing various types of relationship abuse.
Key Insights and Ideas
- MTL for Subtle Abuse Detection: The study highlights that MTL can significantly improve the detection of more nuanced and precursory forms of abuse (Task 1) compared to single-task approaches. This is crucial because these subtle behaviors often precede overt violence.
- Task Weighting Matters: Different weightings of the tasks in the MTL framework can lead to varying performance on specific abuse types. This suggests that tailoring the MTL objective can optimize for particular detection goals.
- Quality Over Quantity (Sometimes): Using a higher-quality subset of the emotional abuse data (MTL Confident) led to improved performance, underscoring the importance of reliable annotations for challenging tasks.
- Emotional Abuse is a Strong Precursor: The success in detecting emotional abuse, even when weighted less heavily, suggests its strong predictive power for overall relationship abuse.
Key Results
- Task 1 Performance: Task 1 (emotional behavior classification) was generally harder for all models to learn, but MTL approaches showed promise.
- Task 2 Performance: All models achieved high testing accuracy (at least 95%) on Task 2 (general relationship abuse detection).
- MTL vs. STL: The MTL models, particularly MTL $\tau=0.6$, demonstrated superior performance in correctly classifying subtle emotional abuse examples (manipulation and belittling) compared to STL models. For instance, the MTL $\tau=0.6$ model classified manipulation with a higher probability (0.9307) than the STL model (0.8224).
- Specific Abuse Type Detection: The MTL models were uniquely capable of identifying belittling as abuse, a task that single-task models struggled with.
Limitations and Future Work
- Gender Bias: The authors acknowledge a significant gender bias in their models. When the gender of the abuser was changed in an example, the probability of detecting abuse changed significantly. This is attributed to the higher prevalence of reported abuse by women in general datasets. Future research needs to address this bias.
- Dataset Requirements: The authors suggest that future work would benefit from an expertly labeled emotional relationship abuse dataset to enable more in-depth analysis and model refinement. This would allow for a more granular understanding of different forms of emotional abuse.
Summary updated