TL;DR
This paper introduces a novel machine learning system designed to detect and flag abusive messages embedded within financial transaction descriptions, a growing concern for domestic violence survivors. By leveraging deep learning and a relationship-centric approach, the system aims to identify high-risk abuse cases within millions of transactions, enabling timely intervention.
What Problem Does This Paper Address?
The advent of real-time payment platforms, like Australia's New Payments Platform (NPP), has enabled longer, more descriptive transaction messages. While this enhances user experience, it has also created a new avenue for technology-assisted abuse, specifically domestic and family violence. Perpetrators are using these transaction descriptions for coercive control, intimidation, and harassment, often through low-value, repetitive messages. Unlike social media, these financial transactions have a longer "life-span" and can be more difficult to erase or block, potentially causing re-traumatization. Existing methods like keyword blocking have proven insufficient as abusers quickly find ways to circumvent them. The challenge lies in identifying these subtle yet harmful patterns within a massive volume of financial data, while also being sensitive to the potential dangers of both action and inaction.
Methods
The authors propose a multi-step approach to detect high-risk abuse at the relationship level (sender-recipient pair), rather than just individual transactions. This is crucial because abusive patterns often emerge over time and across multiple interactions.
The core methodology involves:
- Transaction-Level Feature Generation: Extracting features from individual transaction descriptions. This includes:
- Transaction Details (TRX): Basic transaction metadata like amount, date, and frequency.
- Simple Text Features (ST): Linguistic characteristics of the description text, such as length, casing, word count, and presence of special characters.
- Emotion, Toxicity, and Sentiment Features (ETS): Leveraging pre-trained NLP models (Detoxify, DistilBERT, VADER) to analyze the emotional tone, toxicity levels (obscene, threat, insult, etc.), and overall sentiment of the transaction descriptions.
- Relationship-Level Feature Generation: Aggregating the transaction-level features for all transactions within a specific sender-recipient relationship. This involves statistical aggregations like maximum, minimum, mean, and sum for various features, as well as counting unique days and transaction frequencies within a relationship.
- Incorporating Reciprocity: Adding features that capture the recipient's response (or lack thereof) to the sender's messages. The hypothesis is that victims might avoid replying to abusive senders.
- Machine Learning Model: A Random Forest classifier is trained to predict whether a relationship is "highly abusive" or "non-abusive."
Datasets
The study utilizes transaction data from the Commonwealth Bank of Australia (CBA).
- Training/Validation Data: A dataset containing transaction descriptions from July 2021 to January 2022 was used. This dataset included 1,039 relationships, of which 283 were manually labeled as "highly abusive" by domestic violence experts, achieving an 87% agreement score. Negative sampling was employed to create a balanced dataset.
- Out-of-Sample Test Data: Transaction data from February 2022 was used for an out-of-sample evaluation. To manage the imbalance and manual labeling effort, the top 50 highest-scored relationships from this month were manually reviewed.
Key Insights and Ideas
- Relationship-Level Analysis is Key: Detecting abuse solely at the transaction level is insufficient. The temporal and relational context is vital for understanding intent, especially for subtle forms of abuse like coercive control.
- Leveraging Pre-trained NLP for Financial Text: Applying advanced NLP models (like BERT-based ones for toxicity and sentiment) to the domain of financial transaction descriptions, which are often short and context-limited, proves effective.
- Reciprocity as a Signal: The absence of a recipient's reply to a sender's messages can be a significant indicator of potential abuse, suggesting a victim's reluctance or inability to engage.
- The "Human-in-the-Loop" Necessity: Given the sensitivity and complexity of abuse detection, a human-in-the-loop system is essential for manual review and intervention, with the ML model prioritizing cases for human experts.
Key Results
The paper demonstrates that a combination of transaction details, simple text features, and advanced NLP-derived emotion, toxicity, and sentiment features, along with reciprocity, yields the best performance.
- The best performing model achieved an F1 score of 0.690 and an AUC-ROC of 0.795 during cross-validation using the combined feature sets (ETS + ST + TRX).
- Adding reciprocity features further improved performance, achieving an F1 score of 0.703 and an AUC-ROC of 0.800 on the out-of-sample test set.
- The ROC curve for the top 50 cases on the out-of-sample test set shows that the best system successfully predicted highly abusive cases with the first true negative occurring at the 26th case, indicating high precision for the top-scored predictions.
Limitations and Future Work
- Data Imbalance: The dataset is highly imbalanced, with abusive cases representing a very small fraction of total transactions. While negative sampling was used, this remains a challenge.
- Domain Specificity: While pre-trained models are powerful, they might have limitations with highly domain-specific language. The authors mention retraining models on domain-specific datasets as a potential improvement.
- Foreign Language Coverage: The current system primarily focuses on English. Expanding to cover multiple languages is a stated future improvement.
- Conversation History: The paper primarily uses a one-month window for relationship analysis. Incorporating longer conversation histories could provide richer context.
- Model Robustness: The authors emphasize the importance of regularly retraining the model with verified cases to maintain its robustness and adapt to evolving abusive language patterns.
Summary updated