TL;DR
This paper introduces SAFE, a novel model that combines Recurrent Neural Networks (RNNs) with survival analysis to detect fraudulent users on online platforms before they are suspended. By adapting survival analysis techniques, SAFE provides consistent, timely fraud predictions, outperforming traditional classification and existing fraud detection methods.
What Problem Does This Paper Address?
Online platforms rely on anti-fraud systems to identify and block malicious users. However, a significant challenge is the "detection gap": the time lag between when a user actually commits a fraudulent act and when the platform detects and suspends them. Existing fraud detection methods often use classifiers that predict fraud at each timestamp independently, leading to inconsistent and sometimes delayed detection. This is particularly problematic because training data typically only records the suspension time, not the exact moment of fraudulent activity, making it difficult to train models for truly early detection. The paper aims to bridge this gap by developing a model that can predict fraudulent behavior earlier and more reliably, even with delayed feedback.
Methods
The core of SAFE is its innovative combination of two key components:
- Recurrent Neural Networks (RNNs): SAFE uses an RNN (specifically, a Gated Recurrent Unit or GRU) to process the dynamic, time-varying sequences of user activities. The RNN learns to capture complex patterns and long-term dependencies within these activity streams.
- Survival Analysis: Instead of directly classifying users as fraudulent or not, SAFE models the "time until an event of interest occurs." In this context, the "event" is the user being suspended. Survival analysis naturally models the probability of an event happening over time, providing a more consistent prediction framework than independent classifiers.
The novelty lies in how these are integrated:
- Hazard Rate Prediction: The RNN outputs a hidden state at each timestamp, which is then used to predict the hazard rate (the instantaneous risk of being suspended).
- Survival Probability: From the predicted hazard rates, SAFE calculates the survival probability – the likelihood that a user has not yet been suspended. This probability is guaranteed to be monotonically decreasing over time, ensuring consistent predictions.
- Early Detection Loss Function: A crucial innovation is the modification of the standard survival analysis loss function. The paper introduces an "early-detection-oriented" loss that encourages the model to predict higher hazard rates (and thus lower survival probabilities) for actual fraudsters before their recorded suspension time, while allowing non-fraudulent users to "survive" longer. This directly addresses the challenge of delayed labels in training data.
Datasets
The paper evaluates SAFE on two real-world datasets:
- Twitter: Data collected from over 51,000 Twitter users over 21 timestamps, featuring features like follower counts, tweet counts, and liked tweets. This dataset contains both suspended (fraudulent) and active (right-censored) users.
- Wiki: The UMDWikipedia dataset, used for early vandal detection. It includes editing sequences and features related to edit behavior. This dataset also contains both vandals and benign users.
Key Insights and Ideas
- Survival Analysis for Fraud is a Game Changer: Moving beyond simple classification, survival analysis provides a principled way to model the timing of events, leading to more consistent and interpretable predictions.
- The "Detection Gap" is Solvable: The core challenge of delayed labels in fraud detection can be addressed by carefully designing the model's objective function (the loss function) to prioritize early detection.
- RNNs are Essential for Dynamic Data: User activity is inherently sequential and time-varying. RNNs are well-suited to capture these dynamics, providing the rich input needed for survival analysis.
- No Assumptions Needed: SAFE's ability to learn hazard rates directly from RNN outputs, without assuming specific parametric distributions (like Weibull or Poisson), makes it more robust and applicable to real-world fraud scenarios where underlying distributions are unknown.
Key Results
- Superior Performance: SAFE consistently outperformed baseline methods (SVM, CPH, M-LSTM) across both datasets in terms of Precision, Recall, F1-score, and Accuracy, especially when considering early detection metrics.
- Significant Early Detection: On the Twitter dataset, SAFE achieved an 82% early detection rate for fraudsters, compared to only 24% for M-LSTM. The average early-detected timestamp for SAFE was also notably earlier.
- Balanced Trade-offs: While some baselines achieved high recall, they sacrificed precision (leading to many false positives). SAFE demonstrated a better balance, achieving high precision and recall simultaneously.
- Model Robustness: SAFE's performance was superior even when compared to other neural survival models that did assume specific distributions (e.g., Exponential-RNN, Weibull-RNN), highlighting the advantage of its distribution-agnostic approach.
Limitations and Future Work
- Limitations: The paper acknowledges that while SAFE aims for early detection, it still relies on the suspension time as the ground truth for training. The exact moment of fraudulent activity remains unobserved.
- Future Work: The authors plan to extend SAFE to predict when fraudulent activities are taken, even when only the suspended time is observed. This would further refine the early detection capabilities by trying to infer the actual time of the offense.
Summary updated