SAFE: A Neural Survival Analysis Model for Fraud Early Detection.

Panpan Zheng, Shuhan Yuan, Xintao Wu

TL;DR

This paper introduces SAFE, a novel model that combines Recurrent Neural Networks (RNNs) with survival analysis to detect fraudulent users on online platforms before they are suspended. By adapting survival analysis techniques, SAFE provides consistent, timely fraud predictions, outperforming traditional classification and existing fraud detection methods.

What Problem Does This Paper Address?

Online platforms rely on anti-fraud systems to identify and block malicious users. However, a significant challenge is the "detection gap": the time lag between when a user actually commits a fraudulent act and when the platform detects and suspends them. Existing fraud detection methods often use classifiers that predict fraud at each timestamp independently, leading to inconsistent and sometimes delayed detection. This is particularly problematic because training data typically only records the suspension time, not the exact moment of fraudulent activity, making it difficult to train models for truly early detection. The paper aims to bridge this gap by developing a model that can predict fraudulent behavior earlier and more reliably, even with delayed feedback.

Methods

The core of SAFE is its innovative combination of two key components:

  • Recurrent Neural Networks (RNNs): SAFE uses an RNN (specifically, a Gated Recurrent Unit or GRU) to process the dynamic, time-varying sequences of user activities. The RNN learns to capture complex patterns and long-term dependencies within these activity streams.
  • Survival Analysis: Instead of directly classifying users as fraudulent or not, SAFE models the "time until an event of interest occurs." In this context, the "event" is the user being suspended. Survival analysis naturally models the probability of an event happening over time, providing a more consistent prediction framework than independent classifiers.

The novelty lies in how these are integrated:

  1. Hazard Rate Prediction: The RNN outputs a hidden state at each timestamp, which is then used to predict the hazard rate (the instantaneous risk of being suspended).
  2. Survival Probability: From the predicted hazard rates, SAFE calculates the survival probability – the likelihood that a user has not yet been suspended. This probability is guaranteed to be monotonically decreasing over time, ensuring consistent predictions.
  3. Early Detection Loss Function: A crucial innovation is the modification of the standard survival analysis loss function. The paper introduces an "early-detection-oriented" loss that encourages the model to predict higher hazard rates (and thus lower survival probabilities) for actual fraudsters before their recorded suspension time, while allowing non-fraudulent users to "survive" longer. This directly addresses the challenge of delayed labels in training data.

Datasets

The paper evaluates SAFE on two real-world datasets:

  • Twitter: Data collected from over 51,000 Twitter users over 21 timestamps, featuring features like follower counts, tweet counts, and liked tweets. This dataset contains both suspended (fraudulent) and active (right-censored) users.
  • Wiki: The UMDWikipedia dataset, used for early vandal detection. It includes editing sequences and features related to edit behavior. This dataset also contains both vandals and benign users.

Key Insights and Ideas

  • Survival Analysis for Fraud is a Game Changer: Moving beyond simple classification, survival analysis provides a principled way to model the timing of events, leading to more consistent and interpretable predictions.
  • The "Detection Gap" is Solvable: The core challenge of delayed labels in fraud detection can be addressed by carefully designing the model's objective function (the loss function) to prioritize early detection.
  • RNNs are Essential for Dynamic Data: User activity is inherently sequential and time-varying. RNNs are well-suited to capture these dynamics, providing the rich input needed for survival analysis.
  • No Assumptions Needed: SAFE's ability to learn hazard rates directly from RNN outputs, without assuming specific parametric distributions (like Weibull or Poisson), makes it more robust and applicable to real-world fraud scenarios where underlying distributions are unknown.

Key Results

  • Superior Performance: SAFE consistently outperformed baseline methods (SVM, CPH, M-LSTM) across both datasets in terms of Precision, Recall, F1-score, and Accuracy, especially when considering early detection metrics.
  • Significant Early Detection: On the Twitter dataset, SAFE achieved an 82% early detection rate for fraudsters, compared to only 24% for M-LSTM. The average early-detected timestamp for SAFE was also notably earlier.
  • Balanced Trade-offs: While some baselines achieved high recall, they sacrificed precision (leading to many false positives). SAFE demonstrated a better balance, achieving high precision and recall simultaneously.
  • Model Robustness: SAFE's performance was superior even when compared to other neural survival models that did assume specific distributions (e.g., Exponential-RNN, Weibull-RNN), highlighting the advantage of its distribution-agnostic approach.

Limitations and Future Work

  • Limitations: The paper acknowledges that while SAFE aims for early detection, it still relies on the suspension time as the ground truth for training. The exact moment of fraudulent activity remains unobserved.
  • Future Work: The authors plan to extend SAFE to predict when fraudulent activities are taken, even when only the suspended time is observed. This would further refine the early detection capabilities by trying to infer the actual time of the offense.

Summary updated

Abstract

Many online platforms have deployed anti-fraud systems to detect and prevent fraudulent activities. However, there is usually a gap between the time that a user commits a fraudulent action and the time that the user is suspended by the platform. How to detect fraudsters in time is a challenging problem. Most of the existing approaches adopt classifiers to predict fraudsters given their activity sequences along time. The main drawback of classification models is that the prediction results between consecutive timestamps are often inconsistent. In this paper, we propose a survival analysis based fraud early detection model, SAFE, which maps dynamic user activities to survival probabilities that are guaranteed to be monotonically decreasing along time. SAFE adopts recurrent neural network (RNN) to handle user activity sequences and directly outputs hazard values at each timestamp, and then, survival probability derived from hazard values is deployed to achieve consistent predictions. Because we only observe the user suspended time instead of the fraudulent activity time in the training data, we revise the loss function of the regular survival model to achieve fraud early detection. Experimental results on two real world datasets demonstrate that SAFE outperforms both the survival analysis model and recurrent neural network model alone as well as state-of-the-art fraud early detection approaches.
BibTeX
@inproceedings{zheng2019safe,
  author = {Panpan Zheng and Shuhan Yuan and Xintao Wu},
  title = {{SAFE: A Neural Survival Analysis Model for Fraud Early Detection.}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  url = {https://openreview.net/forum?id=ofJgUoJIrcI},
}
Figure