Interleaved Sequence RNNs for Fraud Detection

Bernardo Branco, Pedro Abreu, Ana Sofia Gomes, Mariana S. C. Almeida, João Tiago Ascensão, Pedro BizarroKDD 2020 · DOI

TL;DR

This paper introduces a novel Recurrent Neural Network (RNN) framework for real-time credit card fraud detection that treats transaction histories as interleaved, unbounded sequences. By eliminating the need for expensive manual feature engineering (profiles), their approach achieves state-of-the-art performance and significantly reduces computational resources, saving millions in fraud detection.

What Problem Does This Paper Address?

Credit card fraud is a massive global problem, costing billions annually. Traditional fraud detection systems often rely on complex rule-based models or machine learning models that require extensive "feature engineering." This involves manually creating features that capture user behavior patterns (e.g., transaction frequency, average spending). Building and maintaining these features is time-consuming, computationally expensive, and creates bottlenecks in production systems due to real-time data requirements. While deep learning, particularly RNNs, shows promise in learning representations directly from data, applying them to fraud detection is challenging because transaction histories are not simple linear sequences. Instead, they are composed of multiple, interleaved, and irregular sub-sequences (e.g., transactions from different cards or payment methods for the same user). This complexity makes it difficult to manage the RNN's internal state effectively in a production environment with strict latency constraints.

Methods

The core of the proposed solution is an RNN framework, specifically using Gated Recurrent Units (GRUs), designed to handle these "interleaved, unbounded sub-sequences."

  • Interleaved Sequence Modeling: Instead of treating each card's history as a single sequence, the system models it as a collection of irregular, interleaved sub-sequences. This allows the RNN to learn from the complex temporal patterns without explicit feature engineering.
  • Efficient RNN Architecture: The framework consists of three main blocks:
  • Feature Engineering & Transformation (f): This block handles non-learnable transformations (like normalization, bucketing, and timestamp encoding) and learnable embeddings for categorical features. Crucially, it avoids manual profile creation.
  • Recurrent Block (g): This is where GRUs are employed to learn and update the "state" of each card's transaction history. The state is a compressed representation of past events.
  • Classifier Block (h): This block uses the current transaction and the card's state to predict the probability of fraud.
  • Offline Training & Batch Inference: The paper details techniques for efficient batch training, including sorting sequences and optimizing GPU utilization. They also describe a batch inference process that mimics production streaming by handling sequences of varying lengths.
  • Streaming Inference: For real-time deployment, the system efficiently manages the GRU state by storing it in an embedded key-value store (HaloDB) and using an LRU cache to minimize latency. This ensures constant time and space complexity per transaction, regardless of the card's history length.

Datasets

The authors evaluated their approach on two large-scale, proprietary European financial institution datasets (Dataset A and Dataset B). These datasets contained billions of transactions from millions of cards, with varying ratios of fraudulent to legitimate transactions and different numbers of categorical and numerical features. The datasets were split chronologically into training, validation, and test sets, with the test set representing the most recent month of data.

Key Insights and Ideas

  • Feature-Free Fraud Detection: The most significant insight is that effective fraud detection can be achieved without relying on manually engineered "profiles." This drastically simplifies the development and deployment pipeline.
  • Interleaved Sequences are Key: Recognizing transaction histories as interleaved, unbounded sub-sequences is crucial for applying RNNs effectively in this domain.
  • State Management is Paramount: Efficiently managing the RNN's state for each entity (card) is critical for achieving low-latency streaming inference. Storing states in an embedded key-value store with caching is a practical solution.
  • Production Constraints Drive Design: The paper emphasizes how real-world production constraints (millisecond latencies, high throughput) heavily influence architectural and implementation choices, often favoring GRUs over LSTMs for their simplicity.

Key Results

  • Performance Improvement: The proposed RNN model, using only raw features, outperformed production-ready LightGBM models (which used extensive profiles) in recall and money recall across both datasets.
  • Dataset A: Achieved ~10% more recall and ~1.5% more money recall, saving an estimated 1 million euros.
  • Dataset B: Achieved ~3% more recall and ~6% more card recall, saving over 8 million euros.
  • Latency: The streaming inference system achieved millisecond-level latencies, with total prediction times around 126ms at the 99.999th percentile, meeting strict SLAs.
  • Efficiency: The feature-free approach significantly reduces computational resources compared to profile-based methods.
  • GRU vs. LSTM: GRUs performed comparably to LSTMs, with GRUs being preferred for production due to their simpler state management.

Limitations and Future Work

  • Profile Inclusion: The authors acknowledge that exploring the inclusion of some profiles might further improve performance and are interested in identifying which types of profiles are most beneficial.
  • Irregular Time Intervals: While the current model implicitly handles irregular time intervals, future work aims to directly embed this information into the GRU architecture for potentially better results.
  • Dataset Specificity: While the general framework is robust, specific hyperparameter tuning (like sequence length) is dataset-dependent.
  • CPU Architecture Optimization: Compiling the model for the specific CPU architecture yielded significant latency improvements, highlighting the importance of hardware-aware optimization.

Summary updated

BibTeX
@inproceedings{branco2020interleaved,
  author = {Bernardo Branco and Pedro Abreu and Ana Sofia Gomes and Mariana S. C. Almeida and João Tiago Ascensão and Pedro Bizarro},
  title = {{Interleaved Sequence RNNs for Fraud Detection}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2020},
  doi = {10.1145/3394486.3403361},
}
Figure