Learning Node Representations Using Stationary Flow Prediction on Large Payment and Cash Transaction Networks

Ciwan Ceylan, Salla Franzén, Florian T. PokornyICML 2021

Understanding financial networks is a critical task for banks and financial institutions, particularly for monitoring millions of daily transactions to combat money laundering (AML). Traditionally, this requires domain experts to manually engineer features—a process that is slow, expensive, and often misses complex, hidden patterns.

This paper presents a novel approach to understanding financial networks by learning node representations directly from transaction flows. By extending the classical gradient flow model with a "gated" mechanism and a robust loss function, the authors provide a way to automate feature discovery in massive, noisy transaction datasets.

The Challenge of Financial Transaction Data

While graph machine learning offers a promising solution for automated monitoring, transaction data presents two major hurdles:

  1. Model Limitations: Standard "gradient flow" models, which treat flow as a simple difference in node "potentials," cannot represent all types of network flows, such as circular or divergence-free flows.
  2. Data Extremes: Financial transactions span many orders of magnitude—from cents to millions of dollars—and are riddled with outliers. Standard loss functions like Mean Squared Error (MSE) fail in this environment because they over-prioritize a few massive transactions, leading to poor generalization.
Transaction amount distributions showing heavy-tailed properties across Ethereum, bank, and synthetic datasets.

Methodology: The Gated Gradient Model

To overcome these limitations, the authors introduce a Gated Gradient Model that acts as a universal approximator for flows on graphs, allowing for automated node representation learning without manual feature engineering. The paper introduces two primary innovations:

1. The Gated Mechanism

The authors extend the simple gradient model—where flow on an edge $(i, j)$ is just $z^{(j)} - z^{(i)}$. Instead of a single scalar potential, they assign a vector of $K$ potentials to each node. They then introduce a gate function $\sigma$ that modulates these potentials for each edge.

  • Universal Approximation: The authors mathematically prove that this gated version can approximate any flow on a graph, overcoming the fundamental limitations of the basic gradient model.
  • Economic Interpretation: The node vectors can be viewed as supply/demand indicators for different "goods" or "services," while the gates represent the "impedance" or willingness of nodes to trade.

2. Multi-scale, Outlier-Robust Loss

To handle data spanning several orders of magnitude, the authors move away from MSE. They propose a custom relative error metric and wrap it in a Student-t log-likelihood loss function. This approach is naturally robust to heavy-tailed distributions and prevents the model from being "blinded" by a few extremely large transactions.

Datasets and Experimental Setup

The researchers evaluated their model on both real-world and synthetic datasets:

  • Ethereum Transaction Data: A publicly available dataset consisting of ~453k nodes and 1.1M edges. It serves as a proxy for private bank data, exhibiting similar statistical "heavy-tail" properties.
  • Synthetic Datasets: 60 different datasets generated using three graph topologies (Cora, Bitcoin, and Complete graphs) and two types of flow distributions (unimodal and multimodal). This allowed the authors to test the model against a known "ground truth."

Key Insights and Results

The model learns representations purely from the stationary flow (the net movement of value), proving that flow prediction is a powerful pretext task for representation learning in financial contexts. Unlike methods like node2vec, which focus on structural proximity (who is near whom), this model focuses on flow dynamics. This is crucial for AML, where the behavior of money movement is more important than simple network connectivity.

Median absolute error comparison on the Ethereum dataset, highlighting the Gated model's superior performance.
  • Superior Accuracy: On the Ethereum dataset, the Gated Gradient Model achieved a median absolute error of 9.34, significantly lower than node2vec (11.25) and models using hand-engineered features (13.10).
  • Multimodal Success: In synthetic tests with complex, multi-peaked flow distributions, the gated model outperformed the standard gradient model by a wide margin, proving its ability to capture complex "hidden" states in the network.
  • The Power of Initialization: The authors found that standard random initialization performed poorly. They developed a strategy called LSQR+, which uses a least-squares solver to find a good starting point for the node potentials before fine-tuning with the gated mechanism.

Limitations and Future Work

While highly effective, the authors noted that the model tends to overfit on very sparse graphs with low clustering coefficients (like the Bitcoin link graph). While L1 regularization helped, more sophisticated techniques—like Dropout adapted for gated graph models—might be necessary for better generalization.

A promising future direction is unifying link prediction (will a transaction happen?) and flow prediction (how much will be sent?) into a single, comprehensive generative model for financial networks.

Summary updated

BibTeX
@inproceedings{ceylan2021learning,
  author = {Ciwan Ceylan and Salla Franzén and Florian T. Pokorny},
  title = {{Learning Node Representations Using Stationary Flow Prediction on Large Payment and Cash Transaction Networks}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
}
Figure