TL;DR
Antom 3-in-1 Transformer is a 7-billion parameter foundation model designed by Ant International to revolutionize payment risk detection. By jointly modeling transaction sequences, tabular features, and graph-based entity relationships, it achieves a massive 37.88% reduction in fraud loss rates in real-world production environments while significantly lowering the amount of labeled data required for new business scenarios.
What Problem Does This Paper Address?
Detecting fraud in global payment systems is a high-stakes game of cat-and-mouse. Fraudsters constantly evolve their tactics, and data distributions vary wildly across different markets and merchant types. Traditionally, risk systems have relied on specialized, siloed models: one might look at a user's transaction history (sequence), another at the specific details of the current transaction (tabular), and a third at how accounts are linked (graph).
The challenge is twofold:
- Fragmentation: These separate perspectives often miss the "complete picture." An ordinary-looking transaction might be high-risk when viewed through the lens of its associated device network.
- Cold-Start & Label Scarcity: Training effective models for a new business or market requires a mountain of labeled data, which is often unavailable.
The authors propose a "foundation model" approach—pretraining a massive, multi-modal model on billions of transactions to learn universal patterns of behavior that can be easily adapted to specific tasks with minimal labels.
Methods
The Antom 3-in-1 Transformer uses a modular architecture to ingest three distinct data modalities:
- Sequence Modality: Independent Transformer towers encode the histories of users, cards, and devices. It uses a novel Prefix-Future Contrast (PFC) pretraining objective, which forces the model to predict a statistical summary of future activity based on a historical prefix.
- Tabular Modality: This captures the "here and now" of a transaction. It uses field tokenization and self-attention to model interactions between attributes like transaction amount, merchant category, and business statistics.
- Graph Modality: A Graph Attention Network (GAT) processes "rolling user graphs" to capture relational risk (e.g., many accounts sharing one device).
- Task-Conditioned Fusion: To manage the 7B parameters efficiently, the model uses a Sparse Mixture-of-Experts (MoE) approach. Only about 1.7B parameters are activated for any given task route. Task-specific "queries" guide how the sequence and tabular representations are fused before being augmented by graph embeddings.
Datasets
The model was trained and evaluated on a mix of massive internal and public datasets:
- Industrial Datasets: Billion-scale internal card-payment data from Ant International, including specific domains like Antom-Merchant A/B and AlipayHK.
- IBM Synthetic Dataset: A standard benchmark containing ~24 million synthetic credit card transactions.
- Public Graph Benchmarks: Amazon, YelpChi, T-Finance, and Elliptic (Bitcoin) were used to validate the graph encoder's performance.
- Fusion Benchmarks: TransXion (AML benchmark) and the Multimodal Banking Dataset (MBD).
Key Insights and Ideas
- The Power of Multi-modality: Sequence data provides the strongest standalone signal for risk, but the "3-in-1" integration is what drives peak performance. Removing any one modality significantly degrades results.
- Label Efficiency is the Killer App: One of the most striking findings is that the pretrained model using only 25% of training labels outperforms a "from-scratch" baseline using 100% of labels. This drastically reduces the barrier to securing new payment scenarios.
- Cross-Scenario Synergy: Richer transaction histories from one domain (e.g., a global e-marketplace) can actually help improve models for a completely different, label-scarce domain. Pretraining learns "reusable behavioral capabilities" that transcend specific merchant types.
- Real-time Feasibility: Despite its size, the model fits into a strict 250ms end-to-end budget for real-time risk decisions. This is achieved by computing graph embeddings offline and using optimized GPU inference for the sequence and tabular encoders.
Key Results
The model's performance was validated through a four-month online A/B test on Ant International’s platform:
- Loss Rate Reduction: Compared to the previous production model (XGB-Online), the Antom 3-in-1 Transformer reduced the overall loss rate by 37.88%.
- Chargeback (CB) Rate: It reduced the pooled CB rate by 37.74%.
- Open-Source Benchmarking: On the IBM synthetic dataset, it achieved an Average Precision (AP) of 0.3829, outperforming the previous state-of-the-art (PRAGMA) by 12.2% and reaching a fraud recall of 83.22% at a 0.5% intervention budget.
Limitations and Future Work
The authors acknowledge several constraints:
- Transparency: Due to competitive and safety considerations, specific details regarding the model architecture, training configurations, and exact dataset construction were omitted from this report.
- Graph Latency: Currently, graph embeddings are computed offline and looked up at scoring time. Integrating dynamic, real-time graph updates into the 250ms inference window remains a challenge.
- Scope: The report focuses on card-payment risk; extending this foundation model to other financial tasks like credit scoring or insurance fraud is a logical next step.
Summary updated