Large Payments Model

Natraj Raman, Abhenandhan Manoharan, Vishwasreddy Thuniki, Dhvani Kansara, Anirudh Som, Hassan Firouzi, Kostis Gourgoulias, Laukik Hoshing, Sudhir Kanala, Austin Reiter, Ebby Abdul, Shantanu Chandra, Ryan Schmiedl, Zack Anderson, Sumitra GanesharXiv 2026 · arXiv

TL;DR

The Large Payments Model (LPM) is a domain-specific foundation model designed to serve as a "reusable intelligence layer" for structured payment data. By pretraining on up to 400 billion tokens of transaction records, LPM achieves massive performance gains—up to an 85% uplift in fraud detection—while meeting the strict sub-60ms latency requirements of global financial systems.

What Problem Does This Paper Address?

In the payments industry, tasks like fraud detection, authorization prediction, and payment optimization are usually handled by independent, siloed models. This is inefficient because these tasks rely on overlapping information about transaction structure and entity behavior.

However, building a "foundation model" for payments is harder than building one for text. Payment records are highly structured, containing a mix of categorical IDs (with millions of unique values), continuous amounts, and timestamps. Furthermore, these models must respect the "operational lifecycle"—they cannot use information that wouldn't be available at the exact moment a transaction is authorized. Existing Large Language Models (LLMs) struggle with this native structure, and standard tabular models often fail to capture long-term temporal dependencies or scale to billions of records.

Methods

The authors introduce two complementary architectures for LPM, both utilizing a Field-Aware Encoder that maps heterogeneous inputs (numerics, text, and high-cardinality IDs) into a common representation space.

  • The Independent Variant: Designed for maximum efficiency, this model processes transactions individually using fixed-size aggregate features for historical context. It uses a hybrid masking strategy during pretraining that combines random masks with "lifecycle-aware" masks to ensure the model doesn't "cheat" by looking at future outcomes.
  • The Sequence Variant: This model captures rich temporal context by processing entity-linked transaction histories. It uses an interleaved attention pattern (alternating between intra-transaction and inter-transaction blocks) to efficiently model dependencies across time without the computational explosion typically seen in long-sequence transformers.

A standout technical detail is the use of non-uniform partitioned embeddings for high-cardinality identifiers (like card tokens or merchant IDs). Instead of a massive, uniform embedding table, LPM allocates more capacity to frequent/important values and less to the "long tail," keeping the model size manageable.

Figure 1: From shared learning to downstream use. A pretrained LPM provides a common starting point for payment tasks, reducing the need to learn transaction structure independently for each application. The reuse routes differ in the task-specific modeling they require, while shared serving infrastructure enables operational delivery.

Datasets

The models were trained on massive, proprietary datasets from JPMorgan Chase:

  • Merchant Services: Card transactions spanning credit, debit, and prepaid rails.
  • Treasury Services: Automated Clearing House (ACH) and wire transfers.
  • Scale: Pretraining corpora ranged from 18 billion to 400 billion tokens, constructed using entity-aware stratified sampling to ensure rare but operationally critical patterns were represented.

Key Insights and Ideas

  • Structured Data as a First-Class Modality: Rather than serializing transactions into text for an LLM, LPM treats the fixed schema as a feature. This allows for specialized numerical encodings (using Fourier and piecewise-linear branches) that capture non-linear patterns like "sharp" spending thresholds.
  • The Value of Sequence: The researchers found that while simple aggregate features are sufficient for some tasks, the Sequence variant provided an additional 23 percentage point uplift in fraud detection, proving that raw temporal patterns contain signals that manual feature engineering misses.
  • Zero-Shot-ish Adaptation: A frozen LPM backbone can be adapted to entirely new tasks (e.g., predicting if a declined transaction will eventually be approved) using a simple linear probe or LoRA. This confirms that the model learns a generalized "language of payments" rather than just memorizing specific labels.
Figure 2: LPM at a glance. LPM encodes heterogeneous transaction fields with a field-aware encoder and supports two variants: an Independent model using aggregate context and hybrid masking, and a Sequence model capturing entity-linked histories for temporal context. We evaluate LPM on Merchant Services and Treasury Services, highlighting relative improvements in Merchant Services fraud detection average precision over a task-specific baseline.

Key Results

LPM demonstrated significant improvements over strong, task-specific production baselines:

  • Fraud Detection: The Independent model saw a 62.7% uplift in Average Precision (AP), while the Sequence model reached 85.5%.
  • Network Response Prediction: Fine-grained prediction of ~100 different response codes improved by 80-89% in macro-F1 score.
  • Treasury Services: ACH fraud prevention saw a 27% uplift in volume-based recall and a 63.5% uplift in value-based recall.
  • Operational Efficiency: In production, the system handles millions of transactions daily with a p99 latency under 60ms.

Limitations and Future Work

  • Fixed Schemas: The current architecture relies on a fixed schema. Extending the model to new fields or different domains currently requires significant retraining.
  • Bounded Context: While the Sequence model is powerful, it uses a bounded history (e.g., the last 32 transactions). Capturing "infinite" or very long-term entity memory remains an open challenge.
  • Cross-Domain Learning: Currently, Merchant and Treasury models are trained separately. The authors suggest that a single, unified cross-payment representation is the next logical step.
  • Generative Use-Cases: The paper focuses on discriminative tasks (prediction). Using LPM for generative tasks, such as synthetic transaction generation for privacy-preserving research, was not explored.

Summary updated

Abstract

Payment applications such as fraud detection, authorization prediction, and payment optimization draw on overlapping transaction information. This motivates a reusable payments intelligence layer that shares learned representations and predictive signals, rather than developing predictive models independently for each task. Realizing this shared capability requires handling heterogeneous transaction fields, high-cardinality identifiers, lifecycle-dependent attributes, long histories, and rare outcomes while meeting latency and cost constraints. We present Large Payments Model (LPM), a domain-specific foundation model designed for operational use and trained from scratch on large-scale structured transaction data to learn reusable representations for payment prediction tasks. LPM employs field-specific encodings and lifecycle-aware self-supervision in two complementary architectures: the Independent variant uses hybrid masking and supports efficient per-transaction inference, while the Sequence variant incorporates entity-linked histories through compact latent representations for richer temporal context. Optional distillation enables resource-constrained deployment. We pretrain LPM on up to 400 billion tokens and study scaling, downstream performance, and representation reuse. LPM improves over strong task-specific baselines across multiple prediction tasks in Merchant and Treasury Services, demonstrating applicability in distinct payment domains. Its representations also facilitate adaptation to a target not observed during pretraining through compact task-specific components. Together with production deployment, these results position LPM as a practical basis for reusable payments intelligence.
BibTeX
@misc{raman2026large,
  author = {Natraj Raman and Abhenandhan Manoharan and Vishwasreddy Thuniki and Dhvani Kansara and Anirudh Som and Hassan Firouzi and Kostis Gourgoulias and Laukik Hoshing and Sudhir Kanala and Austin Reiter and Ebby Abdul and Shantanu Chandra and Ryan Schmiedl and Zack Anderson and Sumitra Ganesh},
  title = {{Large Payments Model}},
  year = {2026},
  eprint = {2611.15814},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.15814},
}
Figure