Fraud in payment and blockchain networks shows up in its shape more than in any single record. Laundered money moves along chains of throwaway wallets. Airdrop farmers fund dozens of fresh accounts from one hub. Wash traders cycle assets around closed loops. Detectors that can see these shapes need labeled examples of them, and such labels are very scarce: illicit entities are usually under one percent of a graph. The usual remedies either invent abstract vectors with no real-world meaning or paste textbook patterns into the graph, and textbook patterns lack the realistic noise that real fraudsters add to hide.
We propose to pre-train a small generative model on unlabeled transaction neighborhoods, written out as sequences of edge "words" (who pays whom, in what direction, how much, how long after the previous transfer). We then use that model to dress verified fraud skeletons in realistic amounts, timing and camouflage edges, conditioned on real local context. The resulting synthetic fraud subgraphs are concrete, inspectable transactions. They are checked by a program that confirms the motif survived. They can be fed to any detector, including the gradient-boosted trees that currently dominate this field. In two sentences: a skeleton fixes what the fraud must structurally be, and a domain-trained sequence model fills in what it would realistically look like. We claim that this beats both hand-templated injection and embedding-space augmentation when labels are scarce.
Core Claim
Label scarcity in structural anomaly detection is a shortage of realistic examples of known shapes, not a shortage of shapes. A generator pre-trained only on the domain's own transaction syntax can supply what templates cannot. It can produce the joint distribution of amounts, inter-transfer gaps, parallel edges and benign-looking side edges that surround a motif in real data. The key ingredient is the split between a programmatic skeleton plus verifier, which supplies validity, and the learned completion, which supplies realism. Our central ablation therefore compares learned completion against template-only injection of the identical skeletons.
Relation to Prior Work
Two papers motivate the representation. TLMG4Eth shows that Ethereum transfers rendered as "transaction sentences" (value, direction, time interval) can be pre-trained BERT-style into embeddings that carry fraud signal, reaching F1 0.9041 on MulDiGraph. It uses its language model only as an encoder of linear per-account sequences and never generates data. We reuse its serialization idea but make the model autoregressive over two-dimensional neighborhoods and use it generatively. Provably Powerful Graph Neural Networks for Directed Multigraphs supplies the vocabulary for that extension. Its reverse message passing, chronological port numbering and ego IDs (a tag marking the center node, so returning paths are visible) are exactly what a serialization needs to make direction, parallel edges and cycle closure unambiguous. We encode these as token fields rather than as architecture.
Three augmentation-adjacent lines set our contrast. DAGAD creates pseudo-anomalies by permuting and concatenating GNN embeddings and gains at least 19% recall, but its samples correspond to no transactions and cannot feed a tree model. ProTAGAD uses offline LLM agents to pseudo-label textual inconsistencies; it assumes text on every node and never alters topology. MLED injects GPT-4 descriptions of fraud concepts at constant cost and gains +3.41% average, but adds no training samples. None of the three synthesizes structural fraud in non-textual multigraphs.
GADBench constrains the design. There, tree ensembles with one-hop neighbor averages beat the best GNN by 12.9 AUPRC points on average. Any method that only helps deep models is therefore suspect, which is why our augmenter is detector-agnostic. ProTAGAD's finding that entangling modalities through message passing blurs anomaly boundaries (up to 28% separability loss) is a further reason to keep the language model out of the detector entirely and use it only upstream as a data source.
Research Program
Serialization. We extract a k-hop ego-subgraph around an account. Each directed transfer becomes one edge token made of five fields: source local index, destination local index, direction relative to the ego, a log-scale amount bucket, and a bucketed time gap since the previous transfer. Local indices follow first appearance, so the same account always reuses its index and cycles are explicit. Edges are ordered chronologically, which gives multigraph port numbering for free.
Pre-training. We train a small decoder-only model from scratch on millions of unlabeled ego-subgraphs, from the Ethereum crawl behind MulDiGraph and B4E and from the simulated AML transaction stream. Training is from scratch rather than continual pre-training of a web LLM, following the evidence in MonoCoder that narrow domain syntax gets swamped in general corpora. No labels are used.
Skeletons and completion. A skeleton library encodes the motifs documented in Analysis of Cryptocurrency Transactions from a Network Perspective, the Sybil funding stars of Detecting Sybil Addresses in Blockchain Airdrops, and the wash-trading loops of Market Manipulation of Bitcoin. Each skeleton is stated only as constrained token positions, for example "edge from node 3 back to node 0." To generate an example, we take a real benign ego-subgraph as prefix, splice the skeleton's fixed fields in, and let the model infill the free fields: amounts, gaps, interleaved side edges to existing benign neighbors, and parallel transfers.
Verification. A verifier rejects samples that break the motif or violate domain constraints such as time order, peeling-chain amount monotonicity, and value conservation within tolerance. This follows the execution-feedback logic of Towards Foundation Models for MILP. Accepted subgraphs are written back as real nodes and edges. Every detector then recomputes its own features on the augmented graph.
Baselines
The naive reference is training each detector on the scarce real labels only, with class reweighting. We run it at 1%, 5% and full label budgets.
The strongest existing competitor is XGB-Graph as specified by GADBench, which is the most robust performer on this question. For context, APF reports XGBGraph at 42.9% average AUPRC across 10 datasets, against APF's own 49.6%. On the multigraph datasets, the relevant deep competitor is the adapted GIN of Provably Powerful GNNs, with reported F1 57.2% on Small High-Illicit AML versus 28.7% for plain GIN. On MulDiGraph it is TLMG4Eth at F1 0.9041. For B4E and SPN we found no reported baseline numbers on Lacuna, so we will rerun all methods ourselves.
DAGAD is the augmentation competitor. Its BlogCatalog Macro-F1 of 0.84 is not comparable to our datasets, so we rerun it.
The ablation that isolates our key ingredient is template-only injection. It uses the same skeletons and the same verifier, but fills the free fields by sampling marginal empirical distributions and adds no camouflage edges. A second ablation conditions on a random rather than a real local prefix.
For fairness, every augmentation method receives the same number of accepted synthetic positives. The same detectors (XGB-Graph, adapted multigraph GIN) are trained on each augmented graph with an identical hyperparameter search budget. Our pre-training compute is reported separately.
We count it as success if our augmentation improves AUPRC on real held-out positives over template injection and over DAGAD, for both a tree and a GNN detector, at the 1% and 5% budgets on at least the Ethereum datasets.
Why not just…?
Why not just inject templated motifs with a script? This is the real threat, and it is the main ablation above, not something to argue away. On Small High-Illicit AML the data is itself simulated from pattern generators, so templates may match or beat us there. We say so in advance. The decisive evidence is the real Ethereum data (MulDiGraph, B4E, SPN), where camouflage edges of the kind MLED documents are real. If template injection matches us there, the learned completion is not worth its cost and we will report that.
Why not just give XGB-Graph motif-count features? We will do this too, adding cycle, chain and star counts per ego-subgraph. This is complementary rather than competing: our augmentation supplies more positive rows to whatever features the tree uses. The experiment crosses feature sets with augmentation to check whether gains persist once trees already see motif counts.
Why not just prompt GPT-4 to write fraudulent transaction lists? Generalist models have no exposure to realistic Wei amounts and timing, and serializing neighborhoods hits the long-context failures reported by GuARD. We include a prompted-GPT-4 generator under the same verifier and compare its acceptance rate and downstream AUPRC.
Why not just use DAGAD's embedding augmentation? It cannot feed trees, and its samples are not inspectable. The head-to-head comparison on the GNN detector will show whether structure-level realism adds anything beyond latent oversampling.
Proposed Experiments and Decisive Checks
The primary experiment uses temporal splits on MulDiGraph, B4E, SPN, Elliptic++ (actor and money-flow views) and Small High-Illicit AML. We measure AUPRC, F1 and Recall@K on real positives only, across the three label budgets, and report means over five seeds.
Two checks decide whether gains are genuine. The first is a shortcut test: a classifier trained to tell synthetic positives from real positives should do little better than chance on the generated samples. If it succeeds easily, detectors may be learning generation artifacts. The second is a held-out-motif test: we withhold one motif family from the skeleton library and see whether augmenting with the other families still helps on that family, or actively hurts it. A further diagnostic plots AUPRC gain against the measured camouflage density of the test graphs. Our hypothesis predicts that the advantage over templates grows with camouflage. If it does not, the realism mechanism is not what is driving the result.
References
- Ethereum fraud detection via joint transaction language model and graph representation learning
- Provably Powerful Graph Neural Networks for Directed Multigraphs
- DAGAD: Data Augmentation for Graph Anomaly Detection
- ProTAGAD: A Foundation Model for TAG Anomaly Detection with Decoupled Topological and Textual Prototypes
- Can LLMs Find Fraudsters? Multi-level LLM Enhanced Graph Fraud Detection
- GADBench: Revisiting and Benchmarking Supervised Graph Anomaly Detection
- MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks
- Analysis of cryptocurrency transactions from a network perspective: An overview
- Detecting Sybil Addresses in Blockchain Airdrops: A Subgraph-based Feature Propagation and Fusion Approach
- Market Manipulation of Bitcoin: Evidence from Mining the Mt. Gox Transaction Network
- Towards Foundation Models for Mixed Integer Linear Programming
- Towards Anomaly-Aware Pre-Training and Fine-Tuning for Graph Anomaly Detection
- GuARD: Effective Anomaly Detection through a Text-Rich and Graph-Informed Language Model