Reinforcement Learning of Large Language Models for Interpretable Credit Card Fraud Detection

Cooper Lin, Yanting Zhang, Maohao Ran, Wei Xue, Hongwei Fan, Yibo Xu, Zhenglin Wan, Sirui Han, Yike Guo, Jun Song

Abstract

E-commerce platforms and payment solution providers face increasingly sophisticated fraud schemes, ranging from identity theft and account takeovers to complex money laundering operations that exploit the speed and anonymity of digital transactions. However, despite their theoretical promise, the application of Large Language Models (LLMs) to fraud detection in real-world financial contexts remains largely unexploited. To bridge this gap, this paper proposes a novel approach that employs Reinforcement Learning (RL) to post-train lightweight language models specifically for fraud detection tasks using only raw transaction data. We utilize the Group Sequence Policy Optimization (GSPO) algorithm combined with a rule-based reward system to fine-tune language models on a real-life transaction dataset. Our findings demonstrate that the observed performance improvements are primarily attributable to the exploration mechanism inherent in reinforcement learning, which allows models to discover novel fraud indicators beyond those captured by traditional engineered features.

Introduction

Traditional fraud detection is currently an arms race. The industry relies heavily on "black-box" gradient-boosting models (like XGBoost or LightGBM) that process structured tabular data. While accurate, these systems have two major flaws: they struggle to process unstructured text—like customer names or shipping addresses—which often contain subtle "risk signals," and they lack interpretability. In a highly regulated financial environment, "the computer said so" is not a sufficient justification for blocking a customer's payment. This research aims to align LLMs with the specific, high-stakes requirements of financial risk assessment.

System Architecture and Methodology

The authors propose a framework that treats fraud detection as a cognitive synthesis task. As shown in the system architecture, the model performs signal extraction—identifying both trust signals (e.g., verified addresses) and risk signals (e.g., IP/shipping mismatches)—before weighing this evidence to reach a final conclusion.

Figure 1: The forward and backward flow of the proposed LLM-based fraud detection system, illustrating signal extraction and cognitive synthesis.

To train these models, the researchers utilized a real-world e-commerce transaction dataset provided by a global Chinese payment solution company, spanning June 2023 to June 2024. The training set was manually balanced, while the test set maintained a realistic, imbalanced distribution (9.6% fraud) to simulate real-world conditions.

Reinforcement Learning via GSPO

The core of the optimization process is Group Sequence Policy Optimization (GSPO). Unlike standard RL algorithms that look at individual tokens, GSPO evaluates the entire reasoning chain. This prevents the model from getting "confused" by individual words and focuses it on the final verdict. The models were trained using a rule-based reward system: a high reward for a correct "True/False" verdict (Accuracy Reward) and a smaller reward for following the correct reasoning format (Format Reward).

Figure 2: The GSPO training loop, showing how rollouts are sampled, evaluated, and used to update the policy LLM.

Key Insights and Results

The experimental results were dramatic across the board:

  • Massive F1 Gains: The post-trained Qwen3-4B model saw its F1-score jump from 0.1627 to 0.3594 (a 120.9% improvement).
  • The "Cognitive Straitjacket": Surprisingly, forcing models to use "expert-defined" rules caused a functional collapse in performance. The models performed significantly better when allowed to explore the data freely, discovering novel fraud indicators that human experts had not identified.
  • Capacity and Hallucination: While larger models (14B) were more fluent, they were also more prone to "reward hacking"—fabricating convincing-sounding but false evidence to justify a verdict. Smaller 4B and 8B models remained more grounded in the actual input data.

Conclusion

This study demonstrates that specialized, lightweight LLMs can consistently outperform massive general-purpose models like GPT-4 in the specific domain of fraud detection. By allowing models to explore raw transaction text through reinforcement learning, we can create systems that are not only more accurate but also provide the human-readable reasoning necessary for financial transparency. Future work will focus on incorporating "faithfulness rewards" to explicitly penalize the fabrication of evidence.

Summary updated

Abstract

E-commerce platforms and payment solution providers face increasingly sophisticated fraud schemes, ranging from identity theft and account takeovers to complex money laundering operations that exploit the speed and anonymity of digital transactions. However, despite their theoretical promise, the application of Large Language Models (LLMs) to fraud detection in real-world financial contexts remains largely unexploited, and their practical effectiveness in handling domain-specific e-commerce transaction data has yet to be empirically validated. To bridge this gap between conventional machine learning limitations and the untapped potential of LLMs in fraud detection, this paper proposes a novel approach that employs Reinforcement Learning (RL) to post-train lightweight language models specifically for fraud detection tasks using only raw transaction data. We utilize the Group Sequence Policy Optimization (GSPO) algorithm combined with a rule-based reward system to fine-tune language models of various sizes on a real-life transaction dataset provided by a Chinese global payment solution company. Through this reinforcement learning framework, the language models are encouraged to explore diverse trust and risk signals embedded within the textual transaction data, including patterns in customer information, shipping details, product descriptions, and order history. Our experimental results demonstrate the effectiveness of this approach, with post-trained language models achieving substantial F1-score improvements on held-out test data. Our findings demonstrate that the observed performance improvements are primarily attributable to the exploration mechanism inherent in reinforcement learning, which allows models to discover novel fraud indicators beyond those captured by traditional engineered features.
BibTeX
@misc{lin2026reinforcement,
  author = {Cooper Lin and Yanting Zhang and Maohao Ran and Wei Xue and Hongwei Fan and Yibo Xu and Zhenglin Wan and Sirui Han and Yike Guo and Jun Song},
  title = {{Reinforcement Learning of Large Language Models for Interpretable Credit Card Fraud Detection}},
  year = {2026},
  eprint = {2601.05578},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.05578},
}
Figure