DeFiScope: Detecting Various DeFi Price Manipulations with LLM Reasoning

Juantao Zhong, Daoyuan Wu, Ye Liu, Maoyi Xie, Yang Liu, Yi Li, Ning Liu

Introduction

Decentralized Finance (DeFi) protocols manage hundreds of billions of dollars using automated price models—mathematical formulas that determine the relative value of tokens. While standard models like Uniswap’s constant product formula are well-understood, nearly 44% of recent attacks target "custom" or non-standard price models. Existing state-of-the-art detection systems, such as DeFiRanger and DeFort, rely on hardcoded rules for standard models and often fail to identify malicious manipulations in protocols with unique, complex pricing formulas. DeFiScope addresses this gap by leveraging the reasoning capabilities of Large Language Models (LLMs) to interpret custom smart contract code and infer price trends.

System Architecture

DeFiScope shifts the burden of understanding code from rigid algorithms to fine-tuned LLMs through a multi-step pipeline designed to identify complex attack patterns.

  1. Operation Recovery: The system constructs a Transfer Graph from raw transaction data to group low-level token transfers into high-level DeFi actions, such as swaps, deposits, or borrows.
  2. LLM Price Inference: DeFiScope feeds the smart contract's price-calculation source code and observed token balance changes into a fine-tuned LLM. The model is tasked with extracting the mathematical model and predicting price tendencies.
  3. Pattern Matching: Finally, the system compares the recovered operations and the LLM’s price inference against eight known attack patterns, including collateral inflation and pump-and-dump schemes.

LLM-Driven Price Inference and Fine-Tuning

To transform an LLM into an expert in DeFi mathematics, the researchers employed Chain-of-Thought (CoT) fine-tuning. Using Foundry to simulate thousands of on-chain transactions on a forked Ethereum mainnet, they created a dataset of 1,000 simulated price-change pairs. The model was trained to first extract the high-level logic from messy Solidity code and then score the credibility of specific price change statements on a scale of 1 to 10.

This approach allows the system to discard low-confidence results, drastically reducing false positives. A key insight from the study was that fine-tuning on standard pricing data significantly improved the model's ability to generalize and detect attacks on custom, non-standard models.

Evaluation and Results

DeFiScope was evaluated against three primary datasets: a benchmark of 95 real-world attacks (D1), 968 suspicious transactions from a security partner (D2), and 96,800 benign transactions (D3). The results demonstrate a significant advancement over existing tools:

  • Superior Detection: DeFiScope achieved an 80% recall rate on real-world attacks, outperforming DeFiRanger (51.6%) and DeFort (52.6%).
  • High Precision: In live-style testing on suspicious transactions, the system reached 96% precision.
  • Real-World Impact: The tool discovered 81 previously unknown historical hacks that had remained undetected for years.
  • Operational Efficiency: Despite the use of LLMs, the system maintains an average overhead of 2.5 seconds per transaction with an inference cost of approximately $0.01, making it viable for real-time monitoring.

Limitations and Future Directions

While effective, DeFiScope currently focuses on single-transaction analysis, potentially missing sophisticated "cross-transaction" attacks that occur over longer durations. Additionally, while the system can handle some closed-source protocols using standard models (Type-II prompts), unique closed-source code remains a challenge. Future work may involve integrating Program-Aided Language models (PAL) to assist the LLM in executing complex scientific computations that involve heavy exponential math.

Summary updated

BibTeX
@misc{zhong2025defiscope,
  author = {Juantao Zhong and Daoyuan Wu and Ye Liu and Maoyi Xie and Yang Liu and Yi Li and Ning Liu},
  title = {{DeFiScope: Detecting Various DeFi Price Manipulations with LLM Reasoning}},
  year = {2025},
  eprint = {2502.11521},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.11521},
}
Figure