LLMs as Validators for Cross-Chain Bridges: Evaluating Source-Level and Transaction-Level Attack Detection on Solana

Yasir Rahman, Ishraq Samiha, Shahnewaz Karim Sakib, Anindya Bijoy DasarXiv 2026 · arXiv

TL;DR

This paper evaluates how effectively open-weight Large Language Models (LLMs) can detect and diagnose critical vulnerabilities in Solana cross-chain bridges. The researchers found a massive gap between "raising an alarm" and "correctly diagnosing a bug": while models frequently flagged code as vulnerable, they only correctly identified the specific underlying flaw in 24% of source-code cases, often relying more on code comments than executable logic.

What Problem Does This Paper Address?

Cross-chain bridges are the "high-security vaults" of the crypto world, locking up billions in collateral to issue wrapped tokens on other chains. However, they are notoriously difficult to secure. On the Solana blockchain, the runtime environment handles basic memory safety, but it leaves "application-level" validation—like checking if an account is actually signed by an admin or if a token belongs to the right "mint"—entirely to the developer.

Missing these checks has led to catastrophic losses, such as the $320M Wormhole hack and the $52M Cashio incident. While LLMs are increasingly used for security audits, we don't yet know if they truly "understand" these subtle Solana-specific logic flaws or if they are just guessing based on patterns. This paper seeks to quantify that understanding by testing models on both raw source code and transaction execution records.

Methods

The authors built a custom "lock-and-mint" bridge in native Rust for Solana and created four vulnerable variants based on real-world exploits and audit classes:

  • A1 (Mint Confusion): Accepting worthless tokens as collateral (modeled after Cashio).
  • A2 (Sysvar Spoofing): Bypassing signature checks by providing a fake instructions account (modeled after Wormhole).
  • B1 (Missing Signer): Allowing admin changes without a valid signature.
  • B2 (Arbitrary CPI): Letting an attacker substitute a malicious program during a cross-program invocation.

They evaluated six configurations of five open-weight models (including DeepSeek-V3.2, Llama-3.3-70B, and gpt-oss-120b) across two tracks:

  1. Source-Code Auditor: Reviewing Rust code to find bugs.
  2. Transaction-Record Monitor: Analyzing logs and balance changes from a completed session to detect an attack.

To ensure rigorous results, they used a "blinded" grading protocol where two independent LLMs (Claude and Gemini) graded the responses against a strict rubric to distinguish between a lucky "yes, it's buggy" guess and a correct technical diagnosis.

Figure 2: Lock-and-mint bridge architecture on a local Solana validator. Steps show (1) collateral deposit, (2) relayer attestation, (3) wrapped-token minting through release, and (4) redemption: token burning and collateral withdrawal. Red boxes identify the validation checks targeted by variants A1–B2. B2 requires changes to instruction construction and account forwarding.

Datasets

The study introduced a new benchmark dataset consisting of:

  • The Bridge Implementation: A native Rust Solana bridge with 4 vulnerable variants and 1 secure baseline.
  • Transaction Traces: Captured records of successful attacks, benign operations, and blocked attempts.
  • Model Responses: A massive set of 16,800 responses from the evaluated LLMs.
  • Graded Labels: 7,200 human-verified, LLM-assisted grades for the "vulnerable" inputs.

Key Insights and Ideas

  • Alarms are Cheap, Diagnosis is Expensive: Most models are "alarm-happy." In many configurations, they flagged 93-100% of secure code as vulnerable. This makes them "noisy" tools that could overwhelm human auditors with false positives.
  • The "Comment" Crutch: When original code comments were restored, the models' ability to diagnose bugs jumped from 24% to 60%. This suggests models are often "reading the map, not the terrain"—they find the bug because a comment describes a check that happens to be missing, rather than inferring the need for that check from the logic itself.
  • Transaction Context Matters: Models performed very differently depending on the input. For example, the "Missing Signer" (B1) bug was rarely caught in source code but was identified in nearly 100% of transaction records because the lack of a signature is glaringly obvious in an execution trace.
  • Thinking Helps (Sometimes): Enabling "thinking" (Chain-of-Thought) in DeepSeek-V3.2 significantly improved its performance in transaction-level detection (from 4% to 91% across three scenarios), though it had negligible impact on source-code auditing.
Figure 3: Number of runs (out of 100) in which each model configuration correctly identified the planted vulnerability for each attack scenario, from (a) source code and (b) transaction records.

Key Results

  • Source-Code Diagnosis: Without comments, models correctly diagnosed only 24% of vulnerable inputs.
  • False Positive Rate: Five out of six model configurations flagged nearly every program—even the secure ones—as buggy, leading to an accuracy of only 50-52% on a balanced dataset.
  • Top Performer: gpt-oss-120b showed the best discrimination, withholding alarms on about one-third of secure runs while achieving the highest diagnosis count (218 out of 400).
  • Transaction Monitoring: While high for the "Missing Signer" scenario, diagnosis averaged only 6% across the other three transaction-based attack scenarios for most models.
Figure 1: Overview. (1) A missing check enables an exploit Solana does not stop. (2) Open-weight LLMs serve as source auditors and transaction-record monitors. (3) Most models flag nearly every program, secure or not, but rarely name the actual bug.

Limitations and Future Work

The study focused on native Rust implementations rather than the popular Anchor framework, which automates many of these checks. Future work could explore if LLMs are better at auditing Anchor-based contracts. Additionally, the study used a limited set of four vulnerability types; a broader "zoo" of exploits would provide a more comprehensive picture.

The authors also suggest that future research should investigate "adversarial comments"—outdated or misleading documentation—to see if they can trick LLMs into giving a clean bill of health to buggy code.

Summary updated

Abstract

Cross-chain bridges have suffered substantial asset losses due to vulnerabilities in their on-chain programs. On Solana, missing validation checks can enable exploits even when the underlying blockchain operates correctly. The ability of large language models (LLMs) to diagnose these flaws from source code or transaction records, rather than merely raise alarms, remains insufficiently understood. We implement a native Rust lock-and-mint bridge on a local Solana validator with four exploitable variants: two model flaws from the Cashio and Wormhole incidents, while the other two represent vulnerability classes documented in Solana security audits. Paired vulnerable and secure programs, together with successful attacks, benign sessions, and blocked attempts, allow us to distinguish correct diagnoses from unsupported alarms. We evaluate six configurations of five open-weight models across 16,800 responses, compare source inputs with and without comments, and grade 7,200 answers on vulnerable inputs using two blinded LLM graders. The results reveal substantial differences across conditions: without comments, models diagnose 24% of vulnerable source inputs while frequently flagging the secure bridge. Transaction diagnosis is high for the missing-signer scenario but averages 6% across the other three scenarios. Original comments, some of which describe the flaw, increase source diagnosis to 60%. These findings highlight the importance of distinguishing correct vulnerability diagnoses from unsupported alarms in bridge security review.
BibTeX
@misc{rahman2026llms,
  author = {Yasir Rahman and Ishraq Samiha and Shahnewaz Karim Sakib and Anindya Bijoy Das},
  title = {{LLMs as Validators for Cross-Chain Bridges: Evaluating Source-Level and Transaction-Level Attack Detection on Solana}},
  year = {2026},
  eprint = {2611.21777},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21777},
}
Figure