Source-Mapped Rationales for Auditing Closed-Source Smart Contracts

Most programs running on Ethereum are published only as compiled bytecode, the low-level instruction stream the blockchain actually executes, with no human-readable source. Estimates in the literature range from 87% to over 99% of deployed contracts. Language-model auditors that explain why a contract is vulnerable work well, but only on source code. Bytecode tools, by contrast, are fast but silent: rule checkers and graph networks give a yes-or-no flag and no reason. This proposal bridges the two with one observation. For the minority of contracts that do publish source, the Solidity compiler emits a source map, which links every bytecode instruction to the source span it came from. Source-level explanations of vulnerabilities can therefore be translated into explanations that point at specific bytecode instructions. A model can then be trained to produce these instruction-anchored explanations while seeing only bytecode.

The bet is that the explanation acts as extra training signal. A vulnerability label says only "this contract is reentrant." An anchored rationale says "the external CALL at offset 0x2a1 executes before the balance write at offset 0x2f7." That tells the model which instructions matter. We expect it to steer the model away from shortcuts, such as opcode counts or compiler-specific block layouts, that break when the compiler version changes or when moving from verified to unverified contracts. A side effect is that every flag comes with offsets an auditor can check, and a cheap program can check them too.

Core Claim

A bytecode-only classifier trained to generate explanations anchored to instruction offsets will be more accurate than the same model trained on labels alone when the compiler version or optimizer setting differs from training, and it will localize the vulnerable instructions. Most of the accuracy gain should survive even when the model skips the explanation at inference and emits only a label. That would show the explanation works as a regularizer during training, not as an expensive reasoning step at run time. The central empirical question is the difference between the anchored-rationale model and a label-only model given identical bytecode, labels, and compute. If that difference is not there, the idea fails.

The closest work does parts of this. Smart-LLaMA trains Llama-3.1-8B on vulnerable Solidity paired with human-reviewed explanations covering reentrancy, timestamp dependency, integer overflow and delegatecall. It reports a 6.49% average F1 gain over its best baseline, but it cannot read bytecode. We reuse its explanations as raw material and change the input modality and the anchoring. SmartBT generates comments from bytecode control-flow graphs, but only as descriptions, never as input to a classification decision. DeFiAligner and Hyperion run symbolic analysis on bytecode and use language models to reason over it. Both need external text, whitepapers or marketing sites, to compare against, which is exactly what closed-source contracts usually lack. None of these papers uses the compiler's own source-to-bytecode mapping to carry explanation supervision across the modality gap.

Research Program

Data construction. We start from verified Etherscan contracts, the pool behind SmartBT's 54,739-contract corpus, and from the Smart-LLaMA explanation data. Each source file is compiled across a grid of solc versions and optimizer settings. This yields many bytecode variants per contract that share the same label and the same explanation.

Each explanation's source references (lines, statements, modifiers) are pushed through the source map to sets of bytecode offsets. The explanation is then rewritten so that it cites those offsets and their opcodes. Where the mapping is ambiguous, for example with inlined or optimizer-merged code, we keep only the offsets of state-touching instructions such as SLOAD, SSTORE, CALL and LOG. Esim showed these "stable instructions" survive compiler changes, reaching 96.3% AUC on cross-compiler similarity, so anchors restricted to them should transfer. The mapping loss rate is itself a reported result.

Model and input. The model sees only disassembled bytecode. It is split per function using the dispatcher's function selectors, carries instruction offsets, and has stable instructions marked. It never sees source, variable names, or documentation.

Training. Training proceeds in two stages, following DynaGuard's design of supervised fine-tuning followed by Group Relative Policy Optimization (GRPO), a reinforcement-learning method that scores a group of sampled outputs against each other. DynaGuard also offers two output modes, a fast label-only mode and a reasoning mode, and we adopt both.

Stage one is supervised fine-tuning on (bytecode, anchored rationale, label) triples, mixed with label-only examples so that the fast mode exists. Stage two rewards three things computed automatically on training data: a correct label, cited offsets that fall inside the source-mapped vulnerable region, and cited opcodes that actually appear at those offsets.

The same automatic check serves as a hallucination filter at inference. A rationale that cites a nonexistent offset or the wrong opcode is caught without any human in the loop. This directly targets the known problem of plausible but ungrounded rationales.

Scope. We use an 8B backbone (Qwen3 or Llama-3.1). Vulnerability classes are the four Smart-LLaMA types plus the bytecode defect classes used by DefectChecker, so that we can evaluate on its bytecode ground truth.

Baselines

The naive reference is a majority-class predictor and an opcode-frequency classifier in the style of the BPSO-Adaboost bytecode classifier. That classifier uses 61 opcode frequencies with boosted decision trees and reached 93.2% accuracy on contract-category classification. We retrain it on our vulnerability labels with the same bytecode splits and a matched hyperparameter search. The proposal must beat it clearly on order-dependent classes like reentrancy, where frequency features have no access to execution order.

The strongest existing bytecode method is DefectChecker. It runs symbolic execution to resolve jumps and applies hand-written rules over control-flow features. It reports an 88.8% F-score on its 587-contract ground-truth set at 0.15 seconds per contract, against 40.9% for Oyente, 42.2% for Securify and 16.5% for Mythril. We run it unchanged on the same test contracts, both on its own ground-truth set and on our compiler-shifted splits. Beating it means either higher F1 on the shared defect classes, or matching F1 while also giving correct offset localization at a fast-mode latency within one order of magnitude of 0.15 seconds.

For graph learning, we train a GNN classifier on Esim-style stable-semantic graphs with the same labels. Esim reports no vulnerability-classification number; its 96.3% AUC is for similarity detection.

The decisive ablation is the same backbone, same bytecode, same labels and the same number of training tokens and steps, trained on labels only. Two further ablations isolate the anchoring: rationales kept as unanchored source-level prose (Smart-LLaMA explanations with no offsets), and rationales randomly shuffled across contracts. The proposal's key ingredient counts as real only if anchored rationales beat all three on held-out compiler versions, averaged over three seeds, by more than the seed spread.

Why not just…?

Why not just decompile the bytecode and run Smart-LLaMA on the result? Decompilation loses exactly the information auditors need. Hyperion reports decompiler failures on nested structs, dynamic arrays and obfuscated control flow. CRUSH shows that compilation flattens types into untyped 256-bit storage slots. Separately, the bytecode classification study found source-level NLP models collapse to 0.242 AUC on unverified contracts. Rather than assume the answer, we run this pipeline as an additional baseline on the same test set. If it matches us, the proposal's value reduces to localization.

Why not just distill labels from a source-level auditor and skip the explanations? This is the real threat. It is precisely the label-only ablation, trained on the same source-derived labels. If the rationale model does not beat it under compiler shift, the core claim is false, and we would report that.

Why not just use symbolic execution tools? They are fast and precise on the patterns they encode. Their failures sit on logic their rules approximate: MAIAN reaches only a 69% true-positive rate on greedy contracts, and DefectChecker's simplified path traversal misfires in complex loops. We compare class by class. If DefectChecker dominates everywhere, the fallback contribution is attaching verified explanations to its flags, which is weaker but still useful.

Proposed Experiments

Compiler-shift test. We train on bytecode from a subset of solc versions and optimizer settings and test on unseen versions and settings. Recompiling verified sources gives exact labels and anchors for the shifted bytecode. We report per-class F1, offset-localization precision and recall against source-mapped regions, and the fraction of rationales rejected by the offset checker.

Fast-mode test. We compare reasoning mode with fast mode. If fast mode keeps most of the gain over the label-only model, rationale supervision acts as a training-time regularizer, and inference cost stays close to that of a plain classifier.

Faithfulness test. For vulnerable test contracts whose fixed version exists or can be written, we recompile the fixed source. The label should flip, and the cited offsets should fall on the changed instructions.

Transfer to unverified contracts. We evaluate on DefectChecker's ground-truth set, on MAIAN's concretely validated contracts, and on a manually audited sample of closed-source contracts. Here any accuracy gain over the label-only model is the evidence that matters for the 87–99% of contracts that have no source.