Commercial agreements that people want to automate as smart contracts (programs on a blockchain that move money or change status automatically when conditions are met) are often 50 to 100 pages long. The rules that matter for automation are scattered through them. A payment milestone defined in Section 3 may be gated by an inspection procedure in Section 11 and voided by a termination clause in Section 18. Current tools for generating smart contracts with language models start from short, pre-digested requirement paragraphs. They also guide generation through a finite state machine, a list of contract states, the events that move between them, and the conditions on each move. Current tools for long legal text either read it in fixed chunks that cannot see each other or pour the whole document into a long-context model. Long-context models are known to lose track of ordered, distant facts. Nothing in between turns a long contract into a correct state machine.
We propose to read the contract the way a careful lawyer drafting code would. The model extracts small state-machine fragments from individual clauses and merges them symbolically. It then inspects the partial machine for structural holes: a guard that mentions an undefined term, a state with no exit, an event that nothing triggers. Each hole becomes a targeted query that tells the system which part of the contract to read next. The context window never has to hold the whole contract. What it holds is one clause plus a short, machine-generated list of what the current machine is still missing. Our claim is that these defect-driven queries recover the scattered, ordered dependencies that similarity-based retrieval and long-context reading both miss. We expect the advantage to grow with document length and with how far apart the dependent clauses are.
Core Claim
The bottleneck in long-document-to-contract synthesis is not model capacity but knowing what to look for. Modular systems that retrieve small support sets and compute over them symbolically avoid context collapse. Database reasoning over text shows this on databases of up to 1,000 facts, but the same work shows end-to-end accuracy falling from about 90% with perfect retrieval to about 67% with learned retrieval. That paper motivates our design and also constrains it: a modular pipeline is only as good as its recall. Our new ingredient addresses that ceiling. A state machine, unlike a bag of retrieved facts, has checkable well-formedness conditions. Every violation names a specific missing piece, which makes a far sharper retrieval query than the original task description.
We treat the partial machine $M$ as having an open set $O(M)$. $O(M)$ contains symbols referenced in guards or actions but never defined, non-terminal states with no outgoing transition, events with no triggering clause, and pairs of transitions whose guards conflict. Extraction stops when $O(M)$ is empty or a retrieval budget is exhausted. Any items still open are reported to the user as unresolved obligations rather than silently filled in. That last property matters because deployed contracts cannot easily be patched, a point made by Smart Contract Repair.
Research Program
The method has three parts.
Fragment extractor. This is a clause-level model that maps a single passage, together with the current open-set list, to a typed state-machine fragment in the SmartFSM format. SmartFSM comes from Guiding LLM-based Smart Contract Generation with Finite State Machine (FSM-SCG), which translates short requirements into this structured representation (metadata, states, variables, functions, guards, events) before writing Solidity. We reuse its representation and its 30,000-item requirement-to-state-machine data for initial fine-tuning of Qwen2.5-7B. Unlike FSM-SCG, our extractor sees only one clause at a time and is trained to output partial fragments with explicit unresolved references. To learn deontic distinctions (obligation versus permission versus prohibition), we add LEXDEMOD-style supervision. The motivation is the finding in LLMs for Law that small fine-tuned contract encoders beat prompted general models on exactly this distinction.
Symbolic merger and checker. This component unifies fragments by matching state and variable names, using defined-term resolution (contracts define their own vocabulary in a definitions section). It then computes $O(M)$. This step is deterministic code, not a model, and it is where ordering is enforced. Sequence is represented as explicit transitions rather than recovered by attention. That is our answer to Sequential-NIAH, which finds that the best long-context model orders scattered needles correctly only 63.50% of the time, with accuracy falling as length grows from 8K to 128K tokens.
Gap-to-query translator. This turns each open item into a retrieval query over the clause index. For example, the item "guard references Acceptance Certificate, undefined" becomes a lexical query for the defined term plus a dense query for its defining sentence. Clause embeddings carry document context, following Context is Gold to find the Gold Passage, which shows that adding whole-document context to chunk representations improves passage retrieval. We use that as the retrieval substrate and change what is asked of it. Lexical matching is kept because LexGLUE found a TF-IDF classifier beating several Transformers on long SCOTUS documents. Exact defined-term matching is likely to matter in contracts for the same reason.
Once $O(M)$ is empty, the machine goes to FSM-SCG's code stage with compiler (py-solc-x) and static-analyzer (Slither) feedback. Optionally, the control-flow skeleton can be exported to a temporal specification for correct-by-construction synthesis in the style of Reactive Synthesis of Smart Contract Control Flows. That work synthesizes 4-to-12-state Solidity controllers in seconds but needs hand-written formal specifications, and it cannot express existential properties. We treat its universal-only fragment as a limit on which extracted machines can take the verified path. Our contribution is upstream of both code generators.
Evaluation data. No existing benchmark pairs long contracts with ground-truth state machines, so we build two. The controlled set takes gold SmartFSM machines and their requirement sentences from the FSM-SCG data and from the ten SCSynt specifications. Each requirement sentence is rewritten in contract register and dispersed into real commercial contracts from CUAD at controlled total lengths (8K, 32K, 128K tokens) and controlled distances between dependent clauses, mirroring the Sequential-NIAH construction. The realistic set is 60 CUAD contracts with payment, milestone, escrow, or termination mechanics, which attorneys and Solidity engineers annotate with gold state machines and the clause provenance of each transition. These are the mechanics found in deployments like the Xiong'an construction fund system.
Baselines
The naive reference is truncation plus FSM-SCG. The first window of the contract that fits the extractor's context is fed to the same fine-tuned model, which produces a whole machine. This shows how much is lost when the system reads only the opening and is the floor any method must clear.
The strongest existing competitor is direct long-context extraction. A long-context model receives the entire contract and is asked for the SmartFSM in one pass, with the same compiler and Slither loop afterward. We run it with the same 7B family at its maximum context and with the strongest long-context model we can access. It is the strongest competitor because it is what practitioners would do first and because it involves no retrieval error at all. On ordered extraction alone, the best number on Lacuna is 63.50% accuracy on Sequential-NIAH. No reported result found on Lacuna for long-contract-to-state-machine extraction. FSM-SCG's 95.1% compilation success on LlaMa3.1-8B is reported for short synthetic requirements, so it is a ceiling for the code stage, not a long-document baseline.
The modular retrieval baseline is the closest design to ours. It follows the structure of Database reasoning over text: retrieve top-$k$ clauses with document-contextual embeddings plus a lexical retriever, using the task description as the query, then extract and merge fragments with the identical symbolic merger. The paper's 90%-to-67% drop under learned retrieval is the reported effect we aim to shrink.
The key ablation keeps everything in our system but replaces gap-derived queries with an equal number of extra similarity-retrieved clauses. Any gain then has to come from what the open set asks for, not from reading more text. We add a second ablation in which an LLM merges fragments in place of the symbolic merger. This isolates whether ordering enforcement matters.
Fairness. All systems share the same extractor weights where applicable, the same fine-tuning data and tuning budget, and the same downstream code stage. Each is reported against total tokens processed per contract, so a method cannot win simply by reading the document several times.
What counts as beating a baseline. Our method beats direct long-context extraction if it achieves higher transition-level F1 and higher behavioral equivalence at 32K and 128K tokens at equal or lower token cost. It beats the modular baseline if it closes at least half of the gap between that baseline and the oracle-retrieval upper bound, where the oracle is given gold clause provenance.
Why not just…?
Why not just give a long-context model the whole contract? This is the real threat. If frontier models keep improving, the window problem may simply disappear. Sequential-NIAH's monotone decline with length and the poor Solidity results in SolEval (best Pass@10 26.29%) suggest it has not disappeared yet, but neither result is measured on contracts. Our length sweep settles it directly. If the long-context model matches us at 128K, the remaining case for our method is cost and the explicit reporting of unresolved obligations, and we will say so.
Why not just use better retrieval, such as contextual chunk embeddings? Better embeddings improve the answer to a given question. They do not supply the right question. A task-level query like "payment conditions" has no reason to surface the inspection procedure that one guard depends on. The ablation that holds retrieval quality fixed and varies only the query source tests exactly this. If similarity retrieval with the same budget matches us, the gap-driven component adds nothing.
Why not just extract per chunk and let an LLM merge the pieces? Map-then-merge with a model reintroduces long-context ordering inside the merge step once fragments accumulate. It also gives no signal about what is missing. The LLM-merger ablation measures this. We expect it to do fine on short documents and to lose transitions or produce out-of-order transitions as fragment counts grow.
Why not just rely on FSM-SCG's compiler and analyzer feedback loop to fix mistakes? Compilers and Slither catch syntax errors and vulnerability patterns. They cannot catch a missing contractual condition, because code with an omitted inspection gate compiles cleanly. Our behavioral-equivalence metric exposes exactly these silent omissions, which compile-rate metrics cannot see.
Proposed Experiments
The primary metrics are transition-level F1 against the gold machine, ordering accuracy over transition chains, and behavioral equivalence. Behavioral equivalence is the fraction of event traces sampled from the gold machine on which the generated Solidity reaches the same states. Downstream metrics follow FSM-SCG and SolEval: compilation success, Slither vulnerability risk, and gas relative to code generated from the gold machine. We also report open-set recall: of the gold transitions missing after the first pass, how many were later recovered by gap queries.
The decisive check is the interaction between length and dispersal on the controlled set. Our hypothesis predicts roughly equal performance to the long-context baseline at 8K tokens, followed by a widening advantage as length and dependency distance grow. It also predicts that the same-budget similarity ablation falls between the two systems. If the curves do not separate, the core claim is wrong.
The second check uses the 60-contract realistic set to test whether synthetic dispersal is representative. We compare clause-distance distributions between the two sets and confirm that the method ranking holds.
The third check is a calibration test. Items left unresolved in $O(M)$ should be enriched for genuine contractual ambiguity, as judged by annotators, rather than for extractor errors. If that holds, the system's refusals are useful to a human reviewer and not just noise.
References
- Database reasoning over text
- Smart Contract Repair
- Guiding LLM-based Smart Contract Generation with Finite State Machine
- LLMs for Law: Evaluating Legal-Specific LLMs on Contract Understanding
- Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts
- Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings
- LexGLUE: A Benchmark Dataset for Legal Language Understanding in English
- Reactive Synthesis of Smart Contract Control Flows
- CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review
- A Blockchain Based Fund Management System for Construction Projects - A Comprehensive Case Study in Xiong'an New Area China
- SolEval: Benchmarking Large Language Models for Repository-level Solidity Smart Contract Generation