Solver-Checked Predicates as Rewards for Hardware Assertion Generation

Small language models fine-tuned on hardware code can now write Verilog circuits and test inputs about as well as frontier models. They still fail badly at the harder half of hardware verification: writing formal assertions, which are logical statements about how a circuit must behave on every clock cycle and which a model checker can prove or refute. The usual training recipe gives a model a pass/fail score on its final answer and reinforces whatever produced a pass. That recipe breaks here for two reasons. A pass is rare, and an assertion that proves true is often vacuous, true of every circuit and so useless. We propose to make the model's intermediate reasoning out of predicates: short claims over circuit signals such as "count < depth whenever full is low." A solver checks each predicate, and a predicate earns reward only when the solver actually needs it to prove the final assertion.

This borrows the central move of predicate abstraction, the classical verification technique that replaces a huge concrete state space with the truth values of a few Boolean predicates. Here it serves as a dense, hard-to-game reward for reinforcement learning, not as a fixed analysis pipeline. In plain terms, the model is trained to write the lemmas a model checker would otherwise have to discover, and every lemma is checked. If this works, a compact specialized model can learn temporal, cycle-by-cycle reasoning that neither scaffolding nor outcome-only rewards have produced. The reasoning trace would also be correct by construction, not a post-hoc story.

Core Claim

For assertion generation from RTL (register-transfer level Verilog, the standard abstraction for digital circuit design), a step-level reward has three parts. First, each predicate the model emits must be proven an invariant of the design. Second, each predicate must be non-redundant given the earlier ones. Third, each predicate must appear in the solver's unsat core, meaning the minimal set of facts the solver used when proving that the final assertion is inductive, or must eliminate a spurious counterexample to that proof. We claim this reward trains a 4B-class model to write functionally correct, non-vacuous assertions at a higher rate than outcome-only reinforcement learning with the same solver budget. We further claim the advantage is largest where the base model is weakest, on designs whose properties require multi-cycle state tracking.

The reward can be written as

$$R = \mathbb{1}[A \text{ proven}] \cdot m(A) + \lambda \sum_{i} \mathbb{1}[p_i \text{ proven}] \cdot \mathbb{1}[p_i \in \mathrm{core}(A)],$$

where $A$ is the final assertion and $m(A)$ is its mutation kill rate, meaning the fraction of injected design bugs under which $A$ becomes false; this term penalizes vacuous assertions. The $p_i$ are the model's intermediate predicates, $\mathrm{core}(A)$ is the set of predicates the solver uses to make $A$ inductive, and $\lambda$ weights step credit. The core-membership term is what stops the model from padding its reasoning with trivially true predicates like x == x.

What Exists and What Does Not

The closest prior work falls into three groups. Each has one piece of this idea.

The first group is classical predicate abstraction, which supplies the method. SLAM replaces program variables with Boolean predicates while keeping control flow. VCEGAR brings counterexample-guided abstraction refinement (CEGAR) to Verilog. CEGAR repeatedly checks an abstract model, tests whether an abstract bug trace is real, and adds new predicates when it is not; VCEGAR tracks word-level predicates on multi-bit buses and finds new ones through weakest preconditions. satabs does the same bit-precisely for C. These systems contain no learning. VCEGAR in particular reports that formulas blow up on long counterexamples, which is the point where a learned predicate proposer could help. We keep their solver checks and replace weakest-precondition predicate discovery with a trained model.

The second group connects predicates to learning, but in the opposite direction or without guarantees. Policy Predicate Abstraction with CEGAR uses predicate abstraction to verify a small Q-learned policy; the learned model is the thing being checked, not the thing proposing abstractions. VisualPredicator has GPT-4V invent executable predicates for robot planning. Those predicates are filtered by trial-and-error simulation, not by a solver, and the work prompts a frontier model rather than training a compact one. We differ from both: a small model is trained to propose predicates, and a solver verifies every one.

The third group is reinforcement learning with verifiable rewards on hardware and math, which both motivates and constrains the design. CHORUS and LLM4Cov train 4B models on simulator rewards for test-input generation and beat much larger models. Neither generates assertions, and LLM4Cov needed about 420,000 simulator calls to bootstrap. VSRM builds step rewards from many sub-rollouts per step. That signal estimates a step's effect on success probability and does not check the step's truth, and it is expensive. Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning shows that outcome-only training often makes the reasoning trace non-causal: causal importance dropped in 19 of 40 ReasoningGym tasks. This is a direct warning for any outcome-only assertion trainer. Our reward is designed so that reasoning which does not contribute to the proof earns nothing.

The gap itself is documented by evaluation work. FVEval finds over 80% syntax correctness but a large functional gap for assertions, worst on Design2SVA, its task of writing assertions directly from RTL; it proposes no training method. CVDP reports Claude 3.7 Sonnet at 6% Pass@1 on checker generation (category cid13), against 48% on RTL generation. To our knowledge, no work in this file trains a model on solver-checked intermediate predicates.

Research Program

The output format is fixed. The model reads an RTL module and emits a sequence of candidate predicates. Each predicate is a SystemVerilog Assertion (SVA) over current and next-cycle signals. The sequence ends with one or more target assertions. A verification harness then does four things. It proves each predicate with a bounded check followed by an induction check, using the predicates already proven as assumptions. It discards any predicate implied by earlier ones. It attempts to prove each target assertion inductively, strengthened by the surviving predicates, and records the unsat core. Finally, it runs the target against a fixed set of mutants generated by operator, constant and wire-swap mutations. When the inductive proof of a target fails, the solver's counterexample to induction is fed back in a memoryless single-step prompt for one refinement round. We take the memoryless format from LLM4Cov, which found that long chat histories hurt.

Training has three stages. First, a short supervised warm-up on predicate-annotated traces, produced by running a classical CEGAR tool on designs from the CodeV-R1-11kRTL corpus used by CHORUS and from the VeriGen corpus (VeriGen). Yu et al. report that as few as 64 expert traces prevented reasoning collapse, so a modest warm-up should suffice. Second, group-relative reinforcement learning with the reward above. Third, an optional distillation stage using the reward-gated on-policy distillation of OPDVR, included only if a stronger teacher run exists.

All rewards are computed by solver calls whose number per sample is bounded by the number of predicates. That makes the method cheaper per step than sub-rollout schemes. We will report solver wall-clock time directly so the claim can be checked.

Baselines

The naive reference point is the 4B-class base model after the supervised warm-up alone, together with zero-shot prompting of a frontier model. On CVDP checker generation, the strongest published frontier number in our file is Claude 3.7 Sonnet at 6% Pass@1 (CVDP). For Design2SVA functional correctness, we found no reported result on Lacuna, so we will measure the frontier and open models ourselves under FVEval's equivalence protocol.

The strongest competitor is outcome-only reinforcement learning in the style of CHORUS and LLM4Cov: the same backbone, the same warm-up data, the same optimizer, and the final-answer reward $\mathbb{1}[A \text{ proven}]\cdot m(A)$ including the mutation term. It is the strongest competitor because the CHORUS recipe already beats a 671B model on a neighboring hardware task (88.0% versus 74.5% Pass@1 on CVDP-ECov, CHORUS), and because adding the mutation term removes the obvious vacuity loophole. The comparison will be fair: equal rollouts, equal total solver calls (outcome-only gets extra rollouts to use the solver budget our step checks consume), and equal hyperparameter search. We beat it if we achieve higher functional Design2SVA correctness and cid13 Pass@1 at matched budget, with non-overlapping confidence intervals across three seeds.

The process-reward competitor is VSRM-style sub-rollout step rewards (VSRM) on the same trace format, matched for wall-clock compute.

The key ablation removes core membership. In this variant, any proven, non-redundant predicate is rewarded. Its purpose is to separate "the solver checks steps" from "the solver credits steps it needs." If the two variants perform equally, the abstraction-refinement part of the idea is not doing work.

For the proof-assistance side, the classical reference is a word-level CEGAR loop in the manner of VCEGAR, run on the same designs with the same timeout.

Why not just…?

Why not just run outcome-only reinforcement learning with a model checker and mutation testing? This is the real threat, and it may work well. Our reason for doubt is that assertions needing multi-cycle invariants are rarely proven by chance, so outcome rewards are extremely sparse exactly where the model is weakest. Yu et al. also show that outcome rewards let reasoning become decoupled from answers. The matched-budget comparison settles the question. If outcome-only reinforcement learning matches us on the hard subset, the step-reward claim is false, and the contribution narrows to the mutation-gated reward itself.

Why not just wrap a frontier model in a refinement loop with formal-tool feedback? ReFoRCE shows that refinement loops spin without targeted grounding. ReDel shows that agentic gains shrink with weaker backbones. The 6% cid13 result shows frontier models lack the skill itself. We will still run a frontier model with our harness's counterexample feedback for five rounds as a reference.

Why not use a learned reward model or rubric grader, as in Crossing the Reward Bridge? That work distills an LLM judge into a 7B reward model for domains without oracles. Hardware has an oracle, and judges cannot reliably separate vacuous assertions from meaningful ones. Structured scoring as in StructVRM still grades answers rather than proving steps. Using an LLM judge here would discard a sound signal we already have.

Why not just use classical CEGAR with no model? For proving given properties, VCEGAR-style refinement is a serious baseline, and we include it. It does not, however, write assertions from RTL, which is the task with the documented gap. Where CEGAR does apply, we test whether learned predicates reduce the number of refinement iterations and avoid the formula blow-up VCEGAR reports on long traces.

Proposed Experiments

The primary evaluation uses three measures. The first is FVEval Design2SVA and NL2SVA-Human, scored by formal equivalence and by our mutation kill rate. The second is CVDP cid13 checker generation, scored by Pass@1, which is out of distribution for training. The third is a held-out set of CVDP designs with given properties, where we measure the fraction proven within a fixed timeout and the number of refinement iterations.

We also report the vacuity rate, meaning the fraction of proven assertions that kill no mutants. Because supervised specialization is known to reduce sample diversity (VerilogEval), we additionally report pass@1, pass@5 and pass@10. To test whether reasoning stays faithful, we adapt the causal-importance metric of Yu et al.: we truncate the predicate trace and measure how much final-assertion correctness drops. Our method should show a large drop; outcome-only training should show a small one.

Three decisive checks determine whether the idea holds. First, at matched solver budget, the step-reward model must beat outcome-only reinforcement learning on the subset of designs whose reference proofs need at least one auxiliary invariant. Second, the core-membership ablation must underperform the full reward. Third, across 1.7B, 4B and 8B backbones, the gap over outcome-only training must be largest at the smallest size. That pattern would show that solver-checked abstraction compensates for the base model's missing state-tracking ability, rather than merely adding compute. If the first check fails, we will report a negative result: verifiable step rewards did not improve on a well-designed outcome reward in this domain.