VeruSteer: Learning What to Repair Next in Verus

Chao Jiang, Ding Wang, Chaohua Yang, Dugang Liu, Zhiwu Xu, Cheng Wen, Zhong Ming, Yuwei Liu, Xinyi Wan, Lin HuangarXiv 2026 · arXiv

TL;DR

VeruSteer is a novel framework that optimizes automated formal verification for Rust by learning how to prioritize which proof error to fix first. By training a transformer-based selector on alternative "repair trajectories," it moves beyond rigid, hard-coded priority rules to make context-aware decisions, significantly increasing the success rate and efficiency of Large Language Model (LLM)-based verification pipelines.

What Problem Does This Paper Address?

Formal verification tools like Verus ensure that Rust programs are memory-safe and functionally correct by checking them against mathematical specifications. However, writing the necessary "proof code" (like loop invariants) is notoriously difficult. Modern systems use LLMs in a "repair loop": the verifier finds errors, and the LLM tries to fix them.

A major bottleneck occurs when the verifier reports multiple errors simultaneously—for example, a failing loop invariant and a potential integer overflow. Existing pipelines like AutoVerus and VeruSAGE use a fixed, "one-size-fits-all" priority list to decide which error to tackle first.

The authors argue this is suboptimal. A repair choice that seems logical now might lead to a dead end three steps later, while a different choice might resolve multiple issues at once. As shown in Figure 1, the standard approach is blind to the program's specific context and the history of previous repair attempts.

Fig. 1. A repair loop with fixed error-type priorities.

Methods

VeruSteer treats error selection as a sequential recommendation problem. Instead of a static list, it uses a learned policy to "steer" the repair process.

  1. Repair-Tree Construction: Offline, the authors built "repair trees" by exploring every possible error-type choice at each step of a verification task. This created a rich dataset of alternative paths (trajectories) from the same starting program.
  2. Trajectory Ranking: They developed a hierarchical preference system to judge these paths. A trajectory is "better" if it leads to full verification, takes fewer rounds to succeed, or makes more measurable progress (more verified obligations) when it fails.
  3. The VeruSteer Selector: This is a neural network using a self-attention (transformer) encoder. It takes three inputs: the current program code, the verifier's feedback, and the history of previous selections.
  4. Dual-Loss Training: The model is trained with two objectives:
  5. Step-wise Imitation: Learning to mimic the individual choices made in successful trajectories.
  6. Trajectory Preference Ranking: Using a Bayesian Personalized Ranking (BPR) loss to directly learn that one complete path is better than another.

Crucially, at deployment, VeruSteer adds zero overhead to the expensive LLM or verifier calls; it simply replaces the hard-coded "if-else" logic with a smarter, context-aware inference step.

Fig. 3. Hierarchical trajectory preference.

Datasets

The researchers evaluated VeruSteer using the StarVerus benchmark, which contains 762 verification tasks. They generated a massive corpus of 3,224 repair trees using two different underlying pipelines (AutoVerus and VeruSAGE) and two different LLMs (DeepSeek-Chat and Qwen-Coder). This resulted in over 56,000 individual repair trajectories for the model to learn from.

Key Insights and Ideas

  • Context is King: The "best" error to fix first changes depending on the program. In one case, fixing an invariant failure first might be the only way to prove an overflow won't happen. In another, the overflow might be the root cause.
  • The "Aha!" Moment in Trajectories: Figure 2 illustrates a motivating example where a fixed priority leads to a 5-round failure, while VeruSteer's learned path reaches verification. The key insight is that the value of a choice is often delayed—you can't judge a repair step solely by the immediate next state.
  • Learning from Failure: Even if a trajectory doesn't reach 100% verification, the relative progress it makes provides a vital signal for training. This allows the model to learn "better" behaviors even on extremely difficult tasks.
Fig. 2. Repair tree of the motivating example.

Key Results

VeruSteer demonstrated a clear advantage over standard priority rules:

  • Overall Improvement: Across 1,270 test trees, it increased the number of verified programs from 185 to 194. While a ~0.7 percentage point gain sounds modest, it represents a significant recovery of "lost" opportunities.
  • High Recovery Rate: In cases where the default rule failed but a successful path existed in the tree, VeruSteer found that path 75% of the time for AutoVerus and 81.8% of the time for VeruSAGE.
  • Efficiency: It frequently found shorter successful paths than the default rules, saving expensive LLM compute time.
  • Robustness: Ablation studies showed that the model's performance relies heavily on its ability to "remember" the repair history; removing positional embeddings (which track the sequence of steps) caused the net gain to plummet.

Limitations and Future Work

  • Search Space: The training data comes from a finite exploration of repair trees. There may be even better trajectories that were never sampled.
  • Fixed Budget: The model was optimized for a 5-round repair budget. Its performance on much longer, complex repair tasks remains an open question.
  • Single Decision Point: VeruSteer currently only decides which error to fix. It does not yet help the LLM decide how to fix it or which of several candidate patches to accept. Expanding this "steering" logic to other parts of the repair loop is a promising next step.

Summary updated

Abstract

LLM-based verification systems repair Rust proofs by revising programs in response to Verus diagnostics. When several error types are reported, pipelines such as AutoVerus and VeruSAGE select the next repair target using a preset priority. Yet each repair changes the program and the errors that follow, so the same reported types can require different choices at different states. We present VeruSteer, which learns to select error types from the outcomes of alternative repair trajectories. It constructs repair trees offline and compares paths from the same initial program by verification success, repair rounds, and final verification progress. These comparisons supervise a selector that combines step-wise imitation with trajectory preference ranking. The selector uses self-attention over the program, verifier feedback, and earlier selections to rank the currently reported types. It replaces the default priority within a single repair trajectory, keeping the existing repair agents and adding no repair or verifier calls for selection. We evaluate VeruSteer by replaying its choices on stored AutoVerus and VeruSAGE test trees under four generation configurations. Across 1,270 test trees, VeruSteer increases verified outcomes from 185 to 194 (14.57\% to 15.28\%), gaining 12 successes and losing three for a net improvement of 0.71 percentage points. Among trees where the default fails but a recorded alternative verifies, VeruSteer succeeds on 3 of 4 AutoVerus trees and 9 of 11 VeruSAGE trees. Among trees with recorded alternatives where the default verifies, it retains success on 14 of 15 AutoVerus trees and 12 of 14 VeruSAGE trees. It also finds shorter successful trajectories and improves final progress on some trees where all recorded trajectories fail.
BibTeX
@misc{jiang2026verusteer,
  author = {Chao Jiang and Ding Wang and Chaohua Yang and Dugang Liu and Zhiwu Xu and Cheng Wen and Zhong Ming and Yuwei Liu and Xinyi Wan and Lin Huang},
  title = {{VeruSteer: Learning What to Repair Next in Verus}},
  year = {2026},
  eprint = {2610.29755},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.29755},
}
Figure