BiVerify: Closing the Loop Between Backward and Forward Verification for Mathematical Self-Correction

Rui Zou, Xin Zhao, Mengqi Wei, Ji-Rong WenarXiv 2026 · arXiv

TL;DR

BiVerify is a new framework that significantly boosts the mathematical reasoning of Large Language Models (LLMs) by "closing the loop" between two complementary verification strategies: backward verification (checking if the answer fits the problem constraints) and forward verification (checking if the logical steps are sound). By requiring a solution to pass both checks, BiVerify achieves an average accuracy gain of 13.5 percentage points over the strongest existing baselines.

What Problem Does This Paper Address?

While LLMs have become impressively good at math, they still frequently "hallucinate" incorrect steps or produce answers that violate the original problem's constraints. Existing self-correction methods usually rely on simple "reflection" (asking the model to look at its work again) or "program-driven" checks.

However, verification is inherently two-sided:

  1. Backward Verification is efficient—it plugs the answer back into the original equation to see if it works. But it can't tell you if your logic was sound or if you missed other possible answers.
  2. Forward Verification tracks the logic step-by-step. It’s great for finding where a derivation went wrong, but it can easily miss a constraint mentioned at the very beginning of a long, complex solution.

The authors identify a gap: no existing system effectively coordinates these two directions in a unified, iterative loop to ensure a candidate answer is truly robust.

Methods

The authors introduce BiVerify, a dynamically scheduled framework that manages the verification process through four main components:

  • The Bidirectional Scheduler: This is the "brain" of the system. It starts with backward verification. If the answer passes, it switches to forward verification. If it fails either, it triggers a revision and resets the state. A solution is only "accepted" once it passes both directions for the same candidate answer.
  • Backward Verifier: It extracts the final answer and checks it against the original problem constraints (e.g., substituting $x$ back into $\sqrt{x+6}=x$).
  • Forward Verifier: It breaks the solution into "key claims" and verifies them sequentially, ensuring each step follows logically from the previous ones.
  • Shared Mathematical Tools: To prevent simple calculation errors from derailing complex logic, both verifiers have access to external tools like SymPy (for symbolic math) and Z3 (for constraint solving).
Figure 2: Overview of BiVerify. The system performs backward and forward verification on the same candidate, while the scheduler coordinates direction switching, candidate updates, state resetting, and joint acceptance. Dashed lines indicate on- demand calls to the shared mathematical tools.

Datasets

The framework was evaluated on 170 problems across five highly challenging, Olympiad-level math benchmarks:

  • AIME 2024 & 2025: Problems from the American Invitational Mathematics Examination.
  • BRUMO 2025: Focuses on complex conditions and multi-step reasoning.
  • HMMT February 2025: Problems from the Harvard-MIT Mathematics Tournament.
  • AMO Bench: A diverse set of high-school competition math problems.

The researchers tested these across seven different model configurations from the DeepSeek, GPT, and Gemini families.

Key Insights and Ideas

  • Complementarity is Key: Backward verification is excellent at catching "extra" invalid solutions (like those introduced by squaring both sides of an equation), while forward verification is better at catching "omissions" (missing a second valid answer).
  • Stability of Correction: Once BiVerify corrects an answer, it stays correct. The authors found a 95% stable repair rate, meaning the bidirectional checks prevent the model from "correcting" itself into a new, different error.
  • Early Gains: Most of the accuracy improvements happen in the first two verification events. The scheduler is smart enough to terminate early for easy problems and spend more "thinking time" on hard ones.
  • Harder Problems, Higher Gains: BiVerify shows its greatest relative strength on "Very Hard" problems (those solved by 0-1 models initially), where it achieved a staggering 232.4% relative accuracy gain compared to the initial score.
Figure 4: Accuracy gains over Initial by empirical difficulty. Difficulty is determined by the number of the seven base models that initially answer a problem correctly. The hori- zontal axis reports relative accuracy gain rather than absolute percentage-point change; to reduce horizontal space, the axis is broken between 100% and 200%.

Key Results

  • Overall Performance: BiVerify reached an average accuracy of 65.7%, a massive jump from the 45.6% initial score.
  • Beating the Competition: It outperformed the strongest public baseline (Self-Refine) by 13.5 percentage points on average.
  • Model-Specific Success: On GPT-Mini, BiVerify outperformed the best baseline by 31.2 percentage points, showing that even smaller, efficient models can reach high performance with the right verification loop.
  • Synergy: The full BiVerify system outperformed "Backward-only" and "Forward-only" versions, proving that the interaction between the two directions is what drives the results.

Limitations and Future Work

  • Computational Cost: BiVerify requires more LLM calls than a single-shot attempt. However, the authors note that it uses fewer total tokens and fewer Python executions than some programmatic baselines because it is more targeted in its checks.
  • Initial Quality: The system still relies on the model being able to generate a somewhat plausible initial solution; if the starting point is completely nonsensical, correction is much harder.
  • Expansion: While currently focused on math, the authors suggest this bidirectional "check the answer vs. check the logic" approach could be extended to scientific reasoning and code generation.

Summary updated

Abstract

Large language models (LLMs) have made substantial progress in mathematical reasoning but still produce incorrect solutions to complex problems, indicating the need for more reliable self-correction mechanisms. Mathematical self-correction typically identifies errors through verification, which can proceed backward or forward. The two directions provide complementary verification capabilities. Backward verification checks a candidate answer against the original constraints, bypassing complex intermediate reasoning, but cannot ensure correct reasoning or objective completeness. Forward verification follows the problem conditions and current solution, enabling it to locate reasoning errors and assess objective completeness, but may overlook key constraints in complex solution chains. We propose BiVerify, a dynamically scheduled framework for bidirectional mathematical verification. BiVerify begins with backward verification to screen the answer against the original constraints, followed by forward verification of the reasoning process; subsequent direction switching, state resetting, and re-verification are governed by verification outcomes. Upon failure, the system revises the candidate or verification flow until the same candidate passes both directions. We also introduce shared mathematical tools that separate logical reasoning from precise computation. We systematically evaluate BiVerify on five challenging mathematical reasoning benchmarks using multiple mainstream DeepSeek, GPT, and Gemini models. BiVerify exceeds the strongest representative public baseline evaluated in this work by 13.5 percentage points in average accuracy, with the margin reaching 31.2 percentage points on GPT. Further analysis shows that BiVerify's overall performance exceeds that of either verification direction alone.
BibTeX
@misc{zou2026biverify,
  author = {Rui Zou and Xin Zhao and Mengqi Wei and Ji-Rong Wen},
  title = {{BiVerify: Closing the Loop Between Backward and Forward Verification for Mathematical Self-Correction}},
  year = {2026},
  eprint = {2611.11368},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11368},
}
Figure