TL;DR
BiVerify is a new framework that significantly boosts the mathematical reasoning of Large Language Models (LLMs) by "closing the loop" between two complementary verification strategies: backward verification (checking if the answer fits the problem constraints) and forward verification (checking if the logical steps are sound). By requiring a solution to pass both checks, BiVerify achieves an average accuracy gain of 13.5 percentage points over the strongest existing baselines.
What Problem Does This Paper Address?
While LLMs have become impressively good at math, they still frequently "hallucinate" incorrect steps or produce answers that violate the original problem's constraints. Existing self-correction methods usually rely on simple "reflection" (asking the model to look at its work again) or "program-driven" checks.
However, verification is inherently two-sided:
- Backward Verification is efficient—it plugs the answer back into the original equation to see if it works. But it can't tell you if your logic was sound or if you missed other possible answers.
- Forward Verification tracks the logic step-by-step. It’s great for finding where a derivation went wrong, but it can easily miss a constraint mentioned at the very beginning of a long, complex solution.
The authors identify a gap: no existing system effectively coordinates these two directions in a unified, iterative loop to ensure a candidate answer is truly robust.
Methods
The authors introduce BiVerify, a dynamically scheduled framework that manages the verification process through four main components:
- The Bidirectional Scheduler: This is the "brain" of the system. It starts with backward verification. If the answer passes, it switches to forward verification. If it fails either, it triggers a revision and resets the state. A solution is only "accepted" once it passes both directions for the same candidate answer.
- Backward Verifier: It extracts the final answer and checks it against the original problem constraints (e.g., substituting $x$ back into $\sqrt{x+6}=x$).
- Forward Verifier: It breaks the solution into "key claims" and verifies them sequentially, ensuring each step follows logically from the previous ones.
- Shared Mathematical Tools: To prevent simple calculation errors from derailing complex logic, both verifiers have access to external tools like SymPy (for symbolic math) and Z3 (for constraint solving).
Datasets
The framework was evaluated on 170 problems across five highly challenging, Olympiad-level math benchmarks:
- AIME 2024 & 2025: Problems from the American Invitational Mathematics Examination.
- BRUMO 2025: Focuses on complex conditions and multi-step reasoning.
- HMMT February 2025: Problems from the Harvard-MIT Mathematics Tournament.
- AMO Bench: A diverse set of high-school competition math problems.
The researchers tested these across seven different model configurations from the DeepSeek, GPT, and Gemini families.
Key Insights and Ideas
- Complementarity is Key: Backward verification is excellent at catching "extra" invalid solutions (like those introduced by squaring both sides of an equation), while forward verification is better at catching "omissions" (missing a second valid answer).
- Stability of Correction: Once BiVerify corrects an answer, it stays correct. The authors found a 95% stable repair rate, meaning the bidirectional checks prevent the model from "correcting" itself into a new, different error.
- Early Gains: Most of the accuracy improvements happen in the first two verification events. The scheduler is smart enough to terminate early for easy problems and spend more "thinking time" on hard ones.
- Harder Problems, Higher Gains: BiVerify shows its greatest relative strength on "Very Hard" problems (those solved by 0-1 models initially), where it achieved a staggering 232.4% relative accuracy gain compared to the initial score.
Key Results
- Overall Performance: BiVerify reached an average accuracy of 65.7%, a massive jump from the 45.6% initial score.
- Beating the Competition: It outperformed the strongest public baseline (Self-Refine) by 13.5 percentage points on average.
- Model-Specific Success: On GPT-Mini, BiVerify outperformed the best baseline by 31.2 percentage points, showing that even smaller, efficient models can reach high performance with the right verification loop.
- Synergy: The full BiVerify system outperformed "Backward-only" and "Forward-only" versions, proving that the interaction between the two directions is what drives the results.
Limitations and Future Work
- Computational Cost: BiVerify requires more LLM calls than a single-shot attempt. However, the authors note that it uses fewer total tokens and fewer Python executions than some programmatic baselines because it is more targeted in its checks.
- Initial Quality: The system still relies on the model being able to generate a somewhat plausible initial solution; if the starting point is completely nonsensical, correction is much harder.
- Expansion: While currently focused on math, the authors suggest this bidirectional "check the answer vs. check the logic" approach could be extended to scientific reasoning and code generation.
Summary updated