NCV: A Node-Wise Consistency Verification Approach for Low-Cost Structured Error Localization in LLM Reasoning

Yulong Zhang, Li Wang, Wei Du, Peilin Li, Yuqin Dai Zhiyuan Zhao, Lingyong Fang, Ziniu Liu, Ru Zhang, Huijia Zhu, Gongshen LiuarXiv 2025 · DOI · arXiv

Introduction

As Large Language Models (LLMs) are increasingly deployed for complex problem-solving, ensuring the reliability of their multi-step reasoning—often expressed through Chain-of-Thought (CoT)—has become a critical challenge. While these models can generate plausible-looking logic, they frequently harbor subtle errors that lead to incorrect conclusions. Traditional verification methods, which evaluate reasoning chains in an end-to-end (E2E) fashion, are often computationally expensive and struggle to pinpoint the exact location of logical failures.

This paper introduces Node-wise Consistency Verification (NCV), a training-free approach designed to provide high-precision error localization with significantly reduced computational overhead. By decomposing long reasoning chains into atomic, verifiable nodes, NCV achieves superior accuracy while consuming up to 58 times fewer tokens than traditional methods.

The Challenge: Attention Dilution and High Costs

Current state-of-the-art verification techniques typically rely on E2E models to critique an entire reasoning path at once. This approach suffers from two primary drawbacks:

  1. Attention Dilution: Verifiers often lose track of specific logical dependencies when processing long, dense reasoning chains, making it difficult to identify the precise step where an error occurred.
  2. Computational Inefficiency: Generating detailed critiques for long sequences requires a high volume of output tokens, leading to increased latency and operational costs.

While Process Reward Models (PRMs) exist to check step-by-step logic, they require expensive supervised training. NCV addresses these issues by providing a training-free framework that is both granular and efficient.

Methodology: Node-wise Consistency Verification

The NCV framework shifts the verification task from holistic critique to a sequence of simple, binary judgments. The process involves three key stages:

  1. Structured Decomposition: Reasoning steps are broken down into "atomic assertions" or nodes. These nodes are organized into a logical sequence, such as a linear chain or a directed acyclic graph (DAG).
  2. Sequential Verification: The verifier evaluates each node individually. Crucially, the model is only provided with the original problem and the set of previously verified correct nodes, preventing error propagation from distracting the verifier.
  3. Binary Mode and Multi-Sampling: To maximize efficiency, NCV restricts the verifier's output to a simple binary judgment (e.g., "Correct" or "Incorrect"). To maintain high reliability, the system employs multi-sampling strategies, such as majority voting (NCV@3-B), to confirm the validity of each node.
A comparison of the traditional End-to-End (E2E) verification approach, which suffers from high token consumption and "attention dilution," versus the proposed NCV approach, which uses structured decomposition for efficient error localization.

Experimental Evaluation

The authors evaluated NCV using ProcessBench, a rigorous benchmark for error localization in mathematical reasoning. The testing spanned four diverse datasets:

  • GSM8K: Elementary word problems.
  • MATH: Competition-level math.
  • OlympiadBench: High-level Olympiad problems.
  • Omni-MATH: Advanced multi-domain mathematical reasoning.

The framework was tested across several prominent LLMs, including Qwen2.5 (7B, 32B, 72B) and Llama-3.3-70B.

Key Results and Performance

NCV consistently outperformed E2E baselines in both accuracy and efficiency.

  • Superior Error Localization: On the challenging OlympiadBench dataset using Qwen2.5-32B, NCV achieved an error localization accuracy of 55.8%, more than doubling the 25.3% achieved by the 8-vote E2E baseline.
  • Dramatic Token Savings: By utilizing binary outputs, NCV reduced token consumption from an average of 1619.2 tokens (for 8-vote E2E) to just 28.1 tokens per sample—a 58x reduction.
  • Robustness Across Scales: NCV improved F1 scores by 10% to 25% across all tested model sizes, demonstrating that structured decomposition is essential for verifying advanced reasoning.
Detailed performance metrics across various benchmarks and models, showing NCV's consistent superiority in F1 scores and error detection compared to greedy and multi-vote E2E baselines.

Conclusion

Node-wise Consistency Verification (NCV) represents a significant advancement in the field of LLM reasoning verification. By transforming complex logical critiques into a series of simple, structured checks, NCV breaks the traditional trade-off between verification accuracy and computational cost. Its training-free nature and extreme token efficiency make it an ideal candidate for large-scale automated process supervision and real-time reasoning applications.

Summary updated

Abstract

Verifying multi-step reasoning in large language models is difficult due to imprecise error localization and high token costs. Existing methods either assess entire reasoning chains, suffering attention dilution, or rely on expensive multi-sampling. We introduce Node-wise Consistency Verification (NCV), a training-free framework that recasts verification as lightweight binary consistency checks at the node level. By decomposing the chain of thought into interconnected verification nodes, NCV precisely localizes errors and avoids unnecessary long-form generation. Experiments demonstrate that our approach enhances interpretability and efficiency, presenting a scalable solution for reliable LLM reasoning verification. On public datasets, NCV achieves a 10\% to 25\% improvement in F1 scores over baselines while utilizing $6\times$~$58\times$ fewer tokens than traditional methods like CoT-based verifiers.
BibTeX
@misc{zhang2025ncv,
  author = {Yulong Zhang and Li Wang and Wei Du and Peilin Li and Yuqin Dai Zhiyuan Zhao and Lingyong Fang and Ziniu Liu and Ru Zhang and Huijia Zhu and Gongshen Liu},
  title = {{NCV: A Node-Wise Consistency Verification Approach for Low-Cost Structured Error Localization in LLM Reasoning}},
  year = {2025},
  eprint = {2510.02816},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.02816},
}
Figure