LOCA: Logical Chain Augmentation for Scientific Corpus Cleaning

You-Le Fang, Dong-Shan Jian, Xiang Li, Ce Meng, Ling-Shi Meng, Chen-Xu Yan, Zhi-Zhang Bian, Yan-Qing MaarXiv 2025 · DOI · arXiv

Introduction

In the development of Large Language Models (LLMs) for the hard sciences, the principle of "garbage in, garbage out" remains a significant hurdle. While LLMs excel at general tasks, their performance in physics and mathematics is often limited by the quality of available training and evaluation data. A primary issue identified by researchers is logical incompleteness: both human experts and AI models frequently skip intermediate steps, creating logical leaps that can mask subtle but critical errors.

Existing automated cleaning methods often rely on final answer consistency, failing to verify the underlying reasoning. To address this, the LOCA (Logical Chain Augmentation) framework introduces a method to automatically clean and repair scientific datasets by enforcing rigorous, step-by-step reasoning.

Methodology: The Augment-and-Review Loop

LOCA utilizes an iterative pipeline designed to transform implicit reasoning into a structured, verifiable logical chain. The core of the framework is an iterative loop of augmentation, specialized review, and feedback.

The LOCA pipeline: An iterative loop of augmentation, specialized review, and feedback.

Key Stages of the Framework:

  1. Logical Chain Augmentation: An agent identifies "logical leaps" in a raw answer and interpolates missing intermediate steps to ensure the reasoning is "atomic." It then performs Structured Decomposition, splitting every step into a Principle (e.g., Newton's Second Law) and a Derivation (the specific mathematical application).
  2. Specialized Review: Rather than a single general reviewer, LOCA employs two specialized agents: a Principle Reviewer ($R_P$) to validate the scientific context and a Derivation Reviewer ($R_D$) to perform symbolic or mathematical checks.
  3. Iterative Feedback: If errors are detected, a Summarization Agent generates a "bug report" for the Augmentation Agent to refine the solution.
  4. Final Partitioning: A solution is only accepted if it achieves internal coherence (passing the review loop) and external consistency (matching the original raw answer).

Experimental Evaluation

LOCA was evaluated on three challenging physics corpora: PHYBench (competition-level problems), PHYSICS (PhD-qualifying exam problems), and ABench-Physics (university-level problems with synthetic reasoning).

Key Results

LOCA consistently outperformed all established baselines, including Chain-of-Thought (CoT), Tree-of-Thought (ToT), and Multi-Agent Debate (MAD).

Comprehensive comparison showing LOCA's robust performance across PHYBench, PHYSICS, and ABench-Physics.

As shown in the results, LOCA reduced residual error rates to below 2% across all three benchmarks. For instance, on PHYBench using Gemini 2.5 Pro, LOCA achieved a residual error rate of just 1.69%, a dramatic improvement over the 6.25% seen with direct prompting and the >10% error rates common in standard self-reflection methods.

Discussion and Insights

The success of LOCA highlights several critical insights for scientific AI:

  • Atomicity as a Hallucination Antidote: Forcing models to show every step makes it significantly harder to "fake" a correct answer through lucky guesses.
  • Separating "Why" from "How": Decomposing steps into Principles versus Derivations allows review agents to focus on specific error types (conceptual vs. algebraic), increasing the sensitivity of the cleaning process.
  • Data Noise in Benchmarks: Expert analysis revealed that major scientific benchmarks often have error rates exceeding 20%. By cleaning these datasets, LOCA reveals the true reasoning capabilities of top-tier models that were previously masked by noisy ground truth.

Conclusion

LOCA demonstrates that automated, high-fidelity cleaning of scientific corpora is possible through structured logical augmentation. While the iterative nature of the framework increases computational costs, the trade-off is a significant reduction in error rates, providing a more reliable foundation for training and evaluating next-generation scientific AI.

Summary updated

BibTeX
@misc{fang2025loca,
  author = {You-Le Fang and Dong-Shan Jian and Xiang Li and Ce Meng and Ling-Shi Meng and Chen-Xu Yan and Zhi-Zhang Bian and Yan-Qing Ma},
  title = {{LOCA: Logical Chain Augmentation for Scientific Corpus Cleaning}},
  year = {2025},
  eprint = {2510.01249},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.01249},
}
Figure