Mathematical Proof as a Litmus Test: Revealing Failure Modes of Advanced Large Reasoning Models

Dadi Guo, Jiayu Liu, Zhiyuan Fan, Zhitao He, Haoran Li, Yumeng Wang, Yi R. Fung

Abstract

Large reasoning models (e.g., R1, o3) have demonstrated remarkable mathematical problem-solving abilities. However, the high reported accuracy of these advanced models on popular datasets, reliance on purely numerical evaluation, and potential benchmark leakage often mask their true reasoning shortcomings. To address this, we propose leveraging the inherent rigor and methodological complexity of mathematical proofs as a diagnostic tool to expose these hidden failures. Specifically, we introduce the RFMDataset (Reveal Failure Modes), a collection of 200 diverse mathematical proof problems, and thoroughly evaluate advanced models' performance on it. Our in-depth analysis of their failures uncovers 10 fine-grained error types, which shows fundamental limitations in current large reasoning models: 1) models grapple profoundly with mathematical proofs, with some generating entirely correct proofs for less than 20% of problems; 2) models exhibit a diverse spectrum of reasoning failures, prominently demonstrating the lack of guarantees for the correctness and rigor of single-step reasoning; and 3) models show hallucination and incompleteness during the reasoning process. Our findings reveal that models' self-reflection is insufficient to resolve the current logical dilemmas, necessitating formalized and fine-grained logical training.

Introduction

While advanced reasoning models like DeepSeek-R1 and OpenAI’s o1-series boast high scores on standard math benchmarks, these numbers often mask deep-seated logical flaws. Current evaluation of Large Language Models (LLMs) in mathematics relies heavily on final numerical answers (e.g., the MATH or AIME benchmarks). However, a correct final answer does not guarantee a correct reasoning process; a model might arrive at the correct result through a series of lucky hallucinations or flawed logic.

This research argues that mathematical proofs serve as a superior "litmus test" for true reasoning. Unlike a multiple-choice question, a proof requires every intermediate step to be logically sound. By shifting the focus from what the answer is to how it was derived, we aim to diagnose the specific ways today’s "reasoning" models actually fail.

Methodology: The RFMDataset and Evaluation Pipeline

To move beyond numerical evaluation, we developed a diagnostic pipeline consisting of three main components: the RFMDataset, a fine-grained error taxonomy, and an automated evaluation framework.

The diagnostic pipeline for evaluating mathematical proofs, from dataset construction to LLM-as-a-judge analysis.

RFMDataset (Reveal Failure Modes)

The RFMDataset contains 200 proof problems across nine subjects, including Geometry, Calculus, and Number Theory. These problems span three difficulty levels: Middle School, High School, and Undergraduate. Unlike existing benchmarks that use formal languages like Lean, RFMDataset focuses on natural language proofs, reflecting how these models are primarily trained and deployed.

Evaluation Framework

We utilized a high-performing model (Gemini 2.5-pro-preview-0506) to act as a "judge," meticulously examining model-generated proofs step-by-step. The judge identifies failures based on a taxonomy of 10 distinct failure modes, such as Circular Reasoning (using the conclusion as a premise), Boundary Neglect (ignoring edge cases), and Hidden Assumptions (using unproven lemmas). This automated judge was validated against human experts, achieving a high correlation (87.61% MCC).

Results: Exposing the Reasoning Gap

Our evaluation reveals a significant "illusion" of reasoning. Many models that score 90%+ on standard benchmarks struggle to solve even 20% of the proof problems in this dataset. This suggests that current "reasoning" may often be a sophisticated form of pattern matching rather than true logical derivation.

Heatmaps showing the distribution of error types (left) and accuracy across different mathematical subjects (right) for various reasoning models.

Key Findings

  • Dominant Fallacies: The most frequent errors are not simple calculation mistakes. Instead, models struggle with Logical Violations and Incomplete Proofs. Many models also rely on Vague Arguments, asserting that a step is "obvious" when it requires complex justification.
  • Subject Performance: Geometry and Number Sequences were the hardest domains for all models, while Algebra and Number Theory were relatively easier.
  • The "Longer is Not Better" Paradox: Surprisingly, there was no clear correlation between the length of the Chain-of-Thought (CoT) and accuracy. In fact, longer reasoning chains sometimes led to lower accuracy as they provided more opportunities for logical errors to accumulate.
  • Self-Reflection Limits: Simply prompting a model to "check its work" yielded only modest improvements. Models appear to lack the domain-specific mastery needed to recognize their own logical gaps.

Conclusion and Future Work

The RFMDataset highlights that current large reasoning models frequently succumb to logical dilemmas when forced to show rigorous work. To address these limitations, future research should focus on:

  • Formal Verification: Integrating models with formally verifiable environments (like Lean or Coq) to provide objective ground truth feedback.
  • Domain-Specific Training: Developing high-quality, proof-centric datasets to train models on the methodology of mathematics rather than just final results.
  • Memory Mechanisms: Implementing mechanisms to store and reference proven lemmas during long-form proofs to prevent error accumulation.

Summary updated

Abstract

Large reasoning models (e.g., R1, o3) have demonstrated remarkable mathematical problem-solving abilities. However, the high reported accuracy of these advanced models on popular datasets, reliance on purely numerical evaluation and potential benchmark leakage, often masks their true reasoning shortcomings. To address this, we propose leveraging the inherent rigor and methodological complexity of mathematical proofs as a diagnostic tool to expose these hidden failures. Specifically, we introduce the RFMDataset (Reveal Failure Modes), a collection of 200 diverse mathematical proof problems, and thoroughly evaluate advanced models' performance on it. Our in-depth analysis of their failures uncovers 10 fine-grained error types, which shows fundamental limitations in current large reasoning models: 1) large reasoning models grapple profoundly with mathematical proofs, with some generating entirely correct proofs for less than 20% of problems and failing even on basic ones; 2) models exhibit a diverse spectrum of reasoning failures, prominently demonstrating the lack of guarantees for the correctness and rigor of single-step reasoning; and 3) models show hallucination and incompleteness during the reasoning process. Our findings reveal that models' self-reflection is insufficient to resolve the current logical dilemmas, necessitating formalized and fine-grained logical training.
BibTeX
@inproceedings{guo2026mathematical,
  author = {Dadi Guo and Jiayu Liu and Zhiyuan Fan and Zhitao He and Haoran Li and Yumeng Wang and Yi R. Fung},
  title = {{Mathematical Proof as a Litmus Test: Revealing Failure Modes of Advanced Large Reasoning Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.582},
}
Figure