The Emperor’s New Reasoning: Format Imitation Overshadows Genuine Mathematical Understanding in SFT

Linyao Yang, Jian-Tao Huang, Yafei Lu, Zhenhui Jessie Li, Guirong XueEMNLP 2025

TL;DR

This paper investigates whether the impressive mathematical reasoning abilities of large language models (LLMs) achieved through supervised fine-tuning (SFT) reflect genuine understanding or merely superficial format imitation. The authors demonstrate that even a small number of correctly formatted examples can lead to significant performance gains, suggesting that current SFT practices may overestimate LLM reasoning capabilities and highlighting the need for more rigorous evaluation.

What Problem Does This Paper Address?

Large Language Models (LLMs) have shown remarkable progress in mathematical reasoning tasks, often surpassing previous benchmarks. This advancement is largely attributed to supervised fine-tuning (SFT), where models are trained on examples of step-by-step reasoning. However, a critical question remains: are these models truly understanding the underlying mathematical principles, or are they simply learning to mimic the format of correct solutions? Prior work has shown that LLMs can be brittle when inputs are slightly perturbed, hinting at a lack of deep understanding. This paper aims to disentangle the contributions of genuine reasoning ability versus format alignment in SFT-driven performance improvements, particularly in small-scale LLMs.

Methods

The core of the paper's methodology involves a series of controlled experiments designed to isolate the impact of format alignment. Key techniques include:

  • Minimal SFT: Using extremely small datasets (as few as 10 examples) to fine-tune LLMs. This allows researchers to observe the effect of minimal exposure to specific training data characteristics.
  • Controlled Data Perturbations: Creating datasets with varying degrees of correctness and structure. This includes examples with:
  • Correct answers and correct format.
  • Only correct answers (no format).
  • Incorrect answers but correct format.
  • Garbled text or disrupted steps within the format.
  • Mismatched solutions (correct format, but the reasoning doesn't lead to the stated answer).
  • Comparative Analysis: Evaluating models under different conditions (e.g., base model vs. fine-tuned, different data types, different teacher models, varying data scales) to understand the drivers of performance.
  • Model Architecture and Size Variation: Testing the impact of format alignment across a range of LLM sizes and architectures.

Datasets

The paper utilizes existing mathematical reasoning benchmarks and a distilled dataset for training.

  • Evaluation Benchmarks:
    • MATH500
    • College Math
    • Olympiad Bench
    • GSM8K
    • GaokaoEn 2023
    • Omni-math
  • Training Data Source: Distilled from the BigMath dataset (Albalak et al., 2025). The authors created minimal SFT datasets by sampling subsets from BigMath that align with the target benchmarks.

Key Insights and Ideas

  • Format is King (in low-resource SFT): The most striking finding is that even with just 10 randomly selected, format-correct examples, models achieve near state-of-the-art performance. This strongly suggests that the observed gains are primarily driven by learning to imitate the step-by-step solution structure, rather than acquiring deep reasoning skills.
  • Content Matters Less Than Format: Experiments with "content-disturbed" data (incorrect answers, garbled text, mismatched solutions) show that as long as the step-by-step format is preserved, performance gains are still substantial, often outperforming the base model. This highlights the robustness of format imitation.
  • Teacher-Student Compatibility is Crucial: The alignment between the teacher model's inductive biases and the student model's capacity is more important than the teacher's raw strength. Smaller or mid-sized teacher models that produce concise, well-structured outputs can be more effective for fine-tuning than larger, more verbose models.
  • Diminishing Returns with Scale: Increasing the amount of training data yields diminishing returns. While performance improves initially, it saturates quickly, suggesting that simply adding more formatted examples won't lead to genuine reasoning breakthroughs.
  • Base Model Capability Matters: Format alignment benefits are not universal. Small but capable models leverage it well, while weaker models struggle to generalize, and very large models show limited marginal gains, likely due to prior exposure to similar patterns.

Key Results

  • 10-Shot Format-Correct Leads to SOTA: Fine-tuning with just 10 format-correct samples resulted in performance improvements of up to +49.8% on MATH500 and +81.6% on GaokaoEn 2023, reaching or surpassing existing SOTA results.
  • Format-Only vs. Answer-Only: Models fine-tuned on 10 examples with only the final answer performed poorly (average 8.3), while those with correct format but incorrect answers achieved much higher scores (average 62.58).
  • Teacher Model Impact: In low-resource SFT, smaller teacher models like Qwen2.5-Math-7B-Instruct often led to better downstream performance for student models compared to larger, more advanced teachers like DeepSeek-R1.
  • Data Scale Saturation: Performance gains from increasing SFT data volume plateaued after around 10-25 samples, with further increases showing little consistent improvement.
  • Model Dependency: Only stronger small-scale models (e.g., Qwen2.5-Math-1.5B, Qwen2.5-Math-7B) showed substantial gains from 10-shot SFT, while weaker models (Mistral-7B-v0.3) and very large models showed limited benefits.

Limitations and Future Work

  • Focus on SFT: The study exclusively examines SFT-based approaches. It remains open whether reinforcement learning or other training paradigms are similarly susceptible to format bias.
  • Conventional Formats: The research focuses on standard step-by-step formats. Future work could explore the impact of format alignment with alternative structures like self-correction mechanisms.
  • Distinguishing Imitation from Understanding: The paper strongly suggests that current SFT practices may overestimate LLM reasoning abilities. Future research should focus on developing frameworks that can reliably distinguish between superficial format imitation and genuine mathematical understanding.

Summary updated

BibTeX
@inproceedings{yang2025emperors,
  author = {Linyao Yang and Jian-Tao Huang and Yafei Lu and Zhenhui Jessie Li and Guirong Xue},
  title = {{The Emperor’s New Reasoning: Format Imitation Overshadows Genuine Mathematical Understanding in SFT}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure