Can Structured Templates Facilitate LLMs in Tackling Harder Tasks? : An Exploration of Scaling Laws by Difficulty

Zhichao Yang, Zhaoxin Fan, Gen Li, Yuanze Hu, Xinyu Wang, Ye Qiu, Yifan Sun, Wenjun WuarXiv 2025 · DOI · arXiv

Abstract

Recent advancements in Large Language Models (LLMs) have significantly improved reasoning capabilities. However, this paper identifies a "Scaling Law by Difficulty," where increasing the volume of low-difficulty synthetic training data can paradoxically degrade performance on complex tasks. We introduce the Structured Solution Template (SST) framework, a three-stage training pipeline designed to move models beyond surface pattern matching toward deep procedural abstraction. By utilizing procedural scaffolds and difficulty-aware curricula, the SST framework achieves substantial gains on challenging benchmarks like AIME24 and a new interdisciplinary benchmark, Dynamic English.

Introduction: The Problem of Surface Pattern Matching

While LLMs excel at many reasoning tasks, they often function as sophisticated next-token predictors, leading to "surface pattern matching." In this mode, models mimic the structure of a solution without grasping the underlying logic. A critical finding of this research is that as the volume of simple, template-generated synthetic data increases, accuracy on hard benchmarks—such as AIME24—actually declines. This suggests that excessive low-difficulty data encourages "shortcut heuristics" rather than robust reasoning.

The Scaling Law by Difficulty

The core insight of this work is the "Scaling Law by Difficulty." Reasoning performance follows a U-shaped curve relative to data complexity. While easy data might help with basic tasks, it can lead to rote imitation that fails when faced with novel, complex problems. High-difficulty data, conversely, forces the model to develop procedural abstractions.

Figure 1: Performance degradation on AIME24 as training dataset size increases with low-difficulty data, illustrating the Scaling Law by Difficulty.

Methods: The Structured Solution Template (SST) Framework

To address these challenges, we propose the Structured Solution Template (SST) framework. This framework transitions the model from simple imitation to structured planning and execution through three stages:

  1. Stage 1: Weighted Structured Fine-Tuning (SFT): Training on high-quality reasoning data (Open-R1) with logical procedures wrapped in XML-style <chain> tags. A dynamically weighted loss prioritizes reasoning tokens early in training.
  2. Stage 2: Prompt-Time Chain Injection: A lightweight 1.5B parameter "chain-generator" model creates concise plans injected into the solver's prompt at inference time, acting as a cognitive scaffold.
  3. Stage 3: Integrated Curriculum Fine-Tuning: Combining planning with self-reflection traces from DeepSeek-R1 and applying Group Relative Policy Optimization (GRPO) to internalize autonomous planning and self-correction.
Figure 2: Overview of the three-stage Structured Solution Template (SST) framework.

Datasets and Evaluation

Models were trained on the OpenR1 dataset (94k samples) and evaluated on standard benchmarks including GSM8K, MATH500, AIME 2024/2025, AMC 2023, and Gaokao English. Additionally, the authors introduced Dynamic English ("Dynamic En"), a benchmark covering nonlinear dynamics in physics, chemistry, and mathematics to test cross-domain procedural reasoning.

Key Results

The SST framework demonstrated superior performance across all benchmarks:

  • GSM8K: 84.80% accuracy, a +6.18 point gain over the baseline.
  • AIME24: 30.48%, outperforming the previous best single-model result.
  • Dynamic En: 55.00%, significantly outperforming standard baselines which clustered in the mid-10s to low-50s.
  • Efficiency: For simple tasks, injected chains reduced token counts by over 60%, while for hard tasks, they provided a necessary scaffold for reflective thought.

Conclusion

This work highlights the risks of over-relying on low-difficulty synthetic data and provides a path forward via structured templates. By teaching models to plan and self-correct through the SST framework, we can move closer to true procedural reasoning in LLMs. Future work will explore the applicability of these laws to non-mathematical domains like legal analysis and creative coding.

Summary updated

Abstract

Structured, procedural reasoning is essential for Large Language Models (LLMs), especially in mathematics. While post-training methods have improved LLM performance, they still fall short in capturing deep procedural logic on complex tasks. To tackle the issue, in this paper, we first investigate this limitation and uncover a novel finding: a Scaling Law by Difficulty, which reveals that model performance follows a U-shaped curve with respect to training data complexity -- excessive low-difficulty data impedes abstraction, while high-difficulty data significantly enhances reasoning ability. Motivated by this, we propose the Structured Solution Template (SST) framework, which uses solution templates and a curriculum of varied difficulty to explicitly teach procedural reasoning. Specifically, SST comprises (1) fine-tuning with structured solution-template chains and dynamically weighted loss to prioritize procedural logic, (2) prompt-time injection of solution templates as cognitive scaffolds to guide inference, and (3) integrated curriculum fine-tuning that explicitly teaches the model to self-plan - execute - self-correct. Experiments on GSM8K, AIME24, and new Dynamic En benchmark show that SST significantly improves both accuracy and efficiency, especially on harder problems.
BibTeX
@misc{yang2025can,
  author = {Zhichao Yang and Zhaoxin Fan and Gen Li and Yuanze Hu and Xinyu Wang and Ye Qiu and Xin Wang and Yifan Sun and Wenjun Wu},
  title = {{Can Structured Templates Facilitate LLMs in Tackling Harder Tasks? : An Exploration of Scaling Laws by Difficulty}},
  year = {2025},
  eprint = {2508.19069},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.19069},
}
Figure