Introduction
Reasoning in Small Language Models (SLMs) is often hampered by the "dual burden" of simultaneously parsing complex natural language and executing logical deduction. While Large Language Models (LLMs) possess the capacity to handle both, SLMs (typically < 1.5B parameters) frequently struggle with the noisy complexity of natural language, leading to fragile performance on math and logic benchmarks.
This paper introduces DURIT (Decoupled Understanding from Reasoning via Iterative Training), a framework designed to boost SLM reasoning by separating linguistic comprehension from logical reasoning. By mapping natural language into a standardized "problem space," DURIT enables models as small as 0.5B parameters to achieve significant performance gains and enhanced robustness.
The Problem: Dimensionality and Noise
The authors argue that the primary bottleneck for SLMs is the sparse state space created by natural language variations. When a model is distracted by redundant details or superficial phrasing, it fails to identify the underlying logical structure. This makes reinforcement learning (RL) exploration inefficient and the resulting models sensitive to minor rewordings.
As illustrated above, the goal is to reduce the dimensionality of the problem by mapping diverse natural language inputs into a canonical, simplified form. This allows the model to focus its limited capacity on the "how" (reasoning) rather than the "what" (parsing).
The DURIT Framework
DURIT employs a three-step iterative process to co-evolve a "mapper" and a "reasoner":
- Problem Mapper Training: A dedicated mapper model is trained using RL (specifically GRPO) to translate complex questions into a concise, canonical form based on a "codebook" of implicit templates.
- Self-Distillation: To ensure the SLM can reason effectively without an external mapper at inference time, it undergoes self-distillation. The model is trained to produce correct reasoning traces on original, "messy" inputs by mimicking the successful traces it generated on the cleaned, mapped inputs.
- Reinforcement Learning: The SLM is further refined using RL on the original dataset, internalizing the structural understanding gained during the mapping and distillation phases.
Key Results and Insights
The iterative nature of DURIT allows the model to progressively tackle more complex problems by refining its internal representation of the problem space. Key findings include:
- Significant Performance Gains: A Qwen2.5-0.5B model trained with DURIT achieved an average accuracy of 42.95% across benchmarks, outperforming standard GRPO baselines. Most notably, the Llama3.2-1B model saw a dramatic improvement, jumping from a base accuracy of 15.53% to 40.03% after two iterations.
- Enhanced Robustness: Evaluated on the GSM-Symbolic benchmark, DURIT models exhibited the lowest relative performance drop when problems were perturbed (e.g., changing names or numbers). This suggests the models are performing genuine reasoning rather than pattern matching.
- Training Efficiency: DURIT demonstrates superior sample efficiency, reaching reward levels comparable to converged baselines in significantly fewer training steps.
Conclusion
DURIT demonstrates that the reasoning capabilities of SLMs can be unlocked by strategically decoupling understanding from deduction. By internalizing a canonical problem space through iterative mapping and distillation, small models can achieve a level of robustness and logic previously reserved for much larger architectures. Future work will explore scaling this method to medium-sized models and expanding the diversity of the canonical spaces used for training.
Summary updated