TL;DR
Instruction tuning is widely assumed to universally improve LLM performance, but this paper reveals that base models paired with advanced decoding strategies frequently outperform their instruction-tuned counterparts in zero-shot reasoning and under distribution shifts. The performance gains from instruction tuning are often brittle, heavily dependent on few-shot formatting patterns, and highly sensitive to how automated evaluators parse answers.
What Problem Does This Paper Address?
Instruction fine-tuning (via Supervised Fine-Tuning or Reinforcement Learning from Human Feedback) has become the standard recipe for deploying Large Language Models (LLMs). It makes models conversational, safe, and compliant with user formatting requests. However, curating these instruction datasets and training models requires massive computational resources.
This paper investigates a fundamental question: Does instruction tuning actually enhance an LLM's intrinsic reasoning capabilities, or does it merely induce surface-level pattern matching?
The authors challenge the implicit assumption that instruction-tuned models are universally superior. They argue that base models possess substantial latent reasoning abilities that are often masked by their poor formatting compliance, leading to their unfair exclusion from comparative analyses or restrictive evaluations.
Methods
The authors conducted a controlled empirical comparison between base models and their instruction-tuned variants under comparable inference-time scaling.
- Base Model Strategy: To unlock the latent reasoning of base models without instruction tuning, the authors used Chain-of-Thought (CoT) decoding (which explicitly branches at the first token by selecting top-$K$ candidates, followed by greedy decoding). Because base models struggle with formatting instructions, they employed a lightweight auxiliary Small Language Model (SLM), specifically Qwen3-0.6B, to act purely as a regex-free answer extractor from the generated text.
- Instruction-Tuned Model Strategy: These models utilized standard repeated stochastic sampling ($K=20$, temperature = 0.05) combined with explicit prompts to output answers in a structured JSON format, supported by a JSON repair library to handle malformed outputs.
- Evaluation Metric: Performance across all settings was measured using the Pass@20 metric. The study evaluated 16 open-weights models spanning 0.6B to 1T parameters across five prominent families: Qwen, LLaMA, SmolLM, DeepSeek, and Kimi.
Datasets
The models were evaluated across a diverse suite of benchmarks:
- Standard Math Reasoning: GSM8K (grade-school math word problems) and Math-500 (advanced mathematical problems).
- Robustness/Perturbation: Math-Perturb Hard, which structurally modifies Math-500 problems to disrupt solution templates and expose reliance on memorized patterns.
- Domain-Shift: MedCalc, a clinical calculation benchmark comprising over 1,000 manually curated medical calculation tasks across 55 distinct types.
Key Insights and Ideas
- Base Models Dominate in Zero-Shot: In zero-shot CoT settings, base models significantly outperform their instruction-tuned counterparts. Instruction tuning appears to alter how reasoning is elicited, making the tuned models highly sensitive to and reliant on few-shot exemplars to trigger correct reasoning paths.
- Brittleness Under Distribution Shift: Instruction-tuned models suffer sharp performance drops when faced with structural perturbations or domain shifts (like clinical math). This suggests that instruction tuning encourages the memorization of specific solution templates rather than robust, generalized reasoning.
- The "Evaluator Illusion": Standard rule-based regex graders for LaTeX math benchmarks introduce massive scoring noise. Changing the evaluator to a more robust parser (like MathVerify) dramatically shifts the apparent performance gap, proving that many reported "gains" of instruction-tuned models are simply formatting artifacts rather than actual problem-solving superiority.
Key Results
- Zero-Shot GSM8K Collapse: In zero-shot CoT settings on GSM8K, base models significantly beat instruction-tuned variants. For instance, Llama3-70B's instruction-tuned variant suffered a massive 32.67% performance drop compared to its base counterpart, while Kimi-K2 dropped by 31.23%.
- Few-Shot Narrowing: When provided with 8-shot CoT exemplars on GSM8K, instruction-tuned models recovered, yet large base models still matched or exceeded them (e.g., Kimi-K2 base scored 97.9% vs. 95.8% for the instruct version).
- MedCalc Superiority: On the domain-specific MedCalc benchmark (zero-shot CoT), base models consistently outperformed instruction-tuned variants across almost all categories and model sizes, with the largest gaps observed in smaller models.
- Template Sensitivity: When moving from Math-500 to Math-Perturb Hard, instruction-tuned models collapsed sharply (e.g., Kimi-K2 dropped from 94.20% to 76.34%), indicating a lack of robustness against template disruption.
- Grading Discrepancies: On Math-500, using a standard grader on Qwen3-14B showed a 10.8% gap favoring the base model, which shrank to just 0.8% when using the more robust MathVerify parser, highlighting how heavily evaluations depend on formatting.
Limitations and Future Work
- Benchmark Coverage: The study focuses strictly on mathematical and clinical calculation datasets. The findings may not necessarily generalize to other complex capabilities like code generation, creative writing, or agentic tool use.
- Prompt and Hyperparameter Sensitivity: The evaluation outcomes remain highly sensitive to generation settings (e.g., max tokens, temperature) and specific prompt templates.
- Lack of Causal Isolation: The analysis is purely empirical and comparative; it does not isolate which specific component of the instruction-tuning pipeline (e.g., data mixture, RLHF/DPO variants, or loss functions) drives the observed brittleness.
- Future Directions: The authors suggest exploring the application of identical advanced test-time sampling methods to instruction-tuned models to see if their zero-shot reasoning capabilities can be recovered.
Summary updated