Do Instruction-Tuned Models Always Perform Better Than Base Models? Evidence from Math and Domain-Shifted Benchmarks

Prateek Munjal, Clement Christophe, Ronnie Rajan, Praveenkumar Kanithi

Abstract

Instruction finetuning is standard practice for improving LLM performance, yet it remains unclear whether it enhances reasoning or merely induces surface-level pattern matching. We investigate this by evaluating base and instruction-tuned models on standard math benchmarks, structurally perturbed variants, and domain-shifted tasks. Our analysis highlights two key limitations of instruction tuning. First, the performance advantage is unstable and depends heavily on evaluation settings. In zero-shot CoT settings on GSM8K, base models consistently outperform instruction-tuned variants, with drops as high as 32.67% (Llama3-70B). Instruction-tuned models only match or exceed this performance when provided with few-shot exemplars, suggesting a reliance on specific prompting patterns rather than intrinsic reasoning. Second, tuning gains are brittle under distribution shift. Our results show that base models surpass instruction-tuned variants on the domain-specific MedCalc benchmark. Additionally, instruction-tuned models show sharp declines on perturbed datasets, indicating sensitivity to prompt structure over robust reasoning.

Introduction

In the current Large Language Model (LLM) landscape, instruction finetuning (IT) is the standard recipe for making models useful. It is widely assumed that an instruction-tuned model, such as Llama-3-Instruct, is inherently more capable than its "base" counterpart. However, this assumption is rarely tested under fair, controlled conditions. This study addresses a core question: Does instruction tuning actually enhance a model's underlying reasoning, or does it simply teach the model to recognize and fill out specific "solution templates"?

If instruction tuning primarily improves surface-level pattern matching, IT models may appear performant on standard benchmarks but fail when problems are phrased differently or belong to specialized domains, such as medicine, that were underrepresented in the tuning data.

Methodology

The researchers conducted a controlled empirical comparison between base and instruction-tuned variants of 16 different models, ranging from 0.6B to 1T parameters. The model families included LLaMA, Qwen, SmolLM, DeepSeek, and Kimi.

To ensure a rigorous evaluation, the study employed:

  • Pass@20 Metric: Instead of a single-point estimate, 20 reasoning paths were sampled to determine if the model could arrive at the correct answer.
  • Inference-Time Scaling: Base models utilized Chain-of-Thought (CoT) decoding to elicit latent reasoning, while IT models used repeated stochastic sampling.
  • Zero-shot vs. Few-shot Comparison: Models were tested with no examples (zero-shot) and with few-shot exemplars to measure reliance on prompt patterns.

The evaluation spanned four challenges: GSM8K (grade-school math), Math-500 (advanced math), Math-Perturb Hard (structurally altered math problems), and MedCalc (clinical calculation tasks).

The Zero-Shot Paradox

A primary finding of this research is the "Zero-Shot Paradox." In zero-shot settings, base models consistently outperformed their instruction-tuned siblings. For instance, the Llama3-70B base model outperformed its tuned version by over 32% on the GSM8K benchmark.

Figure 1: GSM8K Zero-shot CoT performance comparison between Instruct and Base models.

Instruction-tuned models only reached parity with or exceeded base models when provided with few-shot exemplars. This suggests that instruction tuning creates a dependency on specific formats to "trigger" reasoning, whereas base models maintain active reasoning capabilities in their latent space by default.

Brittleness Under Domain Shift

The performance gains attributed to instruction tuning appear highly brittle when models are moved to specialized domains. In the MedCalc benchmark, which involves clinical calculations like dosage adjustments and risk scores, the advantages of instruction tuning largely vanished.

Figure 4: Detailed performance breakdown on the MedCalc benchmark across various clinical categories.

Base models proved more robust in handling clinical calculations. For the Kimi-K2 model, the base version led the overall score by over 6%. This suggests that the alignment process may narrow a model's focus, reducing its ability to generalize to out-of-distribution tasks.

Template Sensitivity and Evaluator Noise

The study further revealed that IT models are sensitive to prompt templates. On the Math-Perturb Hard dataset, models like Kimi-K2 saw performance drops from 94.2% to 76.3%, indicating that they often memorize solution types rather than understanding underlying mathematical logic.

Additionally, the authors identified significant "evaluator noise." Standard automated graders using regex to extract LaTeX answers were found to be unreliable, introducing scoring discrepancies of up to 10.8%. This highlights the need for more robust evaluation frameworks in LLM research.

Conclusion

The evidence suggests that instruction tuning is not a "strict upgrade" but rather a trade-off. While it makes models more user-friendly for chat interfaces, it can introduce brittleness and a reliance on surface-level patterns. Future work should focus on integrating the "test-time reasoning" strengths of base models into instruction-tuned variants without sacrificing usability.

Summary updated

Abstract

Instruction finetuning is standard practice for improving LLM performance, yet it remains unclear whether it enhances reasoning or merely induces surface-level pattern matching. We investigate this by evaluating base and instruction-tuned models on standard math benchmarks, structurally perturbed variants, and domain-shifted tasks. Our analysis highlights two key (often overlooked) limitations of instruction tuning. First, the performance advantage is unstable and depends heavily on evaluation settings. In zero-shot CoT settings on GSM8K, base models consistently outperform instruction-tuned variants, with drops as high as 32.67\% (Llama3-70B). Instruction-tuned models only match or exceed this performance when provided with few-shot exemplars, suggesting a reliance on specific prompting patterns rather than intrinsic reasoning. Second, tuning gains are brittle under distribution shift. Our results show that base models surpass instruction-tuned variants on the domain-specific MedCalc benchmark. Additionally, instruction-tuned models show sharp declines on perturbed datasets, indicating sensitivity to prompt structure over robust reasoning.
BibTeX
@misc{munjal2026do,
  author = {Prateek Munjal and Clement Christophe and Ronnie Rajan and Praveenkumar Kanithi},
  title = {{Do Instruction-Tuned Models Always Perform Better Than Base Models? Evidence from Math and Domain-Shifted Benchmarks}},
  year = {2026},
  eprint = {2601.13244},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.13244},
}
Figure