Compression Technique Interactions in Small Language Models: A Factorial Ablation Study

Bhushit Nema, Siddharth Krishnamurthy, Anjali KadiyalaarXiv 2026 · arXiv

TL;DR

This paper provides a systematic "field guide" for compressing small language models (2.7B–7B parameters), revealing that while INT8 quantization is virtually lossless, combining 4-bit (NF4) quantization with pruning causes total model collapse. Crucially, the authors find that LoRA-based fine-tuning can recover up to 97% of performance lost to aggressive pruning—and it does so effectively even without a teacher model for distillation.

What Problem Does This Paper Address?

Deploying Large Language Models (LLMs) on edge devices like mobile phones or embedded GPUs (e.g., NVIDIA Jetson) requires aggressive compression to fit within tight memory constraints. While techniques like quantization (reducing bit-precision), pruning (removing weights), and distillation (training a student model to mimic a teacher) are well-understood in isolation, practitioners lack a map for how they interact.

Does the order of operations matter? Does quantizing a pruned model work better than pruning a quantized one? When does "stacking" these techniques lead to a non-functional model? This paper replaces the current "trial and error" approach with a rigorous factorial ablation study.

Methods

The researchers conducted a massive sweep of 120 different compression configurations across four popular model architectures: Phi-2 (2.7B), Qwen3-4B, Nemotron-Mini-4B, and Qwen2.5-7B.

Their factorial design evaluated:

  • Quantization: None (FP16), INT8, and INT4 (NF4) using the bitsandbytes library.
  • Pruning: Unstructured L1 magnitude pruning at 30% and 50% sparsity.
  • Recovery: LoRA-based fine-tuning (200 steps) with and without self-distillation.
  • Ordering: Testing different sequences of operations (e.g., Prune $\rightarrow$ Quantize vs. Quantize $\rightarrow$ Prune).

They validated these findings on both GPUs and via llama.cpp for on-device CPU deployment.

Datasets

  • Evaluation: WikiText-2 (perplexity) served as the primary metric, validated against standard benchmarks: MMLU, HellaSwag, ARC-Challenge, and BoolQ.
  • Recovery Training: 256 samples from the C4 dataset were used for the LoRA fine-tuning/distillation steps.

Key Insights and Ideas

  • The "Collapse Zone": The most striking finding is the incompatibility of NF4 quantization and pruning. Every single model configuration that combined 4-bit quantization with pruning collapsed into incoherence. Figure 1: Model collapse count by quantization and pruning level.

  • LoRA is the Recovery Engine: LoRA fine-tuning is remarkably powerful, recovering nearly all performance lost to aggressive 50% pruning in just two minutes of training.

  • Distillation is Optional: Surprisingly, "self-distillation" (using the original model as a teacher) provided no consistent benefit over simple LoRA fine-tuning. This is great news for practitioners because it halves the memory required during the recovery phase.
  • Ordering is a Safety Switch: While ordering doesn't change the performance of "healthy" models, it determines viability. Pruning before quantizing to NF4 can actually prevent model collapse, whereas quantizing first is fatal.
  • The Pruning Paradox on CPU: While pruning reduces the number of non-zero weights, it provides zero speed or memory benefits on standard CPU runtimes like llama.cpp. This is because these runtimes use dense kernels; zeroed-out weights still take up space and compute cycles unless specialized sparse kernels are used.
Figure 1: Model collapse count by quantization and pruning level. All 16 NF4+pruning configura- tions collapsed. No INT8 configuration collapsed.

Key Results

  • INT8 is "Free": INT8 quantization provides a 40–45% memory reduction with less than 0.5% accuracy degradation.
  • 4-bit Speedup: On CPUs, 4-bit quantization runs 46–54% faster than 8-bit, with a negligible perplexity difference (under 0.5 points).
  • Recovery Success: For the Qwen-7B model, LoRA recovery reduced the perplexity of a 50%-pruned model from a non-functional 920.6 down to a respectable 24.3 (a 97.4% reduction in error). Table 3: Recovery from pruning (FP16 models).
Table 3: Recovery from pruning (FP16 models). PPL before and after 200 steps of LoRA training with self-distillation.

Limitations and Future Work

The study focused exclusively on unstructured pruning. The authors noted that structured pruning (like removing entire attention heads) was often incompatible with modern "fused" model architectures. Additionally, the benchmarks were limited to English-only tasks.

Future work could extend this analysis to even newer architectures (like Llama-3 or Gemma-2) and explore different quantization backends like GPTQ or AWQ to see if the "NF4 + Pruning" collapse persists across different quantization algorithms.

Summary updated

Abstract

Deploying small language models on resource-constrained hardware requires compression, yet practitioners lack systematic guidance on how quantization, pruning, and post-compression recovery interact when combined. We present a factorial ablation study evaluating 72 compression configurations across four SLM architectures (2.7B-7B parameters), extended with a 48-configuration ordering analysis, benchmark accuracy validation on 30 compressed models, and robustness checks including multi-seed validation and full-corpus perplexity evaluation. INT8 quantization is effectively lossless (<0.5% accuracy degradation), while NF4 quantization combined with L1 magnitude pruning causes model collapse in all 16 tested combinations. LoRA-based fine-tuning reduces degraded perplexity by up to 97% after aggressive pruning; an ablation against standard fine-tuning without a teacher model shows the recovery is driven by LoRA adaptation itself rather than the distillation signal. Compression ordering shows no significant effect among viable configurations (ANOVA p = 0.96) but determines viability for NF4+pruning combinations. INT8 quantization alone provides 40-45% memory reduction with no quality loss. On-device CPU deployment via llama.cpp confirms these findings transfer off the GPU: 4-bit quantization runs 46-54% faster than 8-bit for a perplexity difference under 0.5 points, while un-structured pruning yields no CPU latency or memory benefit, consistent with its dense storage format.
BibTeX
@misc{nema2026compression,
  author = {Bhushit Nema and Siddharth Krishnamurthy and Anjali Kadiyala},
  title = {{Compression Technique Interactions in Small Language Models: A Factorial Ablation Study}},
  year = {2026},
  eprint = {2611.00464},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.00464},
}
Figure