Language model capabilities fail in a shared, predictable order under compression and perturbation

Zhixiang Wang, Ulas BagciarXiv 2026 · arXiv

TL;DR

When you compress a large language model (LLM) through pruning or quantization, it doesn't lose capabilities at random. Instead, capabilities fail in a remarkably consistent and predictable order across different models and compression methods, driven by how "securely" a model holds a task, how common the facts are, and how much output the format requires.

What Problem Does This Paper Address?

As we move LLMs from research labs to edge devices, compression—via pruning and quantization—is standard practice. While these techniques often preserve a model's average accuracy, they can cause specific capabilities to collapse entirely.

Until now, researchers disagreed on what fails first. Some suggested reasoning degrades before knowledge; others found the opposite. The authors of this paper argue that these disagreements stem from inconsistent evaluation (e.g., comparing a multiple-choice task to a generative one). They set out to determine if there is a universal "fragility profile" for LLMs and what factors actually dictate the order of failure.

Methods

The researchers conducted a massive, preregistered study applying seven types of "damage" to eight different instruction-tuned models (including the Qwen2.5 family, Llama-3.2, Mistral, Gemma-2, and Phi-3.5).

The damages included:

  • Four random perturbations: Gaussian noise, weight zeroing, neuron ablation, and random layer removal.
  • Three compression methods: Wanda pruning, SparseGPT pruning, and GPTQ quantization.

They evaluated 16 short-answer tasks across four different answer formats: Choose (multiple choice), Answer (bare string), JSON, and Twice (repeating the answer). To analyze the decline, they used a "rank-one model" where damage acts like a single dial being turned, and each capability fails at its own specific rate, defined by a "fragility exponent."

Study design showing the seven types of damage and the four answer formats.

Datasets

The study utilized 16 diverse tasks, including:

  • Generated tasks: Arithmetic (addition), naming capitals, key-code lookup, and word copying.
  • Standard benchmarks: SciQ, BoolQ, ARC-Challenge, CommonsenseQA, MMLU (math and recall subsets), HellaSwag, PIQA, WinoGrande, and OpenBookQA.
  • Frequency Analysis: To measure "fact familiarity," the authors used the Dolma v1.7 corpus (2.6 trillion tokens) to count the occurrences of specific answers in pretraining data.

Key Insights and Ideas

  • The Shared Order: Within a model, different types of damage—even random noise—cause capabilities to fail in nearly the same order (median correlation 0.90). This suggests that fragility is a property of the model's internal knowledge structure, not the specific compression algorithm.
  • The Three Axes of Fragility: The order of failure is determined by three separable factors:
  • Task Security: Measured by the "answer margin" (the log-probability gap between the right answer and the best distractor). Secure tasks fail last.
  • Fact Familiarity: Rare facts (low frequency in training data) are lost long before common ones.
  • Answer Format: The more a model has to produce, the more fragile the capability. "Choose" is the most robust; "Twice" is the most fragile.
  • The Semantic Dementia Parallel: The way LLMs fail mirrors human semantic dementia: losing rare knowledge before familiar knowledge, and recognition (choosing) before naming (producing).
  • Predictability: Because these axes are shared, you can measure a model's fragility profile under random noise and use it to accurately predict how it will perform after actual compression (Wanda or SparseGPT).
The three separable axes: task security, fact familiarity, and answer format.

Key Results

  • High Correlation: Task effects under Wanda, SparseGPT, and GPTQ correlated with random-damage effects at a median of 0.875.
  • Accuracy Prediction: Using a fragility profile measured under random noise, the authors predicted held-out task accuracy after compression with a median absolute error of 0.11–0.12. This is significantly better than assuming a uniform decline (0.20–0.21).
  • Cross-Model Transfer: Fragility profiles even transferred across different model families. A profile measured on a Llama model could predict the failure order of a Qwen model almost as well as the Qwen model's own profile.
  • Format Impact: Producing an answer in a specific template (like JSON) makes a task significantly more fragile. In many cases, the model still "knows" the answer but fails to follow the formatting template.
Fragility profiles accurately predict which tasks and facts compression will remove first.

Limitations and Future Work

  • Long-form Generation: The predictable order held for short-answer tasks but was less consistent for long-form tasks like multi-step reasoning, coding, or complex instruction following.
  • GPTQ Outlier: While GPTQ followed the predicted order within a model, the order did not transfer well across models for this specific quantization method.
  • The "Twice" Mystery: The authors found that asking a model to write an answer twice was the most fragile format, but they couldn't fully explain why this specific repetition is so difficult for damaged models.
  • Deployment Safety: The study highlights that aggregate scores hide the loss of the "long tail" of knowledge. For safety-critical applications (like medical diagnosis of rare diseases), developers must specifically test for these rare events before deploying compressed models.

Summary updated

Abstract

Compression can preserve a language model's average accuracy while selectively degrading individual capabilities. Whether these losses follow method-specific patterns or a transferable order is unclear. We applied seven perturbation and compression methods to eight instruction-tuned models, scoring 16 tasks in four answer formats against preregistered predictions. Short-answer capabilities failed in nearly the same order under four random perturbations with different mechanisms (median correlation 0.90) and, within each model, under pruning and quantization (0.875). Intact answer margin, answer frequency and answer format predicted this order. Fragility profiles measured under random perturbation predicted which tasks and facts compression removes first and, after four-task calibration, held-out task accuracy (median absolute error 0.11-0.12, against 0.20-0.21 for uniform decline); profiles borrowed from other model families did almost as well. The order did not transfer across models under GPTQ quantization or hold for long-form tasks. Within short answers, what compression removes is predictable before deployment.
BibTeX
@misc{wang2026language,
  author = {Zhixiang Wang and Ulas Bagci},
  title = {{Language model capabilities fail in a shared, predictable order under compression and perturbation}},
  year = {2026},
  eprint = {2611.11695},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11695},
}
Figure