Scaling Limits of Model Robustness and Prompt Sensitivity

131 concepts102 papers1 proposal

The prevailing intuition in machine learning is that scaling—increasing parameters, data, and compute—acts as a universal solvent for model weaknesses. While larger models undeniably achieve higher accuracy on standard benchmarks, recent research reveals a more complex reality: robustness is not a monotonic function of scale. In many cases, increasing a model's size does not eliminate its sensitivity to how a prompt is formatted, and in some instances, it can even make the model more brittle or stubborn.

The Structural Fragility of Scale

We might expect a trillion-parameter model to be indifferent to a single trailing space or a choice of punctuation. However, Su et al. demonstrate that the "formatting glue" of a prompt—such as the specific character used to separate few-shot examples—can drastically swing the performance of even the largest models. Surprisingly, they found that larger models often exhibit higher performance volatility than smaller ones in response to these minor changes.

This structural sensitivity extends to the physical layout of text. Li et al. show that state-of-the-art models like GPT-4o suffer accuracy drops of up to 40% when text is formatted vertically. This vulnerability suggests that scaling reasoning capabilities does not automatically confer resilience to unconventional text layouts. Similarly, Perez et al. argue that the high capacity of larger models allows for complex decision boundaries that are poorly captured by small validation sets, making it harder to identify stable, optimal prompt configurations in "true" few-shot settings.

The Stubbornness of Strong Priors

As models grow, they internalize a more rigid world model. While this helps them ignore social cues or polite language that might distract smaller models (Salinas et al.), it also leads to a phenomenon known as posterior collapse.

Chochlakis et al. found that in subjective tasks, larger models often ignore the reasoning provided in a Chain-of-Thought (CoT) prompt, relying instead on their internal pre-existing biases. This "malleability gap" is further explored by Ortu et al., who show that larger models are more resistant to counterfactual instructions (e.g., being told a lie). Their internal factual representations become so strong that they override the in-context information provided by the user.

This stubbornness has significant implications for knowledge editing. Ma et al. observed that surgical updates to a model's knowledge can cause more severe degradation of neighboring facts in larger models than in smaller ones. The very scale that makes them powerful also makes their internal knowledge graphs more interconnected and harder to modify without unintended side effects.

The Typos Wall: Scaling as a Buffer, Not a Cure

There is a distinct difference between a model's ability to handle semantic variations (like synonyms) and syntactic noise (like typos). Zhang et al. and Wahle et al. show that while scaling improves robustness to synonymous phrasing, it fails to resolve sensitivity to character-level perturbations. Even the most capable models experience significant performance drops when faced with simple keyboard typos or natural Wikipedia-style edits (Schlegel et al.).

Hagen et al. describe scaling as a "natural buffer" rather than a solution. While a 7B-parameter model is more robust than a BERT-base model, it remains statistically sensitive to minor lexical variations. This suggests that the underlying lack of linguistic robustness is a fundamental property of current neural architectures rather than a simple capacity issue.

Why Scale Doesn't Solve Instability

If scaling doesn't fix robustness, why does it happen? The evidence suggests two competing mechanisms:

  1. Increased Precision: Larger models are more precise, meaning they can distinguish between very subtle patterns. However, this precision makes them "overfit" to the specific structure of a prompt.
  2. Internal Consensus: As Scherrer et al. note, larger models develop a shared consensus on complex tasks, which stabilizes their "beliefs" but makes them less steerable by the user.

Ultimately, Chatterjee et al. argue that sensitivity to input phrasing is a distinct property from general intelligence. A model can be highly accurate yet remain fundamentally unstable. As Isonuma et al. demonstrate, this instability persists even throughout the pretraining process across all scales, suggesting that achieving true structural robustness will require architectural innovations beyond simply adding more layers.

Key Foundational Work

True Few-Shot Learning with Language Models (2021) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_model_scale_robustness_0ba8d43e,
  title = {model scale robustness},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/model-scale-robustness/rdr_0ba8d43e56f44f80815940279987b621},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_model_scale_robustness_0ba8d43e_v1,
  title = {model scale robustness},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/model-scale-robustness/rdr_0ba8d43e56f44f80815940279987b621/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake