Unveiling the Basin-Like Loss Landscape in Large Language Models

Huanran Chen, Zeming Wei, Yao Huang, Yichi Zhang, Yinpeng Dong, Jun Zhu

The AI community has long been puzzled by "alignment brittleness." Developers can spend millions of dollars aligning a Large Language Model (LLM) to be safe, helpful, and proficient in coding, only to have those capabilities vanish after just a few steps of fine-tuning on a new, seemingly benign dataset. Even worse, "adversarial fine-tuning" with just a handful of malicious examples can completely subvert a model's safety guardrails.

Recent research investigates why this happens by mapping the loss landscape—the mathematical "terrain" of a model's performance relative to its billions of parameters. By understanding the geometry of this terrain, researchers are uncovering the mechanisms behind catastrophic forgetting and discovering new ways to make models robust against jailbreaking.

The Geometry of Capability Basins

To visualize this landscape, researchers moved beyond standard "smooth" loss metrics (like Negative Log-Likelihood) and instead evaluated models using generative benchmarks (measuring 0-1 task success) across datasets like MMLU, GSM8K, HumanEval, and AdvBench.

This approach revealed that LLMs reside in "capability basins"—vast, flat regions in their parameter space where performance remains remarkably stable despite random perturbations.

Capability basins across Llama, Qwen, and Mistral models showing wide regions of low loss under random parameter perturbations.

Inside these basins, models exhibit "semantic stability." If you nudge the parameters in random directions, the model might change its phrasing or word choice (syntactic variation), but the final answer remains correct and safe.

Furthermore, these basins are an emergent property of scale. Small models (e.g., Qwen-0.5B) have messy, continuous loss landscapes. As models grow to 7B, 14B, and 32B parameters, clear basins emerge and expand. Larger models are literally "steadier" because they have more room to move in parameter space without falling off a performance cliff. For instance, in Qwen-2.5-7B, researchers established with 99% confidence that over 90% of all possible parameter directions form a strict basin at a perturbation scale of $\alpha = 0.01$.

The Adversarial Cliff

While the vast majority of directions in the parameter space are "safe" (representing the flat bottom of the basin), the landscape is not entirely benign. By comparing how models behave under random perturbations (most-case) versus the single most damaging direction (worst-case), researchers identified a critical vulnerability.

There always exists a "worst-case" direction that acts like an incredibly steep cliff. Adversarial fine-tuning exploits these specific, sharp directions to instantly push the model out of its capability basin, leading to rapid capability collapse.

The loss landscape in the worst-case direction, demonstrating a sharp cliff where performance rapidly degrades.

This geometric understanding provides a heuristic link between parameter basins and input-space robustness: if a model is situated in a wider parameter basin, it is mathematically more difficult for attackers to find "jailbreak" prompts that trigger unsafe behavior.

Smoothing the Landscape: The GO Optimizer

To actively build more resilient models, the researchers adapted "Randomized Smoothing"—a technique typically used to make image classifiers robust to pixel noise—and applied it to the model's weights. They proved theoretically that the size of the "most-case" basin provides a lower bound on performance degradation for any fine-tuning, including adversarial attacks.

Building on this, they proposed the GO Optimizer (Gaussian-augmented Optimizer). This new training approach injects Gaussian noise into the parameters during every forward pass, effectively forcing the model to find wider, flatter basins during pre-training. Models pre-trained with the GO optimizer suffered significantly less "catastrophic forgetting" of their original skills when fine-tuned on new tasks compared to models trained with standard AdamW.

Looking Ahead

While the theoretical bounds are promising, current experiments with the GO optimizer have been limited to smaller scales (e.g., 127M parameters). Future work will need to test these methods on 7B+ parameter models trained on massive datasets. Additionally, researchers aim to uncover a "scaling law" for basin size to predict exactly how much more robust a model becomes for every billion parameters added, paving the way for inherently safer and more stable AI systems.

Summary updated

Abstract

We discover the emergence of basins in the loss landscape of large language models. As model scale increases, LLMs become progressively more resilient to random perturbations in the parameter space, giving rise to expansive stability regions where models exhibit nearly identical performance, but outside of which their capabilities collapse. We observe that pre-training creates a basic capability basin, and subsequent alignment fine-tuning forms specific capability basins (e.g., safety, math, coding). Thus, we argue that benign fine-tuning confined to the basin should preserve prior capabilities. Besides, we also analyze the loss landscape for worst-case directions, which is consistently sharp and detrimental. We find that adversarial fine-tuning moves along the nearly worst-case directions, thus rapidly degrading model capabilities. Finally, we provide a theoretical analysis demonstrating that the basin size bounds the performance degradation of any fine-tuning, including the adversarial ones, while also guaranteeing the model robustness w.r.t. input perturbations, suggesting the benefit of enlarging basins.
BibTeX
@inproceedings{chen2026unveiling,
  author = {Huanran Chen and Zeming Wei and Yao Huang and Yichi Zhang and Yinpeng Dong and Jun Zhu},
  title = {{Unveiling the Basin-Like Loss Landscape in Large Language Models}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=l4q2Zk2yfk},
}
Figure