Abstract
Large Language Models (LLMs) often exhibit "alignment brittleness," where fine-tuning for specific tasks or safety alignment can inadvertently degrade previously acquired capabilities. This research explores the geometric properties of LLM loss landscapes to explain this phenomenon. We identify the existence of expansive "capability basins"—regions where performance remains stable under random parameter perturbations—contrasted with sharp "cliffs" where performance collapses rapidly in specific directions. We propose the Gaussian-augmented Optimizer (GO) to enlarge these basins during pre-training, thereby improving model robustness against catastrophic forgetting.
Introduction
As Large Language Models (LLMs) scale, their behavior during fine-tuning becomes increasingly complex. A common issue is the "brittleness" of alignment: a model fine-tuned for a specific task may lose its safety guardrails or general reasoning abilities. This paper investigates the underlying geometry of the parameter space to explain why these shifts occur and how they can be mitigated.
The Geometry of Capability Basins
The authors analyze the loss landscape—the mapping of model parameters to performance loss—from two distinct perspectives: random perturbations ("most-case") and optimized adversarial perturbations ("worst-case").
Most-Case Landscape: The Basin
When parameters are perturbed in random directions, large models exhibit a flat-bottomed "basin." Within this region, the model's performance on various benchmarks (Safety, Math, Basic, Coding) remains remarkably stable. This basin structure is consistent across major architectures like Llama, Qwen, and Mistral.
The study finds that these basins emerge and expand with model scale. For instance, the 32B version of Qwen-2.5 possesses a capability basin nearly twice as large as the 7B version, suggesting that larger models are inherently more robust to the parameter shifts encountered during fine-tuning.
Worst-Case Landscape: The Cliff
In contrast to the broad basins found via random perturbations, the "worst-case" landscape—found by searching for the specific direction of maximum loss increase—reveals a sharp "cliff." Even a tiny step in this specific direction can lead to a total collapse of model capabilities.
This geometric duality explains why adversarial fine-tuning or "white-box" jailbreaks are so effective: while the model is stable in most directions, there almost always exists a high-dimensional "shortcut" out of the safety basin.
Enhancing Robustness with the GO Optimizer
To address the brittleness of these basins, the researchers introduced the Gaussian-augmented Optimizer (GO). During the pre-training phase, the GO optimizer adds Gaussian noise to the parameters in every forward pass. This technique forces the optimization process to find wider, more resilient basins rather than settling into narrow minima.
Experiments on a GPT2-127M model showed that models pre-trained with GO were significantly better at retaining their original knowledge (e.g., from OpenWebText) after being fine-tuned on new datasets like Alpaca, compared to models trained with standard AdamW.
Theoretical Framework: Randomized Smoothing
The paper leverages Randomized Smoothing—a technique from robust statistics—to provide a theoretical foundation for these observations. By treating the basin size (denoted as $\sigma$) as a measure of robustness, the authors derived a formula showing that the radius of a model's basin directly limits how much an adversary can degrade performance within a certain distance. This allows for a more rigorous prediction of how much a model's capabilities might shift during downstream fine-tuning.
Conclusion and Future Work
This work provides a new geometric lens for understanding LLM training. By identifying the existence of capability basins and the threat of performance cliffs, the authors offer both an explanation for alignment brittleness and a potential solution via the GO optimizer. Future research will focus on scaling these optimization techniques to larger models and establishing formal scaling laws to predict exactly how much more robust a model becomes with every billion parameters.
Summary updated