When a foundation model undergoes pre-training, it maps out a vast, high-dimensional landscape of general knowledge. A large language model learns syntax, reasoning, and world facts; a vision-language model learns universal geometric shapes, object boundaries, and semantic relationships.
However, when we fine-tune this model on a narrow downstream task—such as adapting a machine translation model to a specific technical domain or personalizing a diffusion model to a new subject—we alter the optimization objective. Standard gradient descent aggressively updates the model's weights to minimize loss on this new, specialized dataset. In doing so, it often takes a destructive path, overwriting the delicate parameter configurations that encoded the model's broad, zero-shot capabilities.
This phenomenon is known as catastrophic forgetting or representation degradation. To understand why it happens and how to prevent it, we must look at the geometry of model adaptation.
The Geometry of Forgetting
Why does full-parameter fine-tuning erase a model's foundational capabilities?
During pre-training, the model's parameters settle into a broad, deep valley of the loss landscape that represents generalized understanding. When we introduce a specialized target dataset, we introduce a completely new loss landscape with its own steep, narrow valley. As Thompson et al. observe in neural machine translation, unconstrained optimization forces parameters to shift rapidly away from the pre-trained manifold to satisfy the narrow downstream objective.
Pre-trained Manifold (Broad Valley) ───► Task-Specific Fine-Tuning (Steep Shift)
[ General Knowledge ] [ Overwritten Weights ]
(Syntax, Geometry, Common Sense) (Catastrophic Forgetting)
This weight shift causes what Saxena terms representation drift—a degradation where the model loses its underlying semantic anchors. As Jiang et al. point out, because the weight updates deviate significantly from the base model's learned representations, the model's universal priors collapse. For instance, fully fine-tuning a 2D diffusion model for 3D generation tasks can erase its core visual common sense, leading to structural artifacts like generating characters with multiple faces, as shown by Ma et al..
To resolve this tension, researchers have developed methods to adapt models without destroying their foundational geometry. These strategies broadly fall into three categories: structural isolation, functional constraints, and targeted surgery.
Structural Isolation: Freezing the Core
The most intuitive way to prevent a model from forgetting its pre-trained knowledge is to make that knowledge physically immutable. Instead of modifying the original weights, we freeze the foundation model entirely and append lightweight, trainable sub-networks.
Parameter-Efficient Adapters
Bapna et al. pioneered this by injecting tiny "adapter" layers into frozen pre-trained models. Because the base parameters are locked, the model's core representations cannot drift. The adapter simply learns to rotate or scale the frozen features to fit the new domain. This concept scales remarkably well: Melnyk et al. demonstrated that a frozen English BERT model can be "reprogrammed" to design complex biological antibody sequences simply by training a tiny set of input/output adapter matrices.
Additive Side Networks
Rather than inserting layers inside the model, Jeffrey O. Zhang et al. proposed Side-Tuning, an additive approach where a separate, lightweight "side network" operates parallel to the frozen base model. The final output is a linear combination of the two networks:
$$\text{Output} = \alpha \cdot f_{\text{frozen}}(x) + (1 - \alpha) \cdot g_{\text{side}}(x)$$
This design guarantees that the model retains 100% of its original capabilities while the side network incrementally learns the nuances of the new task.
Functional Constraints: Anchoring the Gradients
If you must update the original parameters, you need a mechanism that penalizes the model if it drifts too far from its pre-trained state. This is achieved by adding regularization terms to the loss function or modifying the gradients directly.
Weight Consolidation and Memory Constraints
Borrowing from continual learning, Trinh et al. utilized Elastic Weight Consolidation (EWC) to protect critical parameters during speech recognition adaptation. EWC calculates a Fisher information matrix to identify which weights are most important to the original task, applying a selective penalty that allows less critical weights to change while locking down foundational ones. Similarly, Liu et al. used Gradient Episodic Memory (GEM) to explicitly constrain fine-tuning gradients, ensuring they never form an obtuse angle with the gradients of the pre-training objective.
Gradient Alignment
Standard regularization techniques like weight decay often fail because they don't account for semantic compatibility. To address this, Zhu et al. introduced Prompt-aligned Gradients. This method explicitly forces task-specific gradient updates to align with the model's original zero-shot directional vectors, ensuring that optimization for a narrow task remains mathematically compatible with broad semantic understanding.
Self-Distillation and Targeted Surgery
Two emerging paradigms offer even more precise control over representation degradation: using the model as its own teacher, and isolating updates to specific functional blocks.
Self-Distillation as an Anchor
When compressing or optimizing models, standard supervised fine-tuning often causes a severe drop in reasoning. Thangarasa et al. resolved this during model pruning by using self-data distillation. Here, the unpruned, original pre-trained model acts as a teacher, forcing the adapted model to match its original output probability distributions.
[ Unpruned Base Model (Teacher) ] ───► Outputs Soft Probabilities (Anchor)
│
▼
[ Adapted/Pruned Model (Student) ] ───► Matches Teacher to Prevent Collapse
As Kujanpää et al. demonstrate, this distributional anchoring allows models to internalize new factual knowledge without suffering the catastrophic collapse in instruction-following and general reasoning that typically plagues standard fine-tuning.
Targeted Surgery
Instead of treating the model as a homogenous block, recent work focuses on localization. Zihao Lin demonstrated that specific types of knowledge (such as artistic styles or structural priors) are localized within distinct blocks of Diffusion Transformers. By performing targeted surgery—fine-tuning only the specific blocks responsible for the concept of interest—we can achieve rapid adaptation while leaving the rest of the network's global parameters entirely untouched.