Model merging has emerged as a powerful paradigm for creating multi-task AI systems without the prohibitive cost of joint training. By mathematically combining the weights of specialized "expert" models—such as a LoRA fine-tuned for cat images and another for barn landscapes—researchers aim to create a single model that retains the capabilities of both.
However, a recurring problem persists: the merged model often performs significantly worse than its individual components. Recent research identifies the culprit as representation bias, a phenomenon where merging weights in the parameter space causes a destructive warping of the model’s internal feature distributions.
The Weight-Representation Gap
The fundamental intuition behind this research direction is that neural network weights are merely a means to an end. What actually matters for a model's performance are its representations—the internal activations and feature vectors it produces in response to an input.
When we merge two models using simple weight averaging, we assume that the relationship between weights and representations is linear. In reality, it is highly non-linear. As Enneng Yang and colleagues demonstrate in Representation Surgery for Multi-Task Model Merging, merging weights causes the resulting feature distributions to shift and become "noisy" compared to the original expert models. Even if the merged weights contain the necessary information, the internal feature space is so distorted that the model's decoders can no longer "read" the data correctly.
The Compounding Nature of Drift
A critical insight from SurgeryV2 is that representation bias is not a localized error. Instead, it is a systemic issue that accumulates as data flows through the network.
Imagine a signal passing through twenty layers. If the first layer introduces a 5% distortion, the second layer processes that distorted signal and adds its own error. By the time the data reaches the final layer, the representation bias has increased monotonically. This "feature drift" means that addressing the problem only at the output layer is insufficient; the internal data flow of the model has already deviated too far from the path the original experts intended.
From Parameter Space to Representation Space
To solve this, researchers are shifting their focus from optimizing weights (parameter space) to correcting activations (representation space). This shift is characterized by several key strategies:
- Representation Surgery: Enneng Yang proposes adding lightweight, task-specific "plugins" that act as corrective lenses. These plugins don't change the merged weights; instead, they intercept the distorted activations and warp them back toward the original expert distributions.
- Closed-Form Alignment: Rather than using slow, iterative optimization, Jialin Wu and Jian Yang introduced ReACT (From Parameter to Representation). This method treats model interference as a linear distortion that can be fixed with a fast, closed-form mathematical correction, aligning the merged model's preferences in milliseconds.
- Probabilistic Modeling: Recognizing that interference between models is often unpredictable, Qi Wei developed ProbSurgery (Representation Surgery in Model Merging with Probabilistic Modeling). This approach treats representation bias as a distribution rather than a fixed value, allowing the model to better handle the inherent uncertainty of multi-task interference.
Why It Works: Restoring Semantic Geometry
The success of these methods—such as OTMF (Merging without Forgetting)—relies on restoring the "semantic geometry" of the feature space. By using techniques like Optimal Transport to align how different models perceive the world, researchers can merge knowledge while preventing the "catastrophic forgetting" that occurs when internal features drift too far.
By treating the merged model as a "broken" representation generator that needs calibration, rather than a collection of weights that needs re-training, this field is closing the performance gap between cheaply merged models and expensive, jointly-trained multi-task systems.