Security and Backdoor Dynamics in Model Merging

This direction is merged into another direction: backdoor attacks, backdoor signal dilution. Its last version stays readable and citable here.

Version 1 of 122 papers39 conceptsfrozen 2026-10-04

Model merging has emerged as a powerful paradigm for creating multi-task LLMs without the prohibitive cost of full retraining. By mathematically combining the weights of specialized models or low-rank adapters (LoRAs), developers can "fuse" capabilities. However, this process introduces a unique security tension: the same mathematical operations that blend task-specific knowledge can either act as a natural filter for malicious behaviors or as a vehicle for stealthy "Trojan" attacks.

The Dilution Principle: Merging as a Defense

The most intuitive way to understand the security of model merging is through the lens of dilution. In a standard weight-averaging merge, the parameter values of multiple models are combined. If one model in the ensemble has been "poisoned" with a backdoor—a specific set of outlier weights that trigger a malicious response—those weights are often neutralized when averaged with the clean weights of other models.

Terry Tong and Muhao Chen demonstrate this effect in the "LLM-as-a-Judge" context. They found that while a single backdoored evaluator can be easily manipulated, merging it with clean models effectively dilutes the specific neural clusters responsible for the backdoor, restoring the model's integrity.

This principle is taken a step further in the OTBR (Optimal Transport-based Backdoor Repairing) framework by Weilin Lin and Hui Xiong. Their approach treats merging as a proactive repair tool: by fusing a backdoored model with a "sanitized" (but lower-performing) pruned version of itself, they use the clean structural information of the pruned model to overwrite malicious features while retaining the original model's accuracy.

Engineering Resilience: How Attacks Survive the Merge

If simple merging dilutes backdoors, how can an attacker ensure their "Trojan" survives? Recent research shows that adversaries are developing "merge-aware" attacks that specifically counteract the dilution effect.

  1. Rescaling and Amplification: Zenghui Yuan and Lichao Sun introduced "Merge Hijacking," where the malicious parameter vector is rescaled and amplified before merging. This ensures the backdoor signal remains dominant enough to trigger even after being fused with multiple benign models.
  2. Sign-Consistent Sparsification: Stanisław Pawlak and Jan Dubiński propose treating backdoors as "task vectors." To make these vectors resilient, they use a strategy that retains only the weights where multiple malicious updates agree on the sign. This creates a consolidated update that resists the "washing out" effect of parameter averaging.
  3. Over-Poisoning: In the context of LoRA "plugins," Tian Dong and Yan Meng show that by using an amplified trigger-loss during training, a malicious adapter can be fused with a benign one such that the benign weights provide general utility while the over-poisoned weights preserve the high-intensity trigger response.

The Interference Paradox

A counter-intuitive finding in this field is that better merging algorithms can lead to worse security.

Standard weight averaging often fails because of "parameter interference"—where different models have conflicting updates for the same weights. Advanced algorithms (like TIES or DARE) were designed to resolve this interference and preserve specialized capabilities. However, Zhenyuan Guo and Chengkun Wei demonstrate that these same techniques inadvertently protect malicious vectors. Because these algorithms are optimized to keep specialized task information from being "blurred" away, they are more effective at preserving a "phishing" model's ability to steal private data from the models it is merged with.

Proactive Defense and Weight Protection

As the risks of the merging supply chain become clearer, researchers are moving toward proactive defenses that happen during or before the merge.

  • Defense-Aware Merging (DAM): Jinluan Yang and Li Shen propose a dual-mask optimization strategy. Instead of merging blindly, DAM identifies a "safety-aware subspace" within the parameters, neutralizing malicious triggers during the fusion process itself.
  • MergeGuard: Rather than defending against a malicious model, Wei-Jia Chen and Cheng-Yi Lee focus on protecting the intellectual property of a benign model. Their "MergeGuard" framework intentionally misaligns parameter subspaces so that while the model works perfectly on its own, any attempt to merge it with another model causes a "performance collapse," effectively acting as a functional poison pill against unauthorized reuse.

By viewing model weights not just as static values but as vectors in a competitive subspace, this research area highlights that the future of model merging depends on balancing the desire for seamless integration with the need for robust parameter-level security.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_security_and_backdoor_dynamics_in_model__391a5c8b,
  title = {Security and Backdoor Dynamics in Model Merging},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/security-and-backdoor-dynamics-in-model-merging/rdr_391a5c8ba095441b92fc9ff2b899b4b6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_security_and_backdoor_dynamics_in_model__391a5c8b_v1,
  title = {Security and Backdoor Dynamics in Model Merging},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/security-and-backdoor-dynamics-in-model-merging/rdr_391a5c8ba095441b92fc9ff2b899b4b6/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}