Introduction
In the current "pretrain-finetune" era, the open-source community has flourished as developers release specialized, high-performance models on platforms like Hugging Face and GitHub. However, this openness has introduced a new security challenge: unauthorized model merging. "Free-riders" can download these fine-tuned models and use low-cost, training-free merging techniques to combine multiple capabilities into a single model, often bypassing licensing agreements and intellectual property protections.
Existing defenses are limited because a defender cannot predict which model an attacker will choose for a merge, what strategy they will use, or the specific target task. To address this, we introduce MergeGuard, a proactive defense framework that makes models "unmergeable" while maintaining their original utility for legitimate users.
Methodology: Dual-Stage Weight Protection
MergeGuard operates through a two-stage process designed to reshape a model's parameter geometry, ensuring that any attempt to merge it results in destructive interference.
Stage 1: Density-Aware Finetuning
Standard model merging relies on the observation that task-specific information is typically concentrated in a few high-magnitude weights. MergeGuard disrupts this by using $L_2$-regularized optimization to "smear" task-critical information across a much wider set of parameters. This redistribution makes the model's functionality dependent on a dispersed set of weights, which are far more susceptible to dilution or disruption during the merging process.
Stage 2: Adversarial Weight Negation
Building on the dispersed structure from Stage 1, the framework identifies "critical layers" and injects structured perturbations into the task vector (the difference between fine-tuned and pretrained weights). This effectively "rotates" the model's task subspace. By pushing the model out of the "shared parameter basin" required by merging algorithms, MergeGuard ensures that merged models land in high-loss regions, leading to a performance collapse.
Experimental Evaluation
MergeGuard was evaluated across three diverse domains to demonstrate its robustness and versatility:
- Image Classification: Tested on eight datasets (e.g., SUN397, GTSRB, EuroSAT), where it reduced merged model accuracy by up to 90%.
- Large Language Models (LLMs): Evaluated using AlpacaEval, GSM8K (math), and HumanEval (coding). The defense caused massive performance drops in merged models while keeping the protected standalone model's utility loss under 1.5%.
- Image Generation: Using Stable Diffusion 1.5, the defense caused merged models to produce semantically incoherent outputs, effectively protecting specific artistic styles.
Key Insights
- Curvature Misalignment: Successful merging depends on models residing in a shared linear subspace. MergeGuard works by pushing the model into a disjoint basin. Even a small rotation (>30°) in the task vector is sufficient to trigger a collapse in the merged model's utility.
- Resilience: MergeGuard proved robust against "adaptive attacks" like Unmask and GradErase, where an attacker is aware of the defense. It maintained a degradation margin of at least 20% even under these adversarial conditions.
Conclusion
MergeGuard provides a powerful, proactive solution for protecting the intellectual property of fine-tuned models. By intentionally misaligning parameter subspaces through a dual-stage approach, it acts as a "poison pill" for unauthorized merging while remaining nearly transparent to legitimate users. Future work may involve integrating these geometric protections with watermarking to further enhance model ownership verification.
Summary updated