Loss-Barrier Probes: Predicting Model-Merge Failure Before You Merge

Model merging — averaging or arithmetically combining the weights of several models that were fine-tuned from the same starting checkpoint — has become a standard way to build multi-task models for free, with no training data and no gradient steps. But it fails silently. Two independently fine-tuned models can occupy incompatible corners of parameter space, and worse, a checkpoint pulled from a public repository can be rigged: it may carry a backdoor that activates only after merging, or an anti-merging defense that deliberately scrambles its internal coordinates so that any unauthorized merge collapses. In every documented case, the failure is discovered only after performing the merge and running a full multi-benchmark evaluation, because a rigged checkpoint behaves perfectly on its own — one protected LLaMA-2 model keeps its 27.80% coding accuracy standalone and drops to 0.00% once merged.

This proposal argues that all of these failures share one geometric signature: a steep barrier in the loss landscape along the straight line between two checkpoints' weights. Theory says interpolation error is governed by the curvature of that path, and recent work on zero-cost proxies shows local curvature and gradient statistics can be estimated in minutes rather than GPU-days. We propose a training-free, label-free probe that walks a few steps along the interpolation path between a base model and a candidate checkpoint, measures how far the intermediate models' outputs drift on two unlabeled minibatches, and uses that drift to predict, before merging, whether the merge will work and whether the checkpoint has been obfuscated. The same probe then doubles as a continuous robustness metric for anti-merging defenses, replacing today's trial-and-error attack batteries, and as a design objective for choosing obfuscation keys that are provably hard to strip.

Core Claim

The claim has three parts. First, parameter obfuscation is invisible at the endpoints of any weight interpolation and catastrophic in the middle: transforms like MergeLock's $W_Q' = A^\top W_Q$, $W_K' = A^{-1} W_K$ leave the standalone model's outputs mathematically identical, but the interpolated weights $(1-t)\,\theta_{\text{base}} + t\,\theta_{\text{obfuscated}}$ no longer realize any benign function, so the drift is concentrated along the path. Second, ordinary merge incompatibility obeys the same geometry — mode connectivity theory proves that the error of linear interpolation between two minima is bounded by the maximum curvature of the connecting curve, and merging methods (task arithmetic, TIES, DARE, weight averaging) are all weighted or sparsified interpolations. Third, both facts are measurable without labels, without test benchmarks, and without performing the actual merge.

Concretely, for two checkpoints $\theta_a$ and $\theta_b$ and a small unlabeled batch $X$, we define a barrier score

$$B(\theta_a, \theta_b; X) \;=\; \max_{t \in \{0.1, 0.2, \ldots, 0.9\}} \; D_{\mathrm{KL}}\!\left( f_{\theta_t}(X) \,\Big\|\, f_{\theta_b}(X)\right), \qquad \theta_t = (1-t)\,\theta_a + t\,\theta_b,$$

where $f_\theta(X)$ is the model's predictive distribution and the KL divergence is averaged over the batch. Nine forward passes on two minibatches. The proposal is that $B$, together with a small family of companion statistics (gradient-magnitude moments and loss values at the same interpolated points), ranks merge outcomes and flags defended checkpoints — and that we can prove which companion statistics are safe to use, because an adversary who knows the audit can spoof some of them.

What Prior Work Does and Does Not Cover

Three lines of work meet at this gap without crossing it. The anti-merging defenses — MergeLock, which inserts cancelling invertible matrices into the attention paths of a Transformer; PaRaMS, which permutes feed-forward neurons and rescales attention heads; MergeBarrier, which adds orthogonal projections and a Learning-With-Errors-style reparameterization; and TRAP², which trains a sharp peak into the loss landscape at weight scale $s=1$ — all deliberately manufacture loss barriers, yet all evaluate their defenses empirically, by attempting merges under known algorithms and attacks MergeLock, PaRaMS, MergeBarrier, TRAP². None measures the barrier it creates. TRAP² comes closest in spirit — it observes that an $N$-way merge roughly rescales each model's contribution by $1/N$ and uses scalar rescaling as a cheap stand-in for merging during training — but it is a training-time defense, not a post-hoc diagnostic, and it offers no way to inspect a third-party checkpoint you did not train yourself.

The zero-cost proxy literature predicts performance without training: ZiCo scores architectures by the inverse coefficient of variation of gradient magnitudes across layers ZiCo; TEG-NAS disentangles expressivity, trainability, and generalization via ReLU-region counts and Neural Tangent Kernel conditioning TEG-NAS; and Proxy R predicts adversarial vulnerability from the NTK's minimum eigenvalue plus a finite-difference Hessian lookahead, achieving a 20x–500x speedup over adversarial evaluation Proxy R. All three operate at random initialization on input-space perturbations or architecture search spaces; none touches weight-space interpolation between fine-tuned checkpoints. The theory we need for the transfer exists but is unexploited: the mode-connectivity analysis proves interpolation error is bounded by path curvature $\kappa_{\max}$ and explains why skip connections smooth mode connectivity Understanding Mode Connectivity via Parameter Space Symmetry, and the LoRA gauge-symmetry work confirms empirically that independently trained adapters sit in different coordinate systems whose mismatch produces exactly $r^2$ near-zero Hessian eigenvalues and artificial loss barriers The GL(r) Gauge Symmetry of LoRA. That last paper uses its insight to re-align adapters so merging works; we use the same geometry in the opposite direction, to detect misalignment and predict failure.

Finally, the threat models motivate the audit setting but leave it unsolved: backdoored checkpoints pass individual audits and activate only under merging BadMerging, Merge Hijacking, and even individually benign components can synthesize a backdoor post-merge From Purity to Peril. We cite these as motivation for why pre-merge screening matters, and we will be explicit about where our probe cannot help with them.

Research Program

The program has three components: an invariance theory, the probe itself, and its applications.

The theory component answers a question no paper has addressed: which training-free statistics survive obfuscation? The answer follows from linear algebra. Any functional-equivalence transform $W' = A^\top W$ rescales gradients as $\nabla_{W'} = A\,\nabla_W$, so parameter-space statistics like ZiCo's per-layer mean-to-std ratio of gradient magnitudes change under a non-orthogonal $A$ even though the model's input-output behavior is untouched. Function-space statistics — output distributions, kernel values on input pairs — are invariant. This yields a clean classification: parameter-space proxies are spoofable and non-invariant; function-space statistics evaluated at interpolated weights are the only quantities that both survive obfuscation and expose it, because obfuscation corrupts exactly the interpolated models, not the endpoints. We will formalize this as a small group-action analysis over the transform families used by MergeLock, PaRaMS, and the LoRA gauge group $(BX, X^{-1}A)$, and verify it empirically.

The method component builds the probe: the barrier score $B$ above, evaluated on paths from the base model to each candidate checkpoint, between candidate pairs, and along the rescaling directions $s \in \{0.8, 1.2\}$ that TRAP² identified as a merge surrogate. The probe requires no labeled data — for language models the companion loss can be plain next-token-prediction loss on generic unlabeled text; for vision Transformers we rely on the output-divergence form, which needs no labels at all. We will characterize how many path points and how much unlabeled data the score needs to stabilize, with ZiCo's two-minibatch budget as the reference standard.

The applications component closes the loop in both directions. As an audit tool, the probe screens repository checkpoints before merging: predict post-merge degradation, flag defended models. As an evaluation tool, it gives MergeLock-, PaRaMS-, and MergeBarrier-style defenses a continuous robustness measure — barrier depth and width — in place of non-exhaustive empirical attack batteries. As a design tool, it lets an owner sample candidate obfuscation keys and keep the one with the highest measured barrier per unit of standalone cost, replacing the fixed random draws used today.

Baselines

The naive reference point is "merge and hope": perform the merge with no screening and accept the outcome. This is the operating condition the literature quantifies — unprotected merges of fine-tuned ViT models on standard task suites work well, while inserting a protected model degrades merged accuracy by over 95% in MergeLock's experiments and by more than 65% under PaRaMS MergeLock, PaRaMS. Any screening method must beat this reference by catching failures before the merge is deployed, at a cost the reference point does not pay, so the comparison metric is detection quality per unit of compute.

The strongest cheap competitor is truncated validation: take the candidate checkpoint (or a trial merge) and evaluate it on one or two batches of data. This baseline is genuinely threatening because it uses the same two-batch budget as our probe, and we will run it under exactly that budget — same batches, same forward-pass count, same tuning effort — in both a labeled variant (task accuracy on a held-out batch, where labels exist) and an unlabeled variant (next-token loss on generic text). Beating it means two things: higher rank correlation with true post-merge accuracy across merge algorithms, and detecting defended checkpoints that truncated validation provably misses, since defenses are explicitly built to preserve standalone performance — MergeBarrier's protected model retains its 27.80% MBPP accuracy while merging collapses it to 0.00% MergeBarrier, so standalone validation should sit near chance as a detector while our path probe should not.

The strongest existing-method baselines are the zero-cost proxies transplanted into this setting: ZiCo, Synflow, and plain parameter counting, plus Proxy R. Their home-turf numbers set the bar for what counts as a useful proxy: on NATSBench-TSS, ZiCo reaches Kendall's $\tau = 0.61$ versus 0.54 for Synflow and 0.57 for parameter counting ZiCo, and Proxy R reaches $\tau = 0.33$ on Tiny-RobustBench in its own robustness-prediction domain Proxy R. We will recompute all of them on the merge-prediction task with the same compute budget as the probe. We will also run the $\tau J_p$ interference metric — the product of the task vector and the pre-trained model's Jacobian, which tracks inter-task interference in task arithmetic — using its official implementation tau-Jp Task Arithmetic, since it is the closest existing predictor of merging interference, though it is designed as a training-time regularizer and requires per-task Jacobian computation rather than being a post-hoc, label-free audit.

The key ablation isolates path evaluation. We run the probe's statistics in two configurations: computed at the endpoints only (equivalent to standard zero-cost proxies on fine-tuned weights) versus computed at interpolated weights $\theta_t$. The endpoint-only configuration is exactly the spoofable, non-invariant regime our theory predicts will fail under obfuscation; if the interpolated configuration does not substantially outperform it on defended-checkpoint detection, the core mechanism is wrong and the proposal should be abandoned. A second ablation varies the probe direction — full base-to-candidate path versus the scalar rescaling direction alone — to test whether TRAP²'s $1/N$-rescaling insight is sufficient on its own.

Why Not Just…?

Why not just evaluate the merged model on a few validation batches? This is the most serious alternative, and for the narrow question "will these two specific models merge," with labels available, it may well suffice — which is why we run it as the primary baseline rather than arguing past it. It falls short in three ways the experiments will quantify. It requires performing the merge first and having labeled multi-task data, which merging platforms frequently lack for exactly the capabilities being combined. It does not scale: screening $M$ candidate checkpoints for $K$-way combinations is combinatorial, while per-checkpoint path probes against a fixed base cost one pass each. And it is precisely the test that rigged checkpoints are engineered to pass — backdoored models perform normally on standard validation batches and activate only under specific triggers BadMerging, and defended models retain standalone utility by construction MergeBarrier. If truncated validation matches our probe's rank correlation and detection rate at equal budget, the honest conclusion is that the simple baseline wins and the contribution narrows to the theory and the label-free, pre-merge setting.

Why not just use existing zero-cost proxies like ZiCo or Synflow? They are architecture rankers, and in merging the architecture is constant — every candidate, the base, and the merged model have identical parameter counts, so parameter counting is uninformative by construction. The deeper problem is parameterization dependence: as shown above, obfuscation transforms rescale gradients non-uniformly, so a model and its obfuscated twin receive different ZiCo scores despite being functionally identical. The literature already documents that standard proxies struggle even in their home domain — Synflow's $\tau = 0.54$ on NATSBench-TSS is worse than naive parameter counting at 0.57 ZiCo. We will still run them empirically rather than assume this, because their failure in our setting is a prediction, not a measured fact.

Why not just re-align the weights before merging? Git Re-Basin-style permutation matching or the Kabsch algorithm can undo discrete neuron permutations, and PaRaMS admits partial recovery this way PaRaMS. But this is the attacker's move, not the auditor's: re-alignment needs optimization machinery and often calibration data, and it fails against continuous transforms — MergeLock reports that Kabsch-based re-alignment recovers only about 5% of original performance MergeLock. Our probe takes the opposite stance: it does not try to undo the transform, only to detect and quantify its consequence, which is cheaper and works against transform families that re-alignment cannot crack. We will nonetheless use re-basin tooling from the TransFusion codebase aimagelab/TransFusion both to build matched fine-tune suites and as an attack in the defense-robustness experiments.

Why not just use the $\tau J_p$ interference metric? It is the closest existing predictor of task interference in model addition, and we run it as a baseline tau-Jp Task Arithmetic. But it requires computing the pre-trained model's Jacobian with respect to weights on task data, which presumes access to the very labeled task distributions a pre-merge audit often lacks, and it measures interference between task vectors rather than the geometric barrier a defense manufactures. If it matches our probe under equal compute and data budgets in the settings where its inputs are available, we will report that plainly and scope our claim to the label-free and obfuscation-detection settings where it cannot be applied.

Proposed Experiments

The vision experiments use the standard merging suite: ViT-B/32 and CLIP-ViT backbones fine-tuned on the eight tasks covered by the TransFusion codebase — CIFAR-100, SUN397, EuroSAT, DTD, SVHN, GTSRB, RESISC45, and ImageNet-R aimagelab/TransFusion, with EuroSAT itself a documented benchmark at 98.57% accuracy for a fine-tuned ResNet-50 EuroSAT. We build all pairwise and up to 8-way merges under task arithmetic, TIES, DARE, and weight averaging — the four algorithms PaRaMS tested — and measure true post-merge per-task accuracy as ground truth. The headline prediction result is Kendall's $\tau$ between the probe score and actual post-merge accuracy drop, with the target set by ZiCo's $\tau = 0.61$ in its own domain: a merge-prediction proxy that reaches $\tau \geq 0.6$ across algorithms at two-batch cost would be the first result of its kind — no reported result found on Lacuna for training-free prediction of post-merge performance, so there is no existing number to beat, only the transplanted baselines we run ourselves.

The defense-detection experiments apply MergeLock, PaRaMS, MergeBarrier, and TRAP² to a subset of fine-tunes and ask the probe to separate defended from undefended checkpoints by AUROC, with truncated validation as the comparison. The expected pattern is stark: defended checkpoints retain standalone accuracy (TRAP² reports protected ViT-B/32 models at an average 88.1%, identical to unprotected fine-tuning TRAP²) while their interpolation paths should produce barrier scores orders of magnitude above undefended ones. The defense-robustness experiments then use the probe as the metric: we measure barrier depth and width for each defense, test whether probe depth predicts resistance to the Kabsch and re-basin attacks each defense paper reports, and run the probe-guided key selection experiment, comparing probe-maximized obfuscation keys against the defenses' default random draws under the same attack battery.

The language-model experiments scale the probe to LLaMA-2-13B code fine-tunes, anchored on MergeBarrier's reported numbers — 27.80% standalone MBPP accuracy collapsing to 0.00% after merging MergeBarrier — and to TRAP²'s Llama-3.1 setting on GSM8K and ASDiv, where merges emit degenerate gibberish rather than degrading gracefully TRAP². The probe here uses next-token loss on generic unlabeled text along the interpolation path, and the cost accounting is explicit: Proxy R demonstrated that curvature-based screening can run in 0.017 GPU days where full evaluation takes roughly one Proxy R; we will report the probe's wall-clock cost against full multi-benchmark evaluation of the same merges, with a target below 0.05 GPU days per checkpoint.

Two decisive checks conclude the program. The first is the adaptive-attacker check: we ask a red team to build a checkpoint that sabotages merges while keeping the interpolation path to the base model benign — essentially inverting the probe into a training objective, in the spirit of TRAP²'s off-nominal loss terms. We expect this to be feasible for backdoor-style attacks, whose trigger vectors are trained to occupy a sparse subspace orthogonal to task gradients BadMerging and which our path probe is not designed to see; we state this limit up front rather than claim it away, and the check will draw the exact boundary between what the probe detects (basin displacement, incompatibility, obfuscation) and what it cannot (orthogonally hidden trigger subspaces). The second is the falsification check from the ablation: if endpoint-only statistics match interpolated statistics on defended-checkpoint detection, or if truncated validation matches the probe at equal budget on merge-outcome prediction, the core mechanism or its practical value is refuted, and the results section will say so.