Choosing Source Data by the Prompt Stability It Transfers

Language models trained for a niche domain often have only a handful of labeled examples to learn from. The standard remedy is to borrow labeled data from richer domains. A second, mostly separate problem is that the resulting models are brittle: rewording the instruction, reordering answer options, or adding a trailing "thank you" can flip their answers. This proposal connects the two problems through one question. When we borrow data from another domain, which sources make the model's answers on the target domain stop depending on how the question is phrased? We propose to measure this directly and cheaply. A short adaptation run on a candidate source domain is checked by how much it reduces disagreement across instruction rewordings on unlabeled target inputs, with a guard that rejects runs where the model simply collapses to one answer. Source data is then chosen by this measured stability transfer instead of by embedding similarity or perplexity, both of which are known to mislead.

The opportunity matters because the field's current tools for choosing source data fail in exactly this setting. Embedding similarity between source and target is a poor predictor of transfer and often coincides with negative transfer (Is Prompt Transfer Always Effective?). Perplexity-based signals can be deceptive, for instance in structured domains like mathematics (Mixture-of-Skills). Picking prompts with cross-validation on 16 target examples carries roughly a 27% risk of choosing a prompt 13% worse than average (True Few-Shot Learning with Language Models). Disagreement across rewordings has two useful properties. It is observable without target labels, and it carries some information about correctness: On the Relation between Sensitivity and Accuracy in In-Context Learning reports correlations up to $r=-0.40$ between sensitivity and correctness.

Core Claim

Prompt stability is partly a transferable skill that can be separated from domain knowledge. Source domains differ substantially in how much of it they transfer. A label-free probe, defined as the reduction in target-side disagreement across instruction variants under a guard against collapse, predicts which sources help target accuracy under rewording better than embedding similarity or perplexity drop. Training on the sources this probe selects, with a consistency loss, should give higher worst-case accuracy across prompt variants in the target domain. The comparison points are uniform source mixing and consistency training on unlabeled target data alone.

The concrete score for a candidate source domain $s$ works as follows. Let $\theta_0$ be the base model and $\theta_s$ the model after a short adaptation on $s$. Let $S_t(\theta)$ be the average disagreement rate across $k$ held-out instruction variants on unlabeled target inputs. Let $H_t(\theta)$ be the entropy of the model's predicted-label distribution over those inputs. The score is

$$\mathrm{score}(s) = \big[S_t(\theta_0) - S_t(\theta_s)\big] - \lambda \max\big(0,\ H_t(\theta_0) - H_t(\theta_s)\big),$$

where $\lambda$ penalizes runs that buy consistency by predicting one class for everything. This is the majority-class collapse that Prompt Consistency for Zero-Shot Task Generalization guards against with a Fleiss' Kappa stopping rule.

How This Differs From the Closest Work

The work closest in spirit falls into two groups, and neither does what is proposed here.

The first group is consistency training, which teaches a model to give the same answer under perturbed prompts. Prompt Perturbation Consistency Learning (PPCL) penalizes the divergence between predictions on clean and perturbed inputs within one task. Swarm Distillation (Prompt Consistency for Zero-Shot Task Generalization) enforces agreement across synonymous prompts using unlabeled data. Both supply our training loss. Neither asks which out-of-domain data to apply that loss to.

The second group is data selection and domain adaptation. Mixture-of-Skills learns sampling weights from embedding transferability and perplexity drop. Prompt Gradient Alignment (PGA) aligns gradients across image domains for continuous prompts. DPOD learns domain-specific soft prompts from out-of-domain data for misinformation detection. Promise meta-learns soft prompts for CLIP that generalize across phrasings and visual shifts. These methods either ignore phrasing sensitivity or treat it only for continuous prompts in vision-language models. None uses measured stability on the target as the signal for choosing source data in text models.

One finding motivates the core worry. Is Prompt Transfer Always Effective? shows that transfer can cause catastrophic forgetting of accuracy. Nobody has measured whether stability is gained or lost in transfer.

Research Program

The work proceeds in three stages.

Stage 1: does stability transfer? We fine-tune an open-weight model of roughly 8B parameters (Llama-3-8B, matching Mixture-of-Skills) with low-rank adapters on one source domain at a time. Training uses supervised cross-entropy plus the PPCL-style consistency term over a training set of instruction paraphrases. We then measure three things on each target domain, all using a disjoint evaluation set of paraphrases and formatting perturbations: accuracy, worst-variant accuracy, and the brittleness share of variance from Brittlebench. The evaluation paraphrases are kept disjoint because stability on prompts the model has seen proves nothing; Promise's disjoint inner/outer prompt sets apply the same logic. This stage produces a source-by-target matrix of stability transfer. That matrix answers whether stability is forgotten, preserved, or gained across domains, and whether gains concentrate in formatting fragility or in semantic rewording.

Stage 2: does the label-free probe predict transfer? For each source–target pair, we compute the probe score from a short run, about 5% of the full training budget. We correlate it with realized target worst-variant accuracy and compare it against embedding cosine similarity and Mixture-of-Skills-style perplexity drop as predictors. This is the go/no-go point. The modest $r=-0.40$ link between sensitivity and correctness, and the "consistently wrong" errors seen on TruthfulQA in Rethinking Hallucinations, both warn that lower disagreement may not mean higher accuracy. If the probe does not beat the alternatives as a predictor, the selection method is dead, but the transfer matrix from Stage 1 still stands as a result.

Stage 3: train on the selected sources. We train the final model on the top-scoring sources (or on score-weighted sampling), together with the 16 labeled target examples, under the same total step budget as all baselines.

Benchmarks and Setting

The main testbed is MASSIVE, which supports intent classification and slot filling across 51 domains. It allows a clean held-out protocol: several domains are treated as low-resource targets with 16 labeled and a few hundred unlabeled utterances, and the rest serve as candidate sources. PPCL's perturbation types (synonyms, oronyms, paraphrases) can be reused directly.

The second testbed is MMLU, with subjects grouped into domains. Here perturbations follow When Benchmarks are Targets (option shuffling, rare option symbols) and Brittlebench's semantics-preserving rewrites.

Med-HALT serves as a genuinely specialized target, where Rethinking Hallucinations reports about 60% ambiguity for Llama2-13B-Chat. Here we measure ambiguity alongside accuracy.

Finally, the 16-dataset QA suite from Is Prompt Transfer Always Effective? gives a setting where negative transfer is already documented. RTE, with source data from MNLI, connects to the Swarm Distillation result.

Baselines

The naive reference is the base model prompted zero-shot and with 1-shot anchoring, evaluated across all evaluation variants. A second naive reference is low-rank adaptation on only the 16 target examples. Together these show how much there is to gain.

The strongest competitor on the label-free axis is Swarm Distillation applied to the unlabeled target inputs, using its Fleiss' Kappa stopping rule (Prompt Consistency for Zero-Shot Task Generalization). It attacks the same symptom with no source data at all. That paper reports RTE zero-shot accuracy rising from 64.6% to 75.2% for T0-3B, using unlabeled MNLI data. We will run it on the same model, the same unlabeled target pool, and the same training paraphrase set, with an equal step and tuning budget.

The strongest competitor on the robustness-training axis is PPCL applied to a uniform mixture of all source domains plus the target examples. That paper reports recovering on average 59% (intent classification) and 69% (slot filling) of perturbation-induced drops, and beating 50,000 augmented samples with 5,000. This baseline is also the ablation that isolates our key ingredient: it differs from our method only in that sources are not selected by the probe.

The strongest competitor on the data-selection axis is Mixture-of-Skills-style weighting by embedding transferability and perplexity drop (Mixture-of-Skills), with the same consistency loss. It reports a 2.45-point gain in its combined MMLU/MT-bench score over the strongest heuristic mixture on Llama-3-8B. A cosine-similarity selector is included as the simplest version of this approach. No reported result exists in the file for any of these methods on our held-out-domain protocol, so all of them will be rerun.

A collapse ablation drops the entropy guard ($\lambda=0$).

All methods get the same total gradient steps, the same adapter rank, and the same hyperparameter search size. The probe's compute is charged against our method. Beating the baselines means statistically significant gains in worst-variant target accuracy over both uniform-mixture PPCL and target-only Swarm Distillation, averaged across held-out MASSIVE target domains and repeated over seeds, without lower mean accuracy.

Why not just…?

Why not just run Swarm Distillation on the unlabeled target inputs and skip source data entirely? This is the most serious threat, because it uses the same unlabeled pool and needs no selection machinery. Our reason to expect it to fall short is that self-consistency without labels can converge on consistent but wrong answers. The method already needs Fleiss' Kappa to avoid collapse, and that criterion falters on some tasks such as WSC. Labeled source data provides an anchor that unlabeled target data cannot. If target-only Swarm Distillation matches our method on worst-variant accuracy, the honest conclusion is that source data adds nothing to stability. We would then report that and keep only the Stage 1 transfer analysis.

Why not just pool all source data with a consistency loss? Pooling is cheap and may be good enough. Against it, negative transfer is documented in Is Prompt Transfer Always Effective?, and Mixture-of-Skills shows static mixtures underperform adaptive ones. The uniform-mixture PPCL baseline tests this directly. If it ties, selection is unnecessary.

Why not just average answers over many prompt rewordings at inference time? Majority voting over $k$ paraphrases removes much of the sensitivity with no training. Its costs are $k$ times the inference compute and a dependence on paraphrases that cover the user's phrasing. We will include a $k$-way voting baseline. If it matches our trained model, our contribution reduces to cheaper inference, and we will state it that way.

Why not just use few-shot anchoring or chain-of-thought? One demonstration stabilizes output format (ProSA). However, Brittlebench finds that few-shot prompting increases sensitivity to formatting and demonstration choice, and that chain-of-thought cuts sensitivity-induced drops by only about 17%. Both are included in the naive reference evaluation.

Decisive Checks

Three results would settle the proposal. The first is the Stage 2 correlation: the probe must predict realized worst-variant target accuracy better than cosine similarity and perplexity drop on held-out MASSIVE domains. Otherwise we stop. The second is the head-to-head comparison against target-only Swarm Distillation and uniform-mixture PPCL at matched compute. The third is the collapse ablation. If removing the entropy guard leaves the probe's predictive power unchanged, the probe is not exploiting collapse. If it destroys that power, the guard is doing the essential work.

A secondary but publishable outcome holds regardless of the selection method's fate. It is the first measurement of whether prompt stability, rather than accuracy, is forgotten or preserved during cross-domain transfer, and whether domain distance predicts formatting brittleness differently from semantic brittleness.