Fine-tuning a large pre-trained language model on a few hundred labeled examples tends to damage the general-purpose features it learned in pre-training. One well-studied fix is a two-stage recipe. First you train only a small linear classifier on top of the frozen model, which is called linear probing. Then you fine-tune everything. The explanation for why this works is that a classifier that starts out large lets the model fit the new task by adjusting its outputs rather than by rewriting its internal features. We point out that the text-to-text formulation popularized by T5 already contains such a classifier. T5 casts every task as generating an answer string, such as the word "entailment" or "positive". The model's pre-trained output embeddings for those answer words act as a classification head that is already trained, meaningful, and sizeable before fine-tuning starts. If that is right, text-to-text fine-tuning is a one-stage version of the probe-then-fine-tune recipe. It should protect features for the same reason, and it should have the same known side effect of overconfident predictions.
The question matters because the regime in which feature damage is worst is a highly over-parameterized model with a tiny dataset, where the model has far more trainable weights than training examples. That is exactly where T5 was observed to memorize small tasks. It is also exactly where no one has measured what happens inside an encoder-decoder model during fine-tuning. We propose to test the claim directly. We will measure, with frozen linear probes placed across the encoder and decoder, how much task-general information survives fine-tuning under different output formulations, head norms, and model sizes. We will then check whether the norm of the answer-word embeddings is a usable dial for trading feature preservation against calibration.
Core Claim
Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective analyzes the two-stage recipe (LP-FT) for RoBERTa. It uses the neural tangent kernel, a linear approximation of how a network's outputs change during training. In that analysis, the norm of the classification head governs how gradient updates are split between two routes: moving the logits while leaving features alone, and rewriting the features themselves. With a large head, most of the fit comes from the first route. On the small SuperGLUE CommitmentBank (CB) task, LP-FT's head norm was about 24.8 versus about 0.96 for ordinary fine-tuning. Representation cosine similarity to the pre-trained features was 0.804 versus 0.336. That study covers only encoder models with freshly initialized heads.
In a text-to-text model, the logit for answer word $c$ at the answer position is
$$z_c = e_c^\top h(x),$$
where $e_c$ is the pre-trained output embedding of word $c$ and $h(x)$ is the decoder's final hidden state for input $x$. Structurally, $e_c$ plays the role of the head in the NTK analysis. It is not initialized near zero; it arrives with pre-trained scale and direction.
Our claim has three parts. First, text-to-text fine-tuning with natural-language answer words should distort encoder features less than fine-tuning a fresh classification head on the same encoder. Second, the effect should be driven mainly by the scale of $e_c$ relative to a fresh head, not only by the meaning of the answer words. Third, the effect should change with model size in a specific way. The decoder is itself a large nonlinear head, so as models grow it may absorb adaptation that would otherwise land in the encoder. If so, encoder distortion would fall with scale while memorization moves into the decoder.
Each of these can be false. Each is checked by probes, not by downstream accuracy alone.
Research Program
The central instrument is a layer-wise probing sweep. We fine-tune a model on a small source task. We then freeze it and train logistic-regression probes, meaning single linear layers, on pooled states from every encoder layer and on decoder states at the answer position. The probes are trained for held-out tasks the model was not fine-tuned on. The drop in probe accuracy relative to the pre-trained checkpoint measures lost general information. That drop is a more meaningful quantity than raw cosine similarity, although we report both for comparability with the LP-FT paper.
We keep probes linear because Measuring Memorization Effect in Word-Level Neural Networks Probing showed that multi-layer perceptron probes memorize lexical identity far more than linear ones. Their seen-versus-unseen gap reached 24.1 points for static embeddings and fell to 0.0 for contextual states with linear probes at scale. That result constrains our design: it rules out nonlinear probes as the primary measure. We reuse its seen-versus-unseen split idea to measure memorization at the decoder answer position, by comparing probe accuracy on training inputs against held-out inputs. Implicit Representations of Meaning in Neural Language Models supplies the practical method for locating and probing token states inside T5 and BART. We start from its released code, belindal/state-probes. That paper studied frozen pre-trained models in synthetic worlds, whereas we study how fine-tuning reshapes those states.
The manipulations are the output formulation, the answer-embedding norm, and model size. For the output formulation, we compare natural answer words; arbitrary existing words such as "apple" and "zebra"; freshly initialized new answer tokens; and a conventional classification head on encoder mean-pooled states. For the answer-embedding norm, we rescale the rows $e_c$ by factors from 0.1 to 10 before fine-tuning. The other rows are left unchanged, and the answer-word logits are temperature-corrected at evaluation. For model size, we use T5 Small through XL, and 11B if compute allows, which brings in over-parameterization. Across these variants the data, optimizer, and step budget are identical.
Beyond the diagnosis, we test one practical consequence. If the norm of $e_c$ is the operative quantity, then scaling the answer-word embeddings up once before ordinary fine-tuning should reproduce LP-FT's feature preservation without a separate probing stage. We call this answer-norm scaling. The price should be the same calibration loss LP-FT shows, which the LP-FT paper found temperature scaling repairs: maximum calibration error on RTE fell from 63.95 to 13.94.
Baselines
The naive reference point is the frozen pre-trained model with a linear probe per task, with no fine-tuning at all. It sets the upper bound on preserved information and the floor on in-task accuracy.
The second baseline is standard T5 text-to-text fine-tuning as in Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. This is the default practice, and at 11B it reported a SuperGLUE score of 88.9. We will not reach that number at smaller scales and do not need to. The comparison is internal and uses matched model sizes.
The strongest competitor is LP-FT applied to the same T5 encoder with a fresh classification head. The first stage trains the head on frozen features to convergence, and the second stage fine-tunes everything under the same step budget and learning-rate sweep as the other arms. It is the method with direct evidence of preserving features: on RoBERTa-base it reached 84.64% on CB versus 81.43% for plain fine-tuning Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective. We found no reported result on Lacuna for LP-FT on T5 or on any encoder-decoder model. We will therefore run it ourselves, giving it the same data, total gradient steps including the probing stage, and hyperparameter grid as every other arm.
Answer-norm scaling beats LP-FT if it matches LP-FT's out-of-distribution accuracy and held-out-probe retention within seed noise while using one training stage. It is a clear win if it exceeds LP-FT on in-task accuracy at the same time.
The key ablation is the fresh-token arm. It keeps the text-to-text decoder but replaces the pre-trained answer embeddings with newly initialized ones at a small norm. If the claim is right, this arm should distort features about as much as a fresh classification head. The arbitrary-word arm separates the norm of $e_c$ from its meaning.
Why not just…?
Why not just use LP-FT on T5 and skip the analysis? That is a real option, and it is one of our arms. But LP-FT adds a stage and a new head, and it discards T5's single-format interface. If text-to-text fine-tuning already delivers most of LP-FT's protection, the extra stage buys nothing. If it does not, we will see that directly as LP-FT beating the text-to-text arm on held-out probes. Either outcome is informative, and neither is known today.
Why not just measure cosine similarity between pre-trained and fine-tuned features, without probes? Cosine similarity is cheap and we report it. However, a representation can rotate while keeping its linearly readable content, or stay close in angle while losing a small task-relevant direction. Held-out-task probe accuracy measures the thing we care about, which is whether other tasks can still be read out. Where the two metrics disagree, the probe result is decisive.
Why not just regularize with weight decay, higher dropout, or loss flooding? Exploring Robust Overfitting for Pre-trained Language Models found that weight decay did not stop the rise in robust test loss and that dropout up to 0.9 cost clean accuracy. Flooding, meaning keeping training loss above a threshold $b$, helped; for example, it raised TextFooler robust accuracy on SST-2 from 8.94% to 11.93%. Flooding is a genuine competitor for the memorization side of our question, so we include it as an arm at the standard size. It limits how far the loss falls, but it does not control which parameters absorb the fit. Our probes will show whether it preserves encoder information or only slows interpolation.
Why not just fix overconfidence with post-hoc calibration? Enhancing In-context Learning via Linear Probe Calibration shows that an affine map on output probabilities fixes calibration from a handful of labels, and temperature scaling repairs LP-FT's calibration. We will apply temperature scaling to every arm. Calibration is not what we are trying to explain, though. Output-side calibration leaves internal feature loss untouched, and that loss is what the out-of-distribution evaluations test.
Proposed Experiments
Source tasks are the smallest SuperGLUE tasks, CB (250 training examples), RTE, WiC, and BoolQ SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems, because that is where over-parameterization is most extreme. For distribution shift, we fine-tune on the in-domain split of BOSS and evaluate on its shifted domains (Dynasent, SemEval, SST-5), following the LP-FT paper's out-of-distribution protocol. We found no reported result on Lacuna for T5 on BOSS. As a secondary robustness check, RTE and SST-2 models are evaluated on AdvGLUE Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models. There, encoders collapse sharply; for example, ELECTRA-Large fell from 93.16 to 41.69, and no generative model was benchmarked. Held-out probe tasks are drawn from GLUE (SST-2, MRPC, CoLA) GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. Metrics are in-task accuracy, out-of-distribution accuracy, per-layer held-out probe retention, representation cosine similarity, maximum and expected calibration error before and after temperature scaling, and the train-versus-held-out probe gap at the decoder answer position. All results are reported over five seeds.
Three results would settle the claim. First, if natural-word text-to-text fine-tuning preserves held-out probe accuracy close to LP-FT while the fresh-token arm falls to fresh-head levels, the pre-trained answer embeddings are doing LP-FT's work. Second, if held-out retention and calibration error both rise monotonically as $e_c$ is rescaled, with arbitrary words tracking natural words at equal norm, then norm rather than meaning is the mechanism. If instead natural words beat norm-matched arbitrary words, semantic grounding matters independently and the NTK story is incomplete. Third, across T5 sizes we test whether encoder distortion shrinks while the decoder memorization gap grows. That pattern would show that in text-to-text models, over-parameterization moves adaptation into the decoder rather than preventing it, and it would locate where the memorization Raffel et al. observed actually lives.
References
- Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective
- Measuring Memorization Effect in Word-Level Neural Networks Probing
- Implicit Representations of Meaning in Neural Language Models
- Exploring the limits of transfer learning with a unified text-to-text transformer
- Exploring Robust Overfitting for Pre-trained Language Models
- Enhancing In-context Learning via Linear Probe Calibration
- SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems.
- Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
- GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding