Abstract While there has been much recent work studying how linguistic information is encoded in pre-trained sentence representations, comparatively little is understood about how these models change when adapted to solve downstream tasks. Using a suite of analysis techniques (probing classifiers, Representational Similarity Analysis, and model ablations), we investigate how fine-tuning affects the representations of the BERT model. We find that while fine-tuning necessarily makes significant changes, it does not lead to catastrophic forgetting of linguistic phenomena. We instead find that fine-tuning primarily affects the top layers of BERT, but with noteworthy variation across tasks. In particular, dependency parsing reconfigures most of the model, whereas SQuAD and MNLI appear to involve much shallower processing. Finally, we also find that fine-tuning has a weaker effect on representations of out-of-domain sentences, suggesting room for improvement in model generalization.
Introduction
Fine-tuning BERT (Bidirectional Encoder Representations from Transformers) has become the standard paradigm for achieving state-of-the-art results in Natural Language Processing (NLP). Despite its success, the internal mechanics of how fine-tuning alters pre-trained representations remain largely opaque. Pre-trained BERT is known to encode a wealth of linguistic information, such as part-of-speech and syntax. A critical question arises: does the model "forget" this foundational knowledge to optimize for task-specific labels, or does it undergo a more surgical reconfiguration?
This study investigates whether fine-tuning leads to a total representation overhaul or a light polish of existing features. Understanding this process is vital for building robust models and explaining why they sometimes rely on shallow shortcuts rather than deep linguistic understanding.
Methodology
The researchers employed a "triangulation" approach, utilizing three distinct analysis techniques to compare the pre-trained Base model with its Fine-tuned counterparts:
- Supervised Probing: "Edge probes" were used to detect linguistic labels (entities, relations), and "structural probes" were used to reconstruct syntactic tree depths from frozen layers. High probe performance indicates the presence of specific information in the embeddings.
- Representational Similarity Analysis (RSA): This task-agnostic technique measures changes in the "distance" between sentence embeddings after fine-tuning, revealing broad shifts in the model's representational space.
- Model Ablations:
- Partial Freezing: Freezing the bottom $k$ layers to determine the minimum number of layers requiring updates for peak performance.
- Truncation: Physically removing top layers to assess task-specific information storage in lower levels.
The study evaluated three diverse benchmarks: MNLI (Natural Language Inference), SQuAD v1.1 (Question Answering), and Dependency Parsing (Dep). Wikipedia text served as an out-of-domain baseline.
Results: The Conservative Nature of Fine-tuning
The most significant finding is that fine-tuning is remarkably conservative. Linguistic features learned during pre-training are largely preserved, with probing F1 scores for fine-tuned models staying within 0.5% to 3% of the Base model.
Task-Specific Depth and the "Top-Heavy" Rule
Changes are not uniform across the model. The bottom layers of BERT remain relatively static, acting as a universal feature extractor, while the top layers undergo the most significant reconfiguration. However, the depth of this reconfiguration varies drastically by task.
As shown in the structural probing results above, dependency parsing reconfigures the model deeply, starting as early as layer 5. In contrast, MNLI and SQuAD primarily affect the final 2–3 layers. This is further supported by partial freezing experiments, where freezing the bottom 9 layers of BERT still yielded near-peak performance for MNLI and SQuAD, suggesting these tasks rely on "shallow" processing of pre-trained features.
Domain Sensitivity
Fine-tuning induces substantial changes in how the model represents sentences from the task's specific domain. Interestingly, when processing out-of-domain Wikipedia sentences, the fine-tuned model's representations remain highly similar to the original pre-trained Base model. This suggests that the model's specialized knowledge is somewhat localized to the target domain.
Conclusion and Future Work
While fine-tuning is conservative and preserves linguistic knowledge, there is a distinction between the presence of information and its utilization. Fine-tuned models often rely on heuristics (e.g., keyword overlap in SQuAD) rather than the deep syntax they possess.
The finding that fine-tuning is primarily a top-layer phenomenon suggests significant "room for improvement." Future research should explore methods to encourage models to better leverage their preserved linguistic knowledge, potentially improving generalization to out-of-domain data and reducing reliance on shallow shortcuts.
Summary updated