Probing Internal Representations of Multi-Word Verbs in Large Language Models

Hassane Kissane, Achim Schilling, Patrick KraussNAACL 2025

TL;DR

This paper investigates how Large Language Models (LLMs) like BERT represent multi-word verbs (phrasal and prepositional verbs). They found that middle layers of BERT are best at distinguishing these verb types, but linear separability tests (GDV) suggest the distinctions are encoded non-linearly, hinting at complex internal representations.

What Problem Does This Paper Address?

Multi-word verbs, such as "give up" (phrasal) and "rely on" (prepositional), are a fundamental part of English. Linguistically, they differ in how their components interact and their syntactic behavior. For instance, phrasal verbs can sometimes separate their verb and particle ("turn the light off"), while prepositional verbs maintain a fixed order ("look at the painting"). While previous linguistic studies have explored how these are stored in the human brain, it's unclear how sophisticated LLMs, which are increasingly used for language tasks, internally represent these nuanced linguistic structures. This paper aims to bridge this gap by probing BERT's internal layers to understand if and how it distinguishes between phrasal and prepositional verbs.

Methods

The researchers employed a "probing" methodology. This involves training simple classification models (Logistic Regression and Support Vector Machines) on the internal representations (embeddings) extracted from different layers of a BERT model. These classifiers are tasked with predicting whether a given verb is phrasal or prepositional.

To assess the linear separability of these representations, they used the Generalized Discrimination Value (GDV). GDV quantifies how well two classes can be separated by a straight line (or hyperplane in higher dimensions). A GDV close to zero suggests poor linear separability, while more negative values indicate better separation.

They also performed a layer-wise analysis, examining how classification accuracy and GDV change across BERT's 12 layers. This helps understand where in the model's processing pipeline these linguistic distinctions are most prominent.

Datasets

The study utilized a dataset extracted from the British National Corpus (BNC, 2001). This dataset contained sentences featuring phrasal and prepositional verbs. The data was manually split into training and testing sets, with specific attention paid to ensuring no overlap between the sets to prevent memorization by the classifiers. The training set included 1920 phrasal verbs and 2070 prepositional verbs, while the test set contained 522 phrasal and 623 prepositional verbs.

Key Insights and Ideas

  • Middle Layers are Key: BERT's middle layers (around layers 6-9) seem to be where the most discriminative features for distinguishing phrasal and prepositional verbs are encoded, as evidenced by the highest classification accuracies.
  • Non-Linear Encoding: Despite high classification accuracies, the GDV analysis revealed weak linear separability. This suggests that while BERT can distinguish between these verb types, it does so through complex, non-linear relationships within its representations, rather than simple linear boundaries.
  • Probing vs. GDV: The mismatch between high probing accuracy and low GDV highlights a crucial point: linear separability (measured by GDV) is not the sole indicator of whether a model has learned a linguistic distinction. Probing classifiers can detect non-linear distinctions that GDV might miss.
  • Task-Specific vs. General Representations: The slight decrease in accuracy in BERT's later layers suggests a shift towards more task-specific representations, which may be less suited for general linguistic distinctions like phrasal vs. prepositional verbs.

Key Results

  • Token-based Classification: Logistic Regression achieved a peak accuracy of 0.99 in layer 6. Linear SVM achieved its highest accuracy of 0.99 in layer 8.
  • Sentence-based Classification: Logistic Regression reached a peak accuracy of 0.85 in layers 6 and 7. Linear SVM peaked at 0.84 in layer 6.
  • GDV Values: For token-based embeddings, GDV values showed improvement (less negative) across layers, reaching their strongest separability at layers 3 and 4 with values around -0.049. For sentence-based embeddings, GDV values remained close to zero throughout the layers, indicating very weak linear separability.
  • Correlation: Statistical analysis (Spearman's rank correlation) showed no statistically significant correlation between classification accuracies and GDV scores, further supporting the idea that linear separability doesn't fully explain the model's ability to distinguish these verb types.

Limitations and Future Work

The authors acknowledge that their study focused on a single LLM architecture (BERT-base-uncased). They suggest that future research should extend this analysis to other models (e.g., RoBERTa-base, BERT-large) to see if these observed patterns generalize across different architectures. They also point out that the limitations of probing classifiers themselves, such as not necessarily indicating how the model actively uses these learned properties, are important considerations. The findings suggest that a comprehensive understanding of LLM representations requires methods that can capture non-linear relationships, moving beyond solely linear separability metrics.

Summary updated

BibTeX
@inproceedings{kissane2025probing,
  author = {Hassane Kissane and Achim Schilling and Patrick Krauss},
  title = {{Probing Internal Representations of Multi-Word Verbs in Large Language Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure