Abstract
We introduce VAST, the Valence-Assessing Semantics Test, a novel intrinsic evaluation task for contextualized word embeddings (CWEs). VAST uses valence—the association of a word with pleasantness—to measure the correspondence of word-level semantics with human judgments. By examining OpenAI's GPT-2 and other models, we investigate the effects of contextualization, tokenization, and model geometry. Our results show that word-level semantics are often masked by syntactic structures in top layers but can be recovered through principal component nullification. Furthermore, isolating these semantics improves performance on word similarity benchmarks and reveals that non-semantic structures often mask underlying social biases.
Introduction
While large language models (LMs) have become central to natural language processing, evaluating their internal representations—Contextualized Word Embeddings (CWEs)—remains challenging. Traditional intrinsic evaluations like SimLex-999 were designed for static embeddings and fail to account for the dynamic nature of CWEs, which shift based on context, tokenization, and learned model geometry. We address this gap with VAST, a test that isolates these factors to determine where and how a model encodes word meaning.
Methodology
VAST utilizes human-rated valence as a proxy for semantic quality. We evaluate models across several dimensions:
- Contextual Settings: Words are tested in four environments: Random (natural context), Bleached (neutral templates), Aligned (context matches word valence), and Misaligned (context clashes with valence).
- SC-WEAT: The Single Category Word Embedding Association Test quantifies the association between a target word and "pleasant" vs. "unpleasant" attributes.
- PC Nullification: We identify and nullify dominant, non-semantic principal components (PCs) in the vector space to uncover masked semantics.
- Layer-wise Analysis: We apply VAST across all model layers to track the evolution of meaning from input to output.
Key Results and Discussion
Contextualization and the Layer 8 "Sweet Spot"
In GPT-2, word-level semantics peak around Layer 8. In the final layers (11 and 12), the model prioritizes surrounding context for next-token prediction, causing individual word meanings to fade. As shown in Figure 1, VAST scores diverge significantly in these later layers depending on the contextual setting.
The Tokenization Lag
Common, singly tokenized words exhibit high semantic clarity from Layer 0. In contrast, rare words split into multiple sub-tokens do not achieve comparable clarity until Layer 8. This suggests that the model requires several layers to "assemble" the meaning of sub-tokens into a coherent semantic representation.
Syntax Masks Semantics
The dominant principal components in top layers appear to encode sentence-level syntax rather than noise. By nullifying these components, the VAST score in GPT-2’s top layer improves from 0.32 to 0.76. This isolated semantic signal allows GPT-2 to achieve a score of 0.50 on SimLex-999, surpassing the previous state-of-the-art of 0.45.
Amplification of Social Bias
Isolating the semantic signal reveals a sobering reality: non-semantic structures in LMs often mask the extent of their social biases. After nullifying non-semantic PCs, 8 out of 10 WEAT bias tests showed stronger stereotype-congruent effect sizes.
Conclusion
VAST provides a robust framework for understanding the internal semantic landscape of contextualized language models. Our findings highlight that while LMs develop sophisticated semantic representations, these are often obscured by syntactic requirements in later layers. By isolating these semantics, we can better evaluate model quality, improve performance on downstream tasks, and more accurately measure—and eventually mitigate—inherent social biases.
Summary updated