Investigating noun-noun compound relation representations in autoregressive large language models

Saffron Kendrick, Mark Ormerod, Hui Wang, Barry DevereuxNAACL 2025

TL;DR

This paper investigates how well autoregressive Large Language Models (LLMs) like GPT-2 and LLaMA understand the semantic relationships within noun-noun compounds. By using representational similarity analysis and probing, they found that these models can indeed decode this information, with the most crucial insights emerging from the final words in a sequence. This work sheds light on the internal workings of LLMs and their ability to grasp complex linguistic structures.

What Problem Does This Paper Address?

Understanding how Large Language Models (LLMs) process and interpret language is a significant challenge in the field of AI. While LLMs have demonstrated impressive capabilities across various Natural Language Processing (NLP) tasks, their internal mechanisms remain largely opaque. This paper specifically focuses on noun-noun compounds (e.g., "coffee cup," "mountain breeze"), which are fundamental to language but pose a complex problem for semantic understanding. These compounds combine two nouns, where one modifies the other, creating a new meaning that isn't always a simple sum of its parts. The research aims to pinpoint when and how autoregressive LLMs (which process text sequentially, predicting the next word) extract the semantic relationship between the head noun and its modifier within these compounds. This is crucial for building more interpretable and reliable AI systems.

Methods

The researchers employed two primary methods:

  • Representational Similarity Analysis (RSA): This technique compares the internal representations (embeddings) of words or phrases within an LLM to a "ground truth" representation of their semantic relationships. They constructed representational dissimilarity matrices (RDMs) for both the models' outputs and the ground truth data. By calculating the correlation between these matrices, they could assess how well the LLM's internal representations captured the intended semantic relationships.
  • Probing: This involves training a simple classifier (in this case, a linear probe) on the LLM's extracted representations to see if it can predict specific linguistic information, such as the semantic relation between nouns in a compound. They adapted this by testing two processing conditions:
  • "Together" condition: The head and modifier nouns are processed within the same sentence.
  • "Separate" condition: The head and modifier nouns are processed in separate sentences, and their representations are then averaged.

They also analyzed representations layer-by-layer to understand when in the model's processing the semantic information becomes decodable.

Datasets

The study utilized two datasets:

  1. Gagné (2001) Dataset: This dataset consists of 300 English noun-noun compounds, organized into groups of five. Each group includes a target compound and variations that share either the head noun or the modifier, but differ in their relation. This dataset was used for the relation category RSA experiments.
  2. Devereux and Costello (2005) Dataset: This dataset comprises 60 noun-noun compounds, for which 34 participants ranked the appropriateness of 18 possible relations. This resulted in 18-dimensional relation vectors for each compound, providing a fine-grained representation of semantic information for the compositional probe experiments.

Key Insights and Ideas

  • Autoregressive LLMs can decode compound semantics: Contrary to some assumptions, autoregressive models are not just predicting the next word but are actively extracting and representing semantic relationships within noun-noun compounds.
  • Context is king (mostly): Processing the head and modifier nouns together in a sentence ("Together" condition) consistently yielded better results than processing them separately and averaging their representations. This highlights the importance of contextual information for LLMs.
  • The final word is crucial: The most significant correlations between model representations and ground truth semantic relations were observed for the final word token in the sequence, especially in the "Together" condition. This suggests that by the end of processing a compound, the model has a strong grasp of its meaning.
  • Head nouns carry significant relational weight: Head noun representations showed a notable correlation with semantic relations. This might be due to the inherent concreteness of many head nouns, which are often tangible concepts that are easier to represent relationally.
  • Modifier nouns hint at relational distributions: While less impactful than head nouns, modifier representations still showed some correlation, potentially reflecting learned associations between modifiers and specific relations from their pre-training data (akin to the CARIN model).
  • Model differences emerge: While most models showed an ability to decode semantics, GPT-2 and DeepSeek-R1 seemed to actively process contextual information, whereas Phi and LLaMA appeared to achieve similar results even when processing words separately, suggesting they might rely more on pre-learned individual word representations.

Key Results

  • RSA Experiments: All models tested (except the non-English BERT-Japanese control) showed the ability to capture semantic relation information. For autoregressive models, the final word representation yielded the highest correlation, followed by the final head noun.
  • Processing Condition Experiments: The "Together" condition consistently outperformed the "Separate" condition across most autoregressive models, with statistically significant differences observed for GPT, Phi, and DeepSeek. This strongly supports the idea that context is vital for these models.
  • Compositional Probe: The "Together" condition generally led to a higher proportion of successful tests, indicating that contextual processing is beneficial for decoding fine-grained semantic information. DeepSeek and GPT showed a notable advantage in the "Together" condition, suggesting they actively leverage context.
  • Layer-wise Analysis: Positive correlations for GPT began at the first head token position and strengthened in middle to later layers for final word representations. For DeepSeek, correlations were most pronounced in middle layers. LLaMA showed an earlier increase in correlation, peaking around the middle layers.

Limitations and Future Work

  • Dataset Size: The datasets used were relatively small (60 compounds for probing, 300 for RSA). Larger datasets would be needed to account for potential noise and model sensitivity, especially for larger LLMs.
  • Novel Compounds: The study primarily used existing compounds. Future work could explore how LLMs generalize to novel or unseen compounds.
  • Model Scale: The models tested were in the 1-3B parameter range. Investigating larger models could reveal how parameter scale affects semantic representation extraction.
  • Fine-tuning: The study used base configurations. Fine-tuning models could reveal how they adapt to contextual composition when they can no longer rely solely on pre-trained associations.
  • MoE Architectures: The unique Mixture-of-Experts (MoE) architecture of DeepSeek warrants further exploration to understand how specific "experts" contribute to semantic interpretation.

Summary updated

BibTeX
@inproceedings{kendrick2025investigating,
  author = {Saffron Kendrick and Mark Ormerod and Hui Wang and Barry Devereux},
  title = {{Investigating noun-noun compound relation representations in autoregressive large language models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure