Overlap in meaning is a stronger predictor of semantic activation in GPT-3 than in humans

Jan Digutsch, Michał Kosiński

TL;DR

This paper investigates how Large Language Models (LLMs) like GPT-3 organize semantic information by comparing their "semantic activation" patterns to those of humans. They found that while GPT-3's overall patterns are similar to humans, its semantic activation is more strongly driven by the meaning of words (semantic similarity) than by how often they co-occur (associative similarity), a pattern that is reversed in humans. This suggests LLMs might be building a more meaning-centric semantic network than previously thought.

What Problem Does This Paper Address?

Modern LLMs, like GPT-3, can generate text that is remarkably human-like and perform impressively on comprehension tasks. However, their internal workings remain largely a "black box." Understanding how these models represent and process language is crucial for explaining their behavior, predicting their outputs, and ensuring their responsible development. A key aspect of language understanding is semantic memory – how we store and retrieve information about word meanings and their relationships. Researchers have long studied semantic memory in humans using phenomena like "semantic priming," where exposure to one word makes it easier to process a related word. This paper aims to bridge the gap between human semantic memory and LLM representations by applying similar experimental paradigms to GPT-3.

Methods

The researchers adapted a classic human psychology experiment, the lexical decision task (LDT), to study semantic activation in GPT-3. In an LDT, participants quickly decide if a presented string of letters is a real word. Semantic priming occurs when a "prime" word (e.g., "lemon") makes it faster to identify a related "target" word (e.g., "lime") compared to an unrelated prime.

For GPT-3, semantic activation was operationalized by calculating the "cosine distance" between word embeddings. Embeddings are numerical representations of words in a high-dimensional space, where similar words are closer together. A smaller cosine distance (or higher cosine similarity) indicates a stronger semantic relationship.

The study involved four analyses:

  1. Comparing priming conditions: They compared GPT-3's semantic activation for word pairs categorized as "first-associate" (most common response), "other-associate" (less common response), and "unrelated."
  2. Comparing association types: They examined semantic activation across 12 different types of prime-target associations (e.g., synonyms, antonyms, functional properties) to see how GPT-3 and humans differ.
  3. Direct comparison of word pairs: They identified specific word pairs where GPT-3 and human semantic activation differed most significantly.
  4. LLM evolution: They compared semantic activation patterns across different versions of GPT-3 and other LLMs to see if newer, more complex models show a stronger reliance on semantic similarity.

Datasets

The study utilized data from the Semantic Priming Project database. This database contains lexical decision task data from human participants, including response times for various prime-target word pairs. The researchers used this existing, publicly available, and de-identified dataset for human responses. For GPT-3, they used its internal representations derived from its training data.

Key Insights and Ideas

  • LLMs mirror human semantic patterns, but with a twist: GPT-3's semantic activation patterns are broadly similar to humans, showing stronger activation for related words than unrelated ones. This suggests LLMs are capturing some fundamental aspects of how humans organize meaning.
  • Meaning over co-occurrence for GPT-3: The most striking finding is that GPT-3's semantic activation is more strongly predicted by the similarity in meaning between words (semantic similarity) than by how often they appear together in text (associative similarity). This is the opposite of what is typically observed in humans, where associative similarity often plays a more dominant role.
  • The "meaning-centric" LLM: This suggests that LLMs like GPT-3 might be building a semantic network that is more fundamentally organized around conceptual meaning, rather than solely on statistical co-occurrence patterns found in text.
  • Evolution towards semantic similarity: Newer and more complex LLMs show an even stronger tendency for their semantic space to be organized around semantic similarity, indicating a trend in LLM development.

Key Results

  • Analysis 1: GPT-3 showed significantly higher semantic activation for "first-associate" word pairs (e.g., "lime" primed by "lemon") compared to "other-associate" and "unrelated" pairs, mirroring human behavior. However, the differences between these conditions were larger in GPT-3.
  • Analysis 2 & 3: GPT-3's semantic activation was particularly strong for word pairs with high semantic similarity (e.g., synonyms like "afraid-scared," antonyms like "day-night," and categories like "lime-lemon"). In contrast, humans showed stronger activation for word pairs that frequently co-occur, such as phrasal associates (e.g., "mountain-top"). For instance, "introvert" strongly activated "extrovert" in GPT-3 (3.81 SD above mean) but was below average in humans (-3.06 SD below mean).
  • Analysis 4: Across different LLMs, more recent and complex models demonstrated a greater reliance on semantic relatedness compared to associative relatedness.

Limitations and Future Work

  • Measurement artifact possibility: The authors acknowledge that the way semantic activation is measured might contribute to the observed differences. Human LDTs can be influenced by factors beyond pure semantic activation, such as expectancy or retrospective matching, which are not present in GPT-3's direct cosine similarity measure. This could potentially inflate the apparent importance of semantic similarity in LLMs.
  • LLMs as models for human cognition: While LLMs can mimic human-like psychological processes, the authors caution that their behaviors can also differ significantly. They suggest that LLMs can be valuable tools for psycholinguistic research, enabling rapid testing of hypotheses that can then be confirmed in humans, but direct equivalence should not be assumed.
  • Future directions: The paper opens avenues for further research into the internal representations of LLMs and how they compare to human semantic memory. Exploring other experimental paradigms and deeper analyses of LLM architectures could provide more nuanced insights. The authors also suggest that LLMs could serve as valuable model participants in psycholinguistic studies.

Summary updated

Abstract

Modern large language models generate texts that are virtually indistinguishable from those written by humans and achieve near-human performance in comprehension and reasoning tests. Yet, their complexity makes it difficult to explain and predict their functioning. We examined a state-of-the-art language model (GPT-3) using lexical decision tasks widely used to study the structure of semantic memory in humans. The results of four analyses showed that GPT-3's patterns of semantic activation are broadly similar to those observed in humans, showing significantly higher semantic activation in related (e.g., "lime-lemon") word pairs than in other-related (e.g., "sour-lemon") or unrelated (e.g., "tourist-lemon") word pairs. However, there are also significant differences between GPT-3 and humans. GPT-3's semantic activation is better predicted by similarity in words' meaning (i.e., semantic similarity) rather than their co-occurrence in the language (i.e., associative similarity). This suggests that GPT-3's semantic network is organized around word meaning rather than their co-occurrence in text.
BibTeX
@inproceedings{digutsch2023overlap,
  author = {Jan Digutsch and Michał Kosiński},
  title = {{Overlap in meaning is a stronger predictor of semantic activation in GPT-3 than in humans}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-32248-6},
}
Figure