Large Language Models as Model Organisms for Human Associative Learning

Camila Kolling, Vy A. Vo, Mariya Toneva

Abstract

This research investigates whether Large Language Models (LLMs) can serve as "model organisms" to understand how the human brain reorganizes information during learning. By analyzing representational changes during in-context learning (ICL), researchers found that LLMs mirror biological activity described by the Non-Monotonic Plasticity Hypothesis (NMPH). Specifically, items with moderate initial similarity undergo differentiation—becoming more distinct—to mitigate memory interference, providing a controllable and scalable framework for cognitive neuroscience.

Introduction

When we learn new associations, such as pairing a face with a name, our brain's internal representational "map" undergoes significant reorganization. Cognitive scientists have long debated whether this process involves integration (increasing similarity) or pattern separation (increasing distinctness). Testing these theories in humans is notoriously difficult due to the challenges of controlling prior similarity and the inherent noise in neuroimaging. This study proposes that LLMs, which can learn associations rapidly through in-context learning without permanent weight updates, offer a transparent laboratory to test these biological hypotheses.

Methods

The researchers adapted a classic associative learning paradigm for LLMs, utilizing the internal vocabularies of six prominent open-source models, including the Llama series, Gemma2, and Mistral-7b. Instead of traditional training, they utilized In-Context Learning (ICL), presenting the model with repeated pairs of tokens $(x, y)$ within a single prompt. The model's "learning" is driven purely by its attention mechanism.

Experimental setup for measuring representational change in LLMs during in-context learning.

To measure Representational Change, the team calculated the cosine similarity between the hidden vector representations of tokens at the model's final layer before and after the learning repetitions. They employed Greedy Coordinate Gradient (GCG) to identify token pairs across the full spectrum of initial similarity. Furthermore, they introduced a "Vocabulary Interference" metric to quantify how much a new association competes with the model's pre-existing knowledge based on the density of similar tokens in its vocabulary.

Key Insights: The Non-Monotonic Plasticity Hypothesis

The study's primary finding is that LLMs adhere to the Non-Monotonic Plasticity Hypothesis (NMPH). During the consolidation phase of learning, items that are already highly distinct or highly similar tend to maintain their relative positions. However, items with moderate initial similarity undergo significant differentiation, being pushed further apart in the model's internal space.

The Non-Monotonic Plasticity Hypothesis (NMPH) predicts a U-shaped relationship where moderate initial similarity leads to differentiation.

The researchers identified three distinct phases in LLM in-context learning:

  1. Encoding: Rapid accuracy gains accompanied by integration (low-similarity pairs moving closer).
  2. Consolidation: High accuracy is maintained while representations reorganize, specifically showing differentiation for mid-similarity pairs (0.55–0.75 cosine similarity).
  3. Forgetting: A phase where accuracy eventually declines.

The Role of Competition

Differentiation is not merely a function of the pair itself but is driven by "global" competition. High vocabulary interference—where a token is surrounded by many similar "distractor" tokens—increases the pressure to differentiate representations to maintain accuracy. When using real-world words from WordNet, which tend to have higher natural interference, the U-shape is less pronounced, and the models show a more monotonic trend toward differentiation.

Conclusion and Future Work

By demonstrating that LLMs reorganize associations similarly to biological systems, this work establishes AI models as viable "transparent brains." While LLMs are not perfect biological replicas, their accessibility allows researchers to probe the drivers of representational change in ways impossible with human subjects. Future research will explore dynamics in earlier layers, developmental trajectories during pre-training, and the influence of semantic meaning on representational reorganization.

Summary updated

Abstract

Associative learning--forming links between co-occurring items--is fundamental to human cognition, reshaping internal representations in complex ways. Testing hypotheses on how representational changes occur in biological systems is challenging, but large language models (LLMs) offer a scalable alternative. Building on LLMs' in-context learning, we adapt a cognitive neuroscience associative learning paradigm and investigate how representations evolve across six models. Our initial findings reveal a non-monotonic pattern consistent with the Non-Monotonic Plasticity Hypothesis, with moderately similar items differentiating after learning. Leveraging the controllability of LLMs, we further show that this differentiation is modulated by the overlap of associated items with the broader vocabulary--a factor we term vocabulary interference, capturing how new associations compete with prior knowledge. We find that higher vocabulary interference amplifies differentiation, suggesting that representational change is influenced by both item similarity and global competition. Our findings position LLMs not only as powerful tools for studying representational dynamics in human-like learning systems, but also as accessible and general computational models for generating new hypotheses about the principles underlying memory reorganization in the brain.
BibTeX
@inproceedings{kolling2025large,
  author = {Camila Kolling and Vy A. Vo and Mariya Toneva},
  title = {{Large Language Models as Model Organisms for Human Associative Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=oOPdUnswhZ},
}
Figure