This study explores a fascinating intersection of cognitive psychology and mechanistic interpretability: the "tip-of-the-tongue" (TOT) state in Artificial Intelligence.
TL;DR
Researchers investigated whether small language models (SLMs) possess internal "name information" for an object even when they fail to generate its name. By using causal patching and similarity read-outs, they found that name-predictive signals consistently emerge in the middle layers of these models, suggesting that an internal representation of a word often precedes its successful production.
What Problem Does This Paper Address?
In humans, the TOT state occurs when you know a concept but cannot retrieve the specific word for it. You might know what a "theodolite" does, but the word itself remains out of reach.
The author asks: Do language models experience something analogous? Specifically, when a model reads a description of an object but fails to output its name, is the "idea" of that name still present in its hidden activations? Previous research has looked at factual recall, but this paper specifically targets the gap between internal state (the "vehicle" of information) and external output (the "content" or name).
Methods
The study employed an internally pre-registered pipeline (using a SHA-256 frozen document for transparency) to test three small models: Qwen2-0.5B, SmolLM2-1.7B, and Phi-3-mini.
The researchers used three primary lenses to examine the models' hidden states:
- Similarity Read-out (CSLS): Comparing the hidden state generated by a description to the hidden state generated by the actual name. They used Cross-Domain Similarity Local Scaling (CSLS) to account for "hubs" (vectors that appear close to many others by default).
- Patchscopes: An attempt to "verbalize" a hidden state by inserting it into a separate "bottleneck" prompt to see if the model can describe its own internal representation.
- Activation Patching: A causal intervention where a hidden state from a "donor" (the description) is pasted into a "receiver" prompt ("This object is called...") to see if it forces the model to output the correct name.
The study also categorized items into groups: Group C (named correctly) and Group A (naming failure, but the model could still correctly pick the description in a multiple-choice test).
Datasets
The researchers developed a custom dataset of 200 concrete nouns (names). For each item, they generated:
- A freshly written definition-style description.
- A paraphrase of that description.
- A WordNet gloss (for 177 items).
- "Hard negatives" (semantically similar items used as controls).
Key Insights and Ideas
- The Middle-Layer Sweet Spot: Name-predictive information is most concentrated in the middle layers (roughly 40–75% depth). This is where the model has processed the description enough to "know" the target but hasn't yet committed to the final token generation.
- "Knowing" vs. "Saying": Interestingly, Patchscopes (trying to get the model to explain its state) stayed near chance. However, Similarity and Patching were highly successful. This suggests that models "know" the name internally in a way that is causally potent, even if they can't easily "talk about" that internal state.
- Causal Sufficiency: The patching experiments proved that the internal state derived from a description is causally sufficient to drive the name output in a naming context, even if the model wouldn't have produced that name on its own.
- The Vehicle/Content Distinction: The author emphasizes that they are measuring "vehicle-level signals" (activations) rather than claiming the model "understands" meaning in a human sense.
Key Results
- H1 Supported: In all nine model-by-description combinations, the internal state was predictive of the name.
- Patching Gains: The "gain" (increase in log-probability of the correct name) was significant across models, reaching up to 18.35 nats in Phi-3-mini.
- Consistency in Failure (H2'): Results were mixed. In Phi-3-mini, the internal consistency between a description and its paraphrase was just as high when the model failed to name the object as when it succeeded. In SmolLM2, there was a slight deficit in consistency during failure, though this result was not robust to all checks.
Limitations and Future Work
- Model Size: The study was limited to very small models (<4B parameters). The author proposes a "Capacity Hypothesis": perhaps larger models maintain more distinct internal states during naming failures than smaller ones.
- The Human Analogy: The author is careful to note that this is an analogy; SLMs don't "feel" frustration or the "ache" of a TOT state.
- Future Directions: A key next step is testing this pipeline on a scaling series of models (e.g., 7B, 13B, 70B) to see if the gap between internal representation and output production changes with scale.
- Precision Effects: One model (Phi-3) required 4-bit quantization to run, which may have introduced noise into the group assignments.
Summary updated