Abstract
Humans possess the capability to reason at an abstract level and to structure information into abstract categories, but the underlying neural processes have remained unknown. Experimental evidence has recently emerged for the organization of an important aspect of abstract reasoning: for assigning words to semantic roles in a sentence, such as agent (or subject) and patient (or object). Using minimal assumptions, we show how such a binding of words to semantic roles emerges in a generic spiking neural network through Hebbian plasticity. The resulting model is consistent with the experimental data and enables new computational functionalities such as structured information retrieval, copying data, and comparisons. It thus provides a basis for the implementation of more demanding cognitive computations by networks of spiking neurons.
Introduction
A fundamental mystery in neuroscience is how the brain represents structured information. Consider the sentences "The truck hit the ball" and "The ball hit the truck." While they contain identical words, their meanings are opposite because the semantic roles of "agent" and "patient" are swapped. In computer science, such distinctions are handled via registers and pointers. However, the brain lacks a central controller or pre-defined memory addresses. Previous models of neural binding often relied on highly specific, hand-coded wiring that is difficult to justify biologically. This paper bridges the gap between Marr’s algorithmic level of language processing and the biological implementation level, demonstrating how structured representations emerge naturally from generic, randomly connected networks.
Methods and Architecture
The proposed architecture consists of two primary types of neural populations: the Content Space ($C$) and Variable Spaces ($V, U$). The Content Space mimics "concept cells" found in the human medial temporal lobe, where specific assemblies represent individual words or concepts like "truck" or "ball." The Variable Spaces represent semantic roles, corresponding to specific subareas of the temporal cortex, such as the left mid-superior temporal cortex (lmSTC), identified in fMRI studies.
Key Mechanisms
- Disinhibition: The model employs "inhibitory locks." A Variable Space remains inactive until it is disinhibited—likely by VIP inhibitory neurons—allowing it to respond to input from the Content Space.
- Assembly Projections: When a concept is active in $C$ and a Variable Space is disinhibited, a new, sparse assembly forms in the Variable Space. This assembly acts as a distributed "pointer."
- Hebbian Plasticity: Spike-Timing-Dependent Plasticity (STDP) strengthens the bidirectional links between the content and the variable. This creates a "handle" that can trigger the recall of the original word without storing the word's full representation within the variable space itself.
- Transient Excitability: Short-term memory is maintained through temporary increases in neuronal excitability, allowing the network to retain the most recent binding after the stimulus fades.
Key Results
The model demonstrates high performance across several cognitive benchmarks. A linear classifier trained on Variable Space activity identified the role of a word (e.g., whether "truck" was the agent) with a test error of only 3.0%. In contrast, decoding the role from the Content Space was no better than random (~44% error), a finding that aligns with biological observations that content and role are represented in distinct cortical areas.
In 250 trials across different random network initializations, the model achieved a 100% success rate in retrieving the correct content from a variable space after a delay. Furthermore, the model successfully implemented "atomic computational primitives" such as COPY (moving a pointer between variables) and COMPARE (detecting if two variables point to the same content). These results show that spiking neurons can behave like a symbolic processor while maintaining biological realism.
Discussion
While the model provides a robust framework for structured representation, it assumes an external process handles the timing of disinhibition (the "controller" problem). Future work must address the higher-level logic that decides which word occupies which semantic slot. Additionally, while the model handles simple "Subject-Verb-Object" structures, scaling to recursive syntax and long-term consolidation of these transient assemblies remains an open area for research. Nonetheless, the emergence of symbolic-like operations from local learning rules and random connectivity offers a promising path toward understanding high-level cognition in the brain.
Summary updated