Abstract
Connectionist models have had a profound impact on theories of language. While most early models were inspired by the classic parallel distributed processing architecture, recent models of language have explored various other types of models, including self-organizing models for language acquisition. In this paper, we aim at providing a review of the latter type of models, and highlight a number of simulation experiments that we have conducted based on these models. We show that self-organizing connectionist models can provide significant insights into long-standing debates in both monolingual and bilingual language development. We suggest future directions in which these models can be extended, to better connect with behavioral and neural data, and to make clear predictions in testing relevant psycholinguistic theories.
Introduction: Beyond Supervised Learning
For decades, connectionist (neural network) models have been central to understanding language acquisition. However, many early models relied on supervised learning algorithms like backpropagation, which require an explicit error signal for every input. This approach lacks biological and developmental plausibility; children do not receive a corrected gradient for every word they speak.
Furthermore, many existing models utilize "toy" lexicons that fail to capture the scale and complexity of real-world language. This paper addresses the need for unsupervised, scalable, and biologically inspired models that can explain both first (L1) and second (L2) language acquisition, specifically addressing the "plasticity-stability" dilemma: how the brain remains flexible enough to learn new information while maintaining stable memories.
The DevLex-II Architecture
The authors propose the Self-Organizing Map (SOM) as a primary architecture. SOMs mimic the topographic organization of the human cortex, where similar inputs activate neurons in adjacent regions. The DevLex-II model is a multi-layer system designed to simulate lexical development through three specialized maps.
Key Components:
- Semantic Map: Organizes word meanings based on co-occurrence statistics.
- Phonological Map: Organizes the sounds of words.
- Articulatory Output (SARDNET): A specialized SOM for temporal sequences, managing the order of phonemes during speech production.
- Hebbian Learning: Associative links between maps (e.g., connecting a sound to a meaning) are strengthened when nodes fire simultaneously.
- Self-Adjusting Plasticity: Unlike standard SOMs, DevLex-II adjusts its learning rate based on "quantization error"—a measure of how well the network currently represents the input.
Topographic Mapping and Semantic Organization
A core feature of SOMs is their ability to create structured representations from raw data. In the semantic map, words with similar meanings or usage patterns cluster together, forming distinct regions for different categories.
Training on Realistic Linguistic Data
To ensure ecological validity, the models were trained on large-scale, realistic datasets rather than synthetic sets:
- MacArthur-Bates CDI: A standard inventory used to simulate the early vocabulary stages of toddlers.
- CHILDES Database: Parental speech transcripts used to derive semantic vectors and study grammatical patterns.
- PatPho & WordNet: Provided phonological patterns and semantic feature vectors, respectively.
Key Insights into Language Development
The simulations yielded several significant findings regarding how humans acquire language:
- Emergent Vocabulary Spurt: The rapid acceleration in word learning around 18-24 months is shown to be an emergent "phase transition." It occurs once the network has established a sufficiently organized topographic base, rather than requiring a new innate "module."
- The "Parasitic" Nature of L2: In late second-language learners, the L2 does not develop a dedicated territory. Instead, L2 words occupy fragmented clusters within the established L1 map. This explains L1 interference and why simultaneous bilinguals perform differently than late learners.
- Priming Asymmetry: The model replicates the phenomenon where an L1 word primes its L2 translation more effectively than vice versa. This is due to the richer, more stable representations of L1 words on the map.
- Grammatical Aspect: The model successfully learned to use grammatical markers (like -ing) by picking up on statistical biases in parental speech, matching empirical data on how children initially "under-extend" these markers.
Future Directions
While SOMs are powerful for lexical organization, they currently struggle with the hierarchical nature of syntax. Future work may involve "recursive SOMs" to handle complex sentence structures. Additionally, the authors emphasize the need for tighter integration with neuroimaging data (fMRI/EEG) to validate the model's "activity bubbles" against actual brain activity and the development of more accessible tools for psycholinguists.
Summary updated