Abstract
Rapid progress in machine learning for natural language processing has the potential to transform debates about how humans learn language. However, the learning environments and biases of current artificial learners and humans diverge in ways that weaken the impact of the evidence obtained from learning simulations. For example, today's most effective neural language models are trained on roughly one thousand times the amount of linguistic data available to a typical child. To increase the relevance of learnability results from computational models, we need to train model learners without significant advantages over humans. If an appropriate model successfully acquires some target linguistic knowledge, it can provide a proof of concept that the target is learnable in a hypothesized human learning scenario. Plausible model learners will enable us to carry out experimental manipulations to make causal inferences about variables in the learning environment, and to rigorously test poverty-of-the-stimulus-style claims arguing for innate linguistic knowledge in humans on the basis of speculations about learnability. Comparable experiments will never be possible with human subjects due to practical and ethical considerations, making model learners an indispensable resource. So far, attempts to deprive current models of unfair advantages obtain sub-human results for key grammatical behaviors such as acceptability judgments. But before we can justifiably conclude that language learning requires more prior domain-specific knowledge than current models possess, we must first explore non-linguistic inputs in the form of multimodal stimuli and multi-agent interaction as ways to make our learners more efficient at learning from limited linguistic input.
Introduction: The Poverty of the Stimulus
For decades, the field of linguistics has been polarized by the "Poverty of the Stimulus" argument. Nativists, such as Noam Chomsky, argue that the linguistic data children receive is too sparse and messy to explain their mastery of complex grammar; therefore, humans must be born with innate, language-specific knowledge. Empiricists, on the other hand, argue that general-purpose learning mechanisms are sufficient to extract these rules from the environment.
While modern Large Language Models (LLMs) like GPT-4 demonstrate incredible grammatical mastery, they currently "cheat" by training on roughly 1,000 times more data than a child hears by age ten. This creates a relevance gap: we cannot easily use a model trained on the entire internet to make claims about a child learning from their parents. To bridge this gap, researchers are using artificial neural networks (ANNs) as "model learners" to simulate human-like acquisition under controlled constraints.
The Model Learner Paradigm
The authors advocate for a specific experimental framework known as the Deprivation (Ablation) Study. This involves identifying a specific advantage—such as massive data volume or a pre-programmed architectural bias—and training a model without it. If the "weakened" model still acquires the target linguistic knowledge (e.g., subject-verb agreement in complex sentences), it provides a proof of concept that the trait is learnable without that specific advantage.
In this paradigm, positive results are more scientifically meaningful than negative ones. If a model succeeds, it proves learnability; if it fails, it may simply be a poor simulation rather than evidence that the trait is inherently unlearnable.
The Scale Problem
A major hurdle in using ANNs for cognitive science is the data-efficiency gap. A 10-year-old human has been exposed to roughly 100 million words, whereas state-of-the-art models require hundreds of billions to reach peak performance.
When models like RoBERTa are restricted to "human-scale" data (the 100M-word mark), their performance on linguistic benchmarks like BLiMP (Benchmark of Linguistic Minimal Pairs) drops significantly. While they may reach near-human levels in some categories, they struggle with complex syntax like "island effects," suggesting that current architectures are significantly less data-efficient than human children.
Comparing Human and Model Advantages
To understand this efficiency gap, we must contrast the innate and environmental advantages of both learners. Humans benefit from sensorimotor stimuli and social interaction, while models rely on massive text corpora and high numerical precision.
Interestingly, current research into "grounding"—adding visual or auditory stimuli to language models—has often been disappointing. In many cases, multimodal input actually decreases linguistic performance compared to text-only models. This suggests that current architectures are not yet capturing the "conceptual scaffolding" that helps human children learn language through their interaction with the physical and social world.
Conclusion and Future Directions
The goal of this research agenda is to move beyond chasing engineering benchmarks and instead build models that fail and succeed in the same ways children do. By systematically ablating advantages, we can rigorously test the necessity of innate linguistic knowledge. Closing the data-efficiency gap will likely require moving beyond static text toward models that learn through interaction, social cues, and multimodal grounding, providing a more faithful simulation of the human learning environment.
Summary updated