When a child sees a Dalmatian and hears the word "dax," they face an inductive leap. Does "dax" refer to that specific dog, the breed of Dalmatians, the category of dogs, or all living animals? This is the word generalization problem. Despite the infinite number of logical possibilities, humans are remarkably consistent and efficient at narrowing down the correct level of abstraction, often after seeing only a few examples.
Research into human category learning suggests this ability is not a collection of hard-coded rules, but a result of ecologically rational inference: a combination of a mathematical "size principle" and priors tuned to the statistical structure of the real world.
The Size Principle: Why Specificity Wins
The core mechanism behind rapid generalization is the size principle. In a Bayesian framework, the size principle acts as a "soft constraint" that favors the most specific hypothesis consistent with the data.
As Fei Xu demonstrates in Word learning as Bayesian inference, the likelihood of a set of examples being drawn from a category is inversely proportional to the size of that category. If you see three Dalmatians labeled "dax," it is highly unlikely they were all drawn at random from the massive category of "animals." It is much more likely they were drawn from the smaller, more specific category of "Dalmatians."
This principle explains the transition from broad, similarity-based generalization to sharp, rule-like categorization:
- One Example: With only one data point, the learner’s prior beliefs (e.g., a bias toward "basic-level" categories like dog) dominate, leading to broad generalization.
- Multiple Examples: As more examples accumulate, the likelihood function—driven by the size principle—overwhelms the prior. The learner quickly converges on the smallest category that encompasses all observed instances.
From Prototypes to Exemplars
This statistical pressure doesn't just narrow categories; it changes how we represent them. Early in the learning process, humans often rely on prototypes—a "fuzzy" average of what a category looks like. However, as we gain experience, we shift toward exemplar-based memory, where we remember specific instances.
Work by Akshay K. Jagadish, Marcel Binz, and Eric Schulz in Human-like Category Learning by Injecting Ecological Priors from Large Language Models into Neural Networks shows that this shift is an optimal response to increasing data density. By training neural networks on thousands of "ecologically valid" tasks, they found that the prototype-to-exemplar transition emerges naturally. This suggests that the stages of human development are not just biological milestones, but are statistically grounded adaptations to the environment.
Ecological Rationality and LLM Priors
Why do humans have a "basic-level bias," preferring to label things as dog rather than mammal or quadruped? Aida Nematzadeh, Erin Grant, and Suzanne Stevenson argue in A Computational Cognitive Model of Novel Word Generalization that these biases emerge from tracking the frequency of both object types and individual encounters.
To capture these "ecological priors"—the background knowledge humans have about how the world is structured—researchers have turned to Large Language Models (LLMs). In the Ecologically Rational Meta-learned Inference (ERMI) framework, Jagadish and colleagues use LLMs to synthesize thousands of real-world tasks.
When a model is meta-learned on these tasks, it begins to exhibit human-like "irrationalities." These biases are not flaws; they are highly tuned shortcuts for a world where data is structured hierarchically and categories have varying densities. As Björn Meder and Eric Schulz synthesize in Unifying principles of generalization: past, present, and future, human generalization is a sophisticated hybrid of flexible similarity and rigid rules, unified by Bayesian inference to navigate complex reward landscapes.