Abstract
Children learn through curiosity, seeking novelty and information as they interact with physical objects and people. This flexible capacity to learn through intrinsically motivated interaction continues throughout life. This paper explores how to engineer artificial agents that learn in this way—flexibly interacting with their environment to build general-purpose representations. We review advances in AI, contrast them with human learning, and discuss how curiosity-driven agents can serve as computational models for human development and neurodiversity.
Introduction: The "Scientist in the Crib"
Developmental psychologists often describe children as "scientists in the crib." From birth, humans are driven by an internal spark to explore, manipulate, and understand the world around them. This learning is not driven by external labels or specific rewards; rather, it is a form of "play" that builds a foundation for complex cognitive tasks. In contrast, modern Artificial Intelligence has largely succeeded through passive observation or narrow task-optimization. To bridge the gap between AI and human intelligence, we must move toward agents that possess intrinsic curiosity.
The Limitations of Supervised and Model-Free Learning
The last decade of AI has been defined by "superhuman" performance in specific domains. Computer vision models, trained on datasets like ImageNet, achieve high accuracy but remain passive learners, dependent on millions of human-annotated labels. Similarly, Reinforcement Learning (RL) agents have mastered complex games like those on the Atari platform, yet they often rely on handcrafted reward functions.
In the real world, rewards are sparse. A child in a playroom does not receive a constant stream of labels or points. They solve the "sparse reward problem" by being intrinsically motivated to understand and control their environment. Engineering this "intrinsic spark" is essential for creating agents that can build general-purpose representations of physics and psychology without constant human intervention.
Engineering Curiosity: World Models and Learning Progress
To replicate human-like learning, researchers are shifting from "model-free" to model-based reinforcement learning. In this framework, an agent builds an internal "world model"—a predictive simulation of how the environment responds to its actions.
The Goldilocks Effect
Early models rewarded agents for encountering states they could not predict (prediction error). However, this leads to the "White Noise Problem," where an agent becomes fixated on random, unpredictable stimuli (like static on a TV) because the error remains high.
A more robust approach rewards the agent for learning progress—the rate at which its prediction error is decreasing. This creates a "Goldilocks Effect," where the agent ignores what it already knows (too simple) and what it cannot possibly learn (too complex/random), focusing instead on the "just right" level of learnable complexity.
Emergent Behavior in Virtual Playrooms
Experimental setups using 3D "playrooms" demonstrate the power of curiosity. In these environments, agents are given no external goals. While a random agent interacts with objects less than 1% of the time, a curiosity-driven agent actively manipulates toys.
Over time, sophisticated behaviors emerge. An agent might learn to gather multiple toys together to see how they interact, effectively creating its own training data. Furthermore, when presented with "animate" stimuli (agents that act with goals), curiosity-driven models naturally develop a preference for social-like interaction. Because social agents are complex yet follow regularities, they provide a rich and sustained source of learning progress.
AI as a Tool for Developmental Psychology
Beyond engineering better robots, these models offer a new lens for developmental psychology. By adjusting the parameters of an agent's curiosity—such as how it weights social versus physical information—researchers can create fine-grained computational models of human learning trajectories.
One promising application is the study of Autism Spectrum Disorder (ASD). By simulating different "curiosity profiles," we can explore how variations in intrinsic motivation might lead to the different developmental outcomes observed in neurodiverse populations. This "triangulation" between AI behavior and human data allows for a more precise understanding of the mechanisms underlying human cognition.
Conclusion
The gap between current AI and the learning capacity of a child remains wide. However, by shifting our focus from passive supervision to curiosity-driven interaction, we move closer to creating truly autonomous, flexible learners. The development of an "ImageNet of Development"—massive datasets of childhood behavior—combined with progress-based intrinsic motivation, will be key to unlocking the next generation of artificial and computational intelligence.
Summary updated