Curiosity and Interactive Learning in Artificial Systems

Nick HaberOpenalex Unknown 2022 · DOI · OpenAlex

Abstract

Children learn through curiosity, seeking novelty and information as they interact with physical objects and people. This flexible capacity to learn through intrinsically motivated interaction continues throughout life. This paper explores how to engineer artificial agents that learn in this way—flexibly interacting with their environment to build general-purpose representations. We review advances in AI, contrast them with human learning, and discuss how curiosity-driven agents can serve as computational models for human development and neurodiversity.

Introduction: The "Scientist in the Crib"

Developmental psychologists often describe children as "scientists in the crib." From birth, humans are driven by an internal spark to explore, manipulate, and understand the world around them. This learning is not driven by external labels or specific rewards; rather, it is a form of "play" that builds a foundation for complex cognitive tasks. In contrast, modern Artificial Intelligence has largely succeeded through passive observation or narrow task-optimization. To bridge the gap between AI and human intelligence, we must move toward agents that possess intrinsic curiosity.

The Limitations of Supervised and Model-Free Learning

The last decade of AI has been defined by "superhuman" performance in specific domains. Computer vision models, trained on datasets like ImageNet, achieve high accuracy but remain passive learners, dependent on millions of human-annotated labels. Similarly, Reinforcement Learning (RL) agents have mastered complex games like those on the Atari platform, yet they often rely on handcrafted reward functions.

In the real world, rewards are sparse. A child in a playroom does not receive a constant stream of labels or points. They solve the "sparse reward problem" by being intrinsically motivated to understand and control their environment. Engineering this "intrinsic spark" is essential for creating agents that can build general-purpose representations of physics and psychology without constant human intervention.

Engineering Curiosity: World Models and Learning Progress

To replicate human-like learning, researchers are shifting from "model-free" to model-based reinforcement learning. In this framework, an agent builds an internal "world model"—a predictive simulation of how the environment responds to its actions.

The Goldilocks Effect

Early models rewarded agents for encountering states they could not predict (prediction error). However, this leads to the "White Noise Problem," where an agent becomes fixated on random, unpredictable stimuli (like static on a TV) because the error remains high.

A more robust approach rewards the agent for learning progress—the rate at which its prediction error is decreasing. This creates a "Goldilocks Effect," where the agent ignores what it already knows (too simple) and what it cannot possibly learn (too complex/random), focusing instead on the "just right" level of learnable complexity.

Emergent Behavior in Virtual Playrooms

Experimental setups using 3D "playrooms" demonstrate the power of curiosity. In these environments, agents are given no external goals. While a random agent interacts with objects less than 1% of the time, a curiosity-driven agent actively manipulates toys.

Over time, sophisticated behaviors emerge. An agent might learn to gather multiple toys together to see how they interact, effectively creating its own training data. Furthermore, when presented with "animate" stimuli (agents that act with goals), curiosity-driven models naturally develop a preference for social-like interaction. Because social agents are complex yet follow regularities, they provide a rich and sustained source of learning progress.

AI as a Tool for Developmental Psychology

Beyond engineering better robots, these models offer a new lens for developmental psychology. By adjusting the parameters of an agent's curiosity—such as how it weights social versus physical information—researchers can create fine-grained computational models of human learning trajectories.

One promising application is the study of Autism Spectrum Disorder (ASD). By simulating different "curiosity profiles," we can explore how variations in intrinsic motivation might lead to the different developmental outcomes observed in neurodiverse populations. This "triangulation" between AI behavior and human data allows for a more precise understanding of the mechanisms underlying human cognition.

Conclusion

The gap between current AI and the learning capacity of a child remains wide. However, by shifting our focus from passive supervision to curiosity-driven interaction, we move closer to creating truly autonomous, flexible learners. The development of an "ImageNet of Development"—massive datasets of childhood behavior—combined with progress-based intrinsic motivation, will be key to unlocking the next generation of artificial and computational intelligence.

Summary updated

Abstract

Abstract As “scientists in the crib,” children learn through curiosity, tirelessly seeking novelty and information as they interact—really, play—with both physical objects and the people around them. This flexible capacity to learn about the world through intrinsically motivated interaction continues throughout life. How would we engineer an artificial, autonomous agent that learns in this way – one that flexibly interacts with its environment, and others within it, in order to learn as humans do? In this chapter, I will first motivate this question by describing important advances in artificial intelligence in the last decade, noting ways in which artificial learning within these methods are and are not like human learning. I will then give an overview of recent results in artificial intelligence aimed at replicating curiosity-driven interactive learning. I will then close by speculating on how AI that learns in this fashion could be used as fine-grained computational models of human learning.
BibTeX
@inproceedings{haber2022curiosity,
  author = {Nick Haber},
  title = {{Curiosity and Interactive Learning in Artificial Systems}},
  booktitle = {Openalex Unknown},
  year = {2022},
  doi = {10.1007/978-3-031-09687-7\_3},
}
Figure