Language and Experience: A Computational Model of Social Learning in Complex Tasks

Cédric Colas, Tracey Mills, Ben Prystawski, Michael Henry Tessler, Noah D. Goodman, Jacob Andreas, Joshua B. Tenenbaum

Abstract

The ability to combine linguistic guidance from others with direct experience is central to human development, enabling safe and rapid learning in new environments. How do people integrate these two sources of knowledge, and how might AI systems? We present a computational framework that models social learning as joint probabilistic inference over structured, executable world models given sensorimotor and linguistic data. We make this possible by turning a pretrained language model into a probabilistic model of how humans share advice conditioned on their beliefs, allowing our agents both to generate advice for others and to interpret linguistic input as evidence during Bayesian inference. Using behavioral experiments and simulations across 10 video games, we show how linguistic guidance can shape exploration and accelerate learning by reducing risky interactions and speeding up key discoveries in both humans and models. We further explore how knowledge can accumulate across generations through iterated learning experiments and demonstrate successful knowledge transfer between humans and models -- revealing how structured, language-compatible representations might enable human-machine collaborative learning.

Introduction

Humans possess a remarkable capacity for social learning, seamlessly integrating "learning by doing" with "learning by listening." While modern AI has made strides in both domains—Reinforcement Learning (RL) for interactive mastery and Large Language Models (LLMs) for instruction following—bridging the two remains a significant challenge. RL agents often require millions of trials to learn simple rules, while LLMs lack the grounded interactive planning needed for physical or simulated environments. This research addresses the core challenge of social learning: how can an agent use a single piece of natural language advice to fundamentally reshape its exploration and decision-making?

Methodology: Social Learning as Probabilistic Inference

The proposed framework treats learning as joint probabilistic inference over structured, executable world models.

Structured World Models

Instead of opaque neural networks, agents maintain beliefs over programs written in Video Game Description Language (VGDL). These programs explicitly define the game's "physics," including sprite types, interaction rules, and termination conditions.

LLMs as Probabilistic Speaker Models

To interpret language as evidence, the framework utilizes a pre-trained LLM (LLaMA-3.1-70B) as a "probabilistic speaker." The model evaluates the likelihood that a human would provide specific advice given a particular internal world model. This allows the agent to perform Bayesian updates on its beliefs based on linguistic input.

Accelerated Inference and Planning

The LLM also serves as a proposal mechanism for new rules. For instance, hearing "yellow kills you" prompts the LLM to suggest specific VGDL code modifications. Once a world model is inferred, the agent employs goal-directed planning to balance exploration and exploitation.

The experimental paradigm and an example of an agent's learning trajectory, showing how it refines its internal 'programs' based on experience and messages.

Experimental Evaluation

The framework was tested across 10 diverse VGDL games designed to evaluate cognitive skills like spatial reasoning and tool use. Performance was benchmarked against human behavioral data (122 participants) and several AI baselines, including deep RL and pure LLM agents.

Results and Discussion

The results demonstrate that linguistic guidance acts as a powerful exploration filter, steering agents away from lethal mistakes and toward critical discoveries.

Learning Efficiency

The proposed model achieved human-level sample efficiency, significantly outperforming standard RL and pure LLM baselines. Human-written advice reduced the median attempts needed to solve games for both humans (from 4 to 2.25) and the model (from 2.5 to 1.25).

Comparison of learning curves: the proposed model (blue) and humans (orange) solve games much faster than standard RL or pure LLM baselines.

Bidirectional Knowledge Transfer

A key finding is the model's ability to act as a teacher. Model-generated advice was often more detailed than human advice, leading to significant performance gains for human learners, particularly in games with complex mechanics. This suggests that structured, language-compatible representations are a viable path toward effective human-machine collaborative learning.

Limitations

Despite its success, the model relies on symbolic VGDL representations, which are difficult to scale to continuous, noisy real-world environments. Furthermore, while it handles factual advice well, it lacks the ability to process human-style analogies or high-level metacognitive strategies. Future work will explore richer programming languages like Python and mechanisms for evaluating the reliability of advice.

Summary updated

Abstract

The ability to combine linguistic guidance from others with direct experience is central to human development, enabling safe and rapid learning in new environments. How do people integrate these two sources of knowledge, and how might AI systems? We present a computational framework that models social learning as joint probabilistic inference over structured, executable world models given sensorimotor and linguistic data. We make this possible by turning a pretrained language model into a probabilistic model of how humans share advice conditioned on their beliefs, allowing our agents both to generate advice for others and to interpret linguistic input as evidence during Bayesian inference. Using behavioral experiments and simulations across 10 video games, we show how linguistic guidance can shape exploration and accelerate learning by reducing risky interactions and speeding up key discoveries in both humans and models. We further explore how knowledge can accumulate across generations through iterated learning experiments and demonstrate successful knowledge transfer between humans and models -- revealing how structured, language-compatible representations might enable human-machine collaborative learning.
BibTeX
@inproceedings{colas2026language,
  author = {Cédric Colas and Tracey Mills and Ben Prystawski and Michael Henry Tessler and Noah D. Goodman and Jacob Andreas and Joshua B. Tenenbaum},
  title = {{Language and Experience: A Computational Model of Social Learning in Complex Tasks}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=UxDu3RFuDV},
}
Figure