Few-Shot Demonstrations Elicit the Use of In-Context World Representations in LLMs

Kohsei Matsutani, Gouki Minegishi, Core Francisco Park, Takeshi Kojima, Yusuke Iwasawa, Yutaka MatsuoarXiv 2026 · arXiv

TL;DR

Large language models (LLMs) can build internal representations of their environment from context, but they often struggle to use them for downstream tasks in zero-shot settings. This paper demonstrates that few-shot demonstrations from different "worlds" bridge this gap by geometrically shifting these internal representations into a usable subspace. This simple prompting technique unlocks the predictive power of LLM world models across synthetic graph tracking and practical benchmarks like ARC-AGI and Othello.

What Problem Does This Paper Address?

When LLMs act as autonomous agents, they must infer the rules, entities, and structures of their environment (a "world model") purely from in-context observations. However, recent research has highlighted a frustrating paradox: while LLMs successfully construct these internal world representations (e.g., mapping out a hidden graph structure from a text sequence), they fail to actually use them to solve subsequent tasks based on that structure. They "know" the world but cannot act on it.

This paper investigates how to overcome this fundamental limitation. The authors explore whether few-shot prompting can elicit the predictive use of these latent representations and delve deep into the internal mechanics of the model to explain why this shift occurs.

Figure 1: (a) Conceptual Framework of In-Context World Modeling. An LLM is provided with random walks generated on a graph topology, where each node is assigned a fruit, and instructed to move two steps to the right. We probe the internal representations to determine whether the model has constructed the latent graph. A linearly decodable subspace of these representations is termed a world representation. See Figure 11 for the actual prompt. In this paper, we treat cucumbers as fruits. (b) LLMs Learned to Use World Representation with Few Shot Demonstrations. Few shot demonstrations consisting of random walks, task instructions, and answers from worlds with the same topology but different assignments of fruits to nodes improve task performance. Interven- tions on the world representation subspace cause larger performance degradation in the few shot setting than in the zero shot setting relative to interventions in random directions. The results are from Qwen3-4B-Base on a 4 × 4 Lattice Grid. (c) Few-Shot Demonstrations Shift World Rep- resentations. As the number of shots increases, the graph representation shifts orthogonally to the representation subspace. The left shows a 3D PCA of hidden states of the nodes, and the right shows a 2D PCA of mean hidden states over the random walks in the test set. The results are from Figure 6.

Methods

The authors evaluate LLMs using a Graph Tracking Task. In this setup, the model observes a random walk on a hidden graph topology (such as a lattice grid where each node is assigned a fruit name) and is then asked to perform a task requiring two-step reasoning over that graph (e.g., predicting the node two steps to the right of a given fruit).

To analyze what happens inside the model, they employ several key techniques:

  • Linear Probing: They train a low-dimensional linear probe on the model's hidden states to determine if the graph's structural layout (adjacency matrix) can be recovered.
  • Dirichlet Energy (DE): A metric computed directly from raw hidden states to evaluate how well the representations naturally align with the graph topology without any probing.
  • Causal Intervention (Subspace Erasure): Using iterative nullspace projection, they selectively erase graph information from the identified world representation subspace to prove whether the model causally relies on it for downstream predictions.
  • Few-Shot Prompting with Different Worlds: Crucially, the few-shot demonstrations consist of random walks and tasks from different worlds (different fruit-to-node assignments or entirely different graph topologies) to ensure the model learns the mechanism of using the map rather than just memorizing specific answers.

Datasets

The paper evaluates its hypotheses across both synthetic and practical domains:

  • Synthetic Graph Topologies: 4×4 Lattice Grid, 5×5 Lattice Grid, 3×5 Lattice Grid, 4×4 Triangle Grid, and 4×4 Torus Grid.
  • Practical Benchmarks: ARC-AGI-1 & 2 (abstract visual reasoning puzzles), Mind2Web (cross-website web agent trajectories), and Othello (predicting legal board moves based on game history).

Key Insights and Ideas

  • The Map is Built, but Unread: Increasing the number of few-shot demonstrations does not improve how well the LLM constructs the world model. Probing AUROC and Dirichlet Energy remain virtually identical between zero-shot and multi-shot settings. The model always builds the map; it just doesn't know how to read it initially.
Figure 4: Probe Performance and Normalized Dirichlet Energy (DE). Probe AUROC and nor- malized DE of hidden states across shots and five graph topologies for layer 20 of Qwen3-4B-Base. Gray lines show random feature probe baselines.
  • The "Shot Vector" Shift: Few-shot examples induce a systematic geometric shift in the hidden states (termed the Shot Vector). This shift moves the world representation in a direction nearly orthogonal to its original subspace, effectively relocating it to a "readout-ready" zone where downstream layers can access it.
  • Causal Proof of Usage: Intervening to erase the graph representation subspace degrades performance significantly more in few-shot settings than in zero-shot settings. This provides direct causal evidence that few-shot examples force the model to actively utilize its internal world model for downstream predictions.

Key Results

  • Consistent Performance Boosts: Across 6 models from 4 families (Qwen3, Llama-3, Gemma-4, Ministral-3), downstream task accuracy steadily increases with the number of shots, typically saturating between 4 and 16 shots.
  • Generalization to Practical Tasks: Providing few-shot examples from different worlds significantly boosts performance on practical agent benchmarks. For instance, it improves pixel-level accuracy on ARC-AGI-1&2, step success rates on Mind2Web, and legal move rates in Othello.

Limitations and Future Work

  • Static Worlds Only: The authors define the world model strictly in terms of static structures (like grids or game boards). They do not address state transitions, temporal dynamics, or causal physics.
  • Confounding Factors: While the relocation of world representations is a major driver of performance gains, the authors note it is likely only one of several contributing factors behind few-shot learning success.
  • Prompting vs. Training: This study modifies activations purely through prompting. The authors suggest that future work should explore how explicit training paradigms—like Reinforcement Learning (RL) or Supervised Fine-Tuning (SFT)—affect the construction and predictive use of these internal representations.

Summary updated

Abstract

Large language models (LLMs), when acting as agents, are expected to take observed data in context, infer the latent state space underlying the world, and leverage it for downstream prediction. However, prior work demonstrated that LLMs struggle to use representations learned in context on a graph tracking task, where the model needs to construct a representation of the graph governing data generation process and use it for subsequent predictions. In this paper, we show that extending this to few-shot settings, where each demonstration is generated from a different world with either the same or different graph topologies, enhances its prediction on 6 models from 4 model families. To understand this improvement, we linearly probe a low-dimensional world representation that encodes graph information in the hidden states. Notably, we find that few-shot demonstrations relocate the world representation and increase its predictive use. Specifically, for each model, these world representations shift in directions nearly orthogonal to their original subspace, and interventions on these representations selectively impair performance more than interventions on other subspaces. Consistent with this insight, we show that few-shot demonstrations with observations from different worlds improve performance on ARC-AGI-1&2, web agent tasks, and Othello. Our findings elucidate the role and internal mechanisms of few-shot demonstrations in in-context world modeling. More broadly, our work advances our understanding of how LLM agents learn from in-context observations and provides implications for their further improvement.
BibTeX
@misc{matsutani2026few,
  author = {Kohsei Matsutani and Gouki Minegishi and Core Francisco Park and Takeshi Kojima and Yusuke Iwasawa and Yutaka Matsuo},
  title = {{Few-Shot Demonstrations Elicit the Use of In-Context World Representations in LLMs}},
  year = {2026},
  eprint = {2609.24352},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.24352},
}
Figure