Contextual inference underlies the learning of sensorimotor repertoires

James B. Heald, Máté Lengyel, Daniel M. Wolpert

Abstract

Humans spend a lifetime learning, storing, and refining a repertoire of motor memories. However, it remains unknown how our continuous stream of sensorimotor experience is segmented into separate memories and how we adapt this growing repertoire. We present the COntextual INference (COIN) model, a principled theory of motor learning where memory creation, updating, and expression are all controlled by a single computation: contextual inference. Unlike dominant theories of single-context learning, the COIN model accounts for key features of motor learning that previously lacked a unified explanation and predicts novel phenomena, such as evoked recovery. Our results suggest that contextual inference is the core principle underlying how diverse experiences are reflected in motor behavior.

Introduction: The Challenge of Motor Repertoires

Traditional theories of motor learning often treat the brain as a simple adaptive system that minimizes errors within a single, fixed context. For example, if a person picks up a heavy mug, the brain adjusts motor commands to compensate for the weight. However, real-world behavior is far more complex. We do not just learn one task; we maintain a vast repertoire of skills for different tools, environments, and objects.

This "repertoire problem" presents three fundamental challenges:

  1. Memory Creation: When should the brain decide a situation is sufficiently novel to warrant a new memory?
  2. Memory Expression: Given multiple stored memories, which one should be used to guide current action?
  3. Memory Updating: When an error occurs, which specific memory in the repertoire should be adjusted?

Existing models typically address these questions through separate, heuristic processes. We propose that these are all manifestations of a single underlying Bayesian process: contextual inference.

The COIN Model: A Unified Bayesian Framework

The COIN model posits that the brain maintains a set of internal models (memories), each associated with a specific context. The brain never directly observes the current context; instead, it must infer it from two streams of information: sensory cues (e.g., the visual appearance of an object) and state feedback (e.g., the forces felt during movement).

The COIN model framework: Contextual inference integrates sensory cues and state feedback to manage memory creation, expression, and updating.

The model uses a "sticky" Hierarchical Dirichlet Process to represent an environment where an infinite number of contexts could exist, but they tend to persist over time. Inference is performed in real-time using particle learning (a sequential Monte Carlo method), allowing for three unified operations:

  • Creation: A new memory is formed when the probability of all existing contexts, given the current observations, falls below a certain threshold.
  • Expression: Motor output is a weighted average of all memories in the repertoire. The weights are the "predicted probabilities" of each context being active.
  • Updating: When an error is sensed, the brain updates its internal models in proportion to their "responsibility"—the posterior probability that a specific context was active during the movement.

Experimental Validation and Key Results

We tested the COIN model using a series of experiments involving a vBOT robotic manipulandum, which applies controlled force fields to a user's hand. We compared the COIN model against the dominant "dual-rate" state-space models, which explain learning through the interaction of fast-adapting and slow-adapting processes within a single context.

Comparison of COIN and dual-rate models: COIN accurately predicts spontaneous and evoked recovery by inferring the underlying context.

A key test was the phenomenon of spontaneous recovery. In a typical paradigm, a participant learns a force field (P+), which is then "washed out" by a brief period of the opposite field (P-). When the field is then removed (Pc), the original motor memory (P+) spontaneously reappears. While dual-rate models explain this as the slow process outlasting the fast process, the COIN model shows it is an inference phenomenon: as the "washout" ends, the brain's inference engine begins to believe the original P+ context is likely again.

Furthermore, the COIN model predicted a novel phenomenon: evoked recovery. We found that a very brief "reminder" of the original context (just two trials of P+) during the washout phase could trigger a massive, long-lasting return to the original motor memory. This result is inconsistent with dual-rate models but is a natural consequence of the COIN model's Bayesian inference. Beyond these experiments, COIN successfully explained five distinct motor learning phenomena, including savings and anterograde interference, that previously required separate explanations.

Conclusion

The COIN model provides a parsimonious and powerful explanation for how the brain manages a lifetime of motor experiences. By treating context as a hidden variable to be inferred, the model unifies diverse phenomena under a single computational principle. This suggests that the fundamental unit of motor learning is not just the adjustment of a motor command, but the continuous refinement of our beliefs about the world we inhabit.

Summary updated

Abstract

Humans spend a lifetime learning, storing and refining a repertoire of motor memories. However, it is unknown what principle underlies the way our continuous stream of sensorimotor experience is segmented into separate memories and how we adapt and use this growing repertoire. Here we develop a principled theory of motor learning based on the key insight that memory creation, updating, and expression are all controlled by a single computation – contextual inference. Unlike dominant theories of single-context learning, our repertoire-learning model accounts for key features of motor learning that had no unified explanation and predicts novel phenomena, which we confirm experimentally. These results suggest that contextual inference is the key principle underlying how a diverse set of experiences is reflected in motor behavior.
BibTeX
@inproceedings{heald2021contextual,
  author = {James B. Heald and Máté Lengyel and Daniel M. Wolpert},
  title = {{Contextual inference underlies the learning of sensorimotor repertoires}},
  booktitle = {Nature},
  year = {2021},
  doi = {10.1038/s41586-021-04129-3},
}
Figure