Unifying the Sensory and Motor Components of Sensorimotor Adaptation

Adrian Haith, Carl P. T. Jackson, R. Chris Miall, Sethu VijayakumarNeurIPS 2008 · OpenReview

When we learn to move in a new environment—such as wearing prism goggles that shift our vision—our brains adapt in two distinct ways. First, there is motor adaptation, which involves changing how we move our limbs. Second, there is sensory adaptation, which involves changing where we perceive our limbs and targets to be in space.

Historically, researchers have treated these as separate, isolated processes. Existing models, often based on Maximum Likelihood Estimation (MLE), assumed sensory adaptation was driven entirely by "sensory discrepancy"—for example, when your eyes see your hand in one place, but your muscles feel it in another. Under this traditional view, if there is no discrepancy between what you see and what you feel, your sensory perception should not change.

This paper challenges that independent view, proposing a unified Bayesian framework for sensorimotor adaptation. The authors argue that the brain does not treat sensory and motor changes as separate silos; instead, it uses a single principle of optimal error estimation to drive both components simultaneously.

The Unified Framework

To investigate this, the authors compared two computational approaches:

  1. The Combined MLE Model: A traditional, "bolted-together" model where sensory adaptation is driven strictly by the difference between visual and proprioceptive cues, while motor adaptation is driven by the remaining task error.
  2. The Unified Bayesian Model: A state-space model that treats visual shifts, proprioceptive shifts, and motor disturbances as hidden variables that evolve over time.
Graphical representation of the Unified Bayesian Model

The Bayesian model uses a Kalman filter to perform "credit assignment." When an error occurs (e.g., the hand misses the target), the brain doesn't know with absolute certainty if the eyes were wrong, the arm was pushed, or the muscles misfired. The model distributes the "blame"—and thus the resulting adaptation—across all three components based on their relative uncertainties and historical volatility.

Global Error Attribution and Force Fields

The core insight of this unified model is that error attribution is global. Any observed error provides information about all potential sources of disturbance. If you miss a target, your brain considers the possibility that your vision is slightly off, even if your proprioceptive senses agree with your vision.

This leads to a striking and counterintuitive prediction: adapting to a purely physical force (a motor disturbance) should cause a shift in visual perception, even though there is zero discrepancy between visual and proprioceptive feedback.

To test this, the authors conducted a reaching experiment using a robotic manipulandum. Subjects adapted to a velocity-dependent force field (a physical push on the arm) while visual feedback remained perfectly aligned with their actual hand position. They interleaved reaching tasks with "alignment tests"—asking subjects to localize a visual target or their unseen fingertip—to measure perceptual shifts.

Key Results

The experimental data strongly supported the unified Bayesian framework over the traditional MLE approach.

  • Superior Model Fit: The Bayesian model provided a significantly better fit to existing visuomotor rotation data (from Krakauer et al., 2006) than the MLE-based model. This was quantitatively confirmed by the Akaike Information Criterion, where the Bayesian model achieved an AIC of 126.7 compared to the MLE model's 159.6.
Comparison of model fits for directional error over trials
  • Experimental Confirmation of Perceptual Shifts: As predicted by the Bayesian model—and directly contradicting the MLE model—subjects in the force field experiment showed significant lateral shifts in both visual and proprioceptive localization ($p < .05$). Even though their eyes and hands were never out of sync, the physical disturbance altered their visual perception.
  • Accurate Timecourse: The Bayesian model successfully captured not just the final magnitude of these perceptual shifts, but also the trial-by-trial timecourse of how those shifts developed during the adaptation process.

Limitations and Future Directions

While the unified model offers a powerful new lens for understanding adaptation, the authors note a few limitations. The model uses a highly simplified "first-order" approximation of reaching dynamics, whereas real-world movement involves complex, multi-joint physics. Additionally, the model relies on knowing the "volatility" (trial-to-trial variance) of different sensors, and it remains unclear exactly how the brain estimates these parameters dynamically.

Future work could explore whether this unified Bayesian framework holds for other types of learning, such as adapting to delayed feedback or navigating more complex, non-linear force fields. Nonetheless, the findings strongly suggest that the nervous system solves adaptation by performing joint inference on the state of the body and the environment, rather than running separate correction loops.

Summary updated

Abstract

Adaptation of visually guided reaching movements in novel visuomotor environments (e.g. wearing prism goggles) comprises not only motor adaptation but also substantial sensory adaptation, corresponding to shifts in the perceived spatial location of visual and proprioceptive cues. Previous computational models of the sensory component of visuomotor adaptation have assumed that it is driven purely by the discrepancy introduced between visual and proprioceptive estimates of hand position and is independent of any motor component of adaptation. We instead propose a unified model in which sensory and motor adaptation are jointly driven by optimal Bayesian estimation of the sensory and motor contributions to perceived errors. Our model is able to account for patterns of performance errors during visuomotor adaptation as well as the subsequent perceptual aftereffects. This unified model also makes the surprising prediction that force field adaptation will elicit similar perceptual shifts, even though there is never any discrepancy between visual and proprioceptive observations. We confirm this prediction with an experiment.
BibTeX
@inproceedings{haith2008unifying,
  author = {Adrian Haith and Carl P. T. Jackson and R. Chris Miall and Sethu Vijayakumar},
  title = {{Unifying the Sensory and Motor Components of Sensorimotor Adaptation}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2008},
  url = {https://openreview.net/forum?id=rkZpGtW_-S},
}

Loading related content...

Figure