Predictive Sensorimotor Control via Internal Forward Models

33 concepts28 papers1 proposal

When an agent—whether a biological organism or an autonomous robot—interacts with the physical world, it faces a fundamental engineering challenge: sensory feedback is slow, noisy, and deeply ambiguous. If a system relies purely on reactive control, waiting for sensory signals to travel from physical sensors to a central processor before executing a correction, its movements will be sluggish, unstable, and prone to failure.

To overcome these physical delays, intelligent systems shift from reactive feedback loops to proactive, anticipatory control. They achieve this by utilizing an internal forward model—a neural or computational simulator that predicts the sensory consequences of an action before the actual feedback arrives.

The Architecture of Anticipation

In a purely reactive paradigm, an agent issues a motor command, changes the environment, and then waits to perceive the result. In contrast, a predictive architecture splits the motor command into two pathways. One pathway sends the command to the physical actuators (muscles or motors). The second pathway sends a duplicate copy of the command—known as an efference copy—directly to the internal forward model.

As Yukie Nagai outlines in Predictive learning: its key role in early cognitive development, this internal predictor module combines the agent's current state with the incoming action efference to forecast the subsequent sensory state. By simulating the physics of the interaction internally, the agent can anticipate what its sensors should experience in the next instant.

                  +------------------+
                  |  Motor Control   |
                  +--------+---------+
                           |
            +--------------+--------------+
            |                             | (Efference Copy)
            v                             v
    +---------------+             +---------------+
    |   Actuators   |             | Forward Model |
    +-------+-------+             +-------+-------+
            |                             |
            v (Physical Action)           v (Sensory Prediction)
    +---------------+                     |
    |  Environment  |                     |
    +-------+-------+                     |
            |                             |
            v (Actual Feedback)           v
            +--------------+--------------+
                           |
                           v
               [ Sensory Comparison ] ---> Mismatch = Prediction Error

This predictive loop allows systems to maintain precise coordination even when external feedback is severely delayed or absent. For instance, a study on brain-machine interfaces published in Nature Communications (Rapid control and feedback rates enhance neuroprosthetic control) demonstrated that increasing the frequency of motor commands improves control performance even when visual feedback remains slow. This suggests that the brain relies on a high-rate internal forward model to fill the gaps between slow visual updates.

In robotics, this architecture enables machines to navigate complex environments without dense, computationally expensive sensor arrays. Johannes Schröder-Schetelig et al. demonstrated this in Using efference copy and a forward internal model for adaptive biped walking, showing how a bipedal robot could adapt to changing slopes "blindly." By comparing its actual joint states to the expectations generated by a forward model, the robot detects environmental changes purely through internal discrepancies. This builds on earlier work by Poramate Manoonpong et al. (Adaptive, Fast Walking in a Biped Robot under Neuronal Control and Learning), which emphasized that integrating efferent copies of movement commands allows for sophisticated coordination and rapid error correction in unpredictable terrains.

Even in early robotic frameworks, such as the one developed by Alan H. Bond et al. (Learning of Sensory-Motor Schemas in a Mobile Robot), mapping sequences of motor actions to expected future states was recognized as the mechanism that allows an agent to transition from simple, reactive reflexes to anticipatory, goal-directed planning.

Distinguishing Self from World via Sensory Cancellation

Beyond accelerating control, the forward model solves a deeper computational problem: distinguishing between self-generated sensory changes and external environmental stimuli. When you move your eyes, the image on your retina shifts dramatically. Yet, you do not perceive the world as shaking. Why?

Because the forward model predicts the exact visual shift caused by your eye movement and subtracts it from the incoming sensory data. This process is known as neural cancellation. By filtering out predictable, self-generated sensory noise, the system isolates unexpected external events.

This mechanism is highly apparent in human speech. A neuroimaging study in PLoS ONE (The Sensory Consequences of Speaking: Parametric Neural Cancellation during Speech in Auditory Cortex) showed that the auditory cortex actively suppresses its response to our own voice when the sound matches internal forward model predictions. When this predictive transmission fails or becomes noisy—as Kyunghee X. Kim et al. observed in patients with Alzheimer’s Disease (Impaired Speaking-Induced Suppression in Alzheimer’s Disease)—the sensory system cannot identify self-generated actions, causing an over-reliance on external feedback loops. Similarly, André Lee et al. (Alteration in forward model prediction of sensory outcome of motor action in focal hand dystonia) discovered that focal hand dystonia in musicians is characterized by a failure of the forward model to accurately predict and filter out redundant self-generated tactile information, degrading fine motor control.

In biology, sensory cancellation can be so absolute that it completely redefines how a sense operates. Peter Gyring demonstrated in Sensory limits in the rodent whisker system predict an internal forward model for sensorimotor estimation of object touch that the physical signals generated when a rodent's whisker touches an object are often too faint to be detected directly against the background noise of active whisking. Instead of directly "feeling" a touch, the rodent's brain predicts the unhindered movement of the whisker. A touch is detected purely as a prediction error—a deviation that occurs because the physical object prevents the whisker from reaching its internally expected state.

The Genesis of Agency and the Active Self

The continuous comparison between predicted and actual sensory states forms the cognitive foundation for the sense of agency—the subjective experience of being the author of one's own actions.

As reviewed by Jingjin Gu et al. (The sense of agency and the attribution of responsibility in human behavior), when the actual sensory feedback matches the internal prediction generated by the efference copy, the brain concludes, "I caused this event." If there is a mismatch, the brain attributes the event to an external cause. Di Wu et al. (Sense of agency based on action and causation: The mechanism of intentional binding effect for voluntary action) note that this low-level predictive mechanism operates within a tight temporal window, subjectively compressing the perceived time between a voluntary action and its sensory outcome.

In developmental robotics, this exact mechanism is used to bootstrap an artificial "sense of self." Work by Phuong D. H. Nguyen et al. (Sensorimotor representation learning for an "active self" in robots: A model survey) and Frank Röder et al. (The Embodied Crossmodal Self Forms Language and Interaction: A Computational Cognitive Review) illustrates how an agent can delineate its own physical boundaries from the environment by minimizing prediction errors. When a robot can consistently and accurately predict the sensory changes resulting from its own motor signals, it identifies those signals as originating from its own body, establishing a computational foundation for an active self.

Learning and Adaptation through Prediction Errors

What happens when the forward model's prediction is wrong? The resulting discrepancy—the sensory prediction error—is the primary driver for motor learning and behavioral discovery.

When an agent encounters a change in environmental physics, its forward model initially generates inaccurate predictions. The resulting error signals force the system to update its internal representations. This adaptation happens automatically and unconsciously from an early age. For example, L.L. Fassbender et al. (A comparison of force adaptation in toddlers and adults during a drawer opening task) demonstrated that toddlers as young as 1.5 years old possess the neural capacity to update internal models of environmental resistance. When an unexpected physical force is suddenly removed, the children overshoot their movements. These "aftereffects" prove that their motor systems are not merely reacting to forces in real-time, but are actively anticipating them based on an internal model.

Similarly, Hiroaki Gomi et al. (Recognition of Manipulated Objects by Motor Learning) showed that integrating visual cues with motor feedback allows neural networks to predictively identify object dynamics before physical manipulation occurs, enabling smooth control transitions.

In autonomous systems, prediction errors can even be leveraged to discover complex behaviors without pre-defined goals. Ralf Der demonstrated in Information Driven Self-Organization of Complex Robotic Behaviors that by using the discrepancy between predicted and actual sensor states to compute gradients for a controller, high-dimensional robots can spontaneously self-organize coordinated behaviors like crawling, rolling, and swinging.

Ultimately, the transition from reactive feedback to forward-modeling prediction marks a fundamental milestone in cognitive development. As argued by Giulio Sandini et al. (Cognitive Development in a Humanoid Robot) and Georg Martius et al. (Emergence of behavioral primitives in self-organizing control and composition of behavior for autonomous robots), learning to anticipate the sensory consequences of one's own actions allows an agent to move beyond immediate stimulus-response patterns, laying the groundwork for intentional interaction, environmental stability evaluation, and the capacity to interpret the actions of others.

Key Foundational Work

Learning of Sensory-Motor Schemas in a Mobile Robot (1981) — contributed 1 concept elements to this direction.
Recognition of Manipulated Objects by Motor Learning (1991) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_internal_forward_model_717c3aa8,
  title = {internal forward model},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/internal-forward-model/rdr_717c3aa880044f2cae5b7b32c4c6bc2d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_internal_forward_model_717c3aa8_v1,
  title = {internal forward model},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/internal-forward-model/rdr_717c3aa880044f2cae5b7b32c4c6bc2d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake