Dissociation of Self-Motion and Object Motion by Linear Population Decoding That Approximates Marginalization

Ryo Sasaki, Dora E. Angelaki, Gregory C. DeAngelisJournal of Neuroscience 2017 · DOI

Abstract

We use visual image motion to judge the movement of objects, as well as our own movements through the environment. Generally, image motion components caused by object motion and self-motion are confounded in the retinal image. Thus, to estimate heading, the brain would ideally marginalize out the effects of object motion (or vice versa), but little is known about how this is accomplished neurally. Behavioral studies suggest that vestibular signals play a role in dissociating object motion and self-motion, and recent computational work suggests that a linear decoder can approximate marginalization by taking advantage of diverse multisensory representations. By measuring responses of MSTd neurons in two male rhesus monkeys and by applying a recently-developed method to approximate marginalization by linear population decoding, we tested the hypothesis that vestibular signals help to dissociate self-motion and object motion. We show that vestibular signals stabilize tuning for heading in neurons with congruent visual and vestibular heading preferences, whereas they stabilize tuning for object motion in neurons with discrepant preferences. Thus, vestibular signals enhance the separability of joint tuning for object motion and self-motion. We further show that a linear decoder, designed to approximate marginalization, allows the population to represent either self-motion or object motion with good accuracy. Decoder weights are broadly consistent with a readout strategy, suggested by recent computational work, in which responses are decoded according to the vestibular preferences of multisensory neurons. These results demonstrate, at both single neuron and population levels, that vestibular signals help to dissociate self-motion and object motion.

The Challenge of Confounded Motion

Imagine navigating a busy environment, like a crowded train station. As you move forward, your eyes receive "optic flow"—a pattern of visual expansion. However, if an object moves across your path, its motion distorts this pattern. To navigate safely, the brain must solve a "source separation" problem: distinguishing visual motion caused by self-movement from motion caused by external objects.

Illustration of confounded visual signals: (A) actual motion in the world, (B) the resulting confounded image motion, and (C) the experimental directions for object and self-motion.

In statistical terms, the ideal solution is marginalization—integrating out the "nuisance" variable (e.g., object motion) to isolate the variable of interest (e.g., heading). Because marginalization is often mathematically complex and non-linear, it has been unclear how biological neural populations, which are inherently noisy and diversely tuned, implement such a computation.

Methods: Recording from Area MSTd

Researchers investigated this problem by recording the activity of 164 neurons in the dorsal medial superior temporal area (MSTd) of rhesus monkeys. MSTd is a critical multisensory hub that processes both visual optic flow and vestibular (inner ear) signals.

The experimental design involved 64 combinations of self-motion (simulated via optic flow and/or a physical motion platform) and object motion (a cluster of moving spheres). To analyze the neural population, the team employed a novel technique called Approximate Linear Marginalization (ALM). Unlike standard decoding methods that focus on a neuron's preferred direction, ALM uses multinomial logistic regression to find linear weights optimized to produce a population response approximating the true marginal posterior distribution of the target variable.

Key Insights: The Power of Diversity and Vestibular Cues

The study revealed several critical features of how MSTd handles confounded motion:

  • The Role of Neural Diversity: The MSTd population is heterogeneous. "Congruent" cells, where visual and vestibular preferences match, see their heading tuning stabilized by vestibular signals. Conversely, "opposite" cells, where preferences differ, show stabilized tuning for object motion when vestibular cues are present.
  • Vestibular "Cleaning": Vestibular signals provide a non-visual ground truth for self-motion. By incorporating this multisensory input, the brain makes the joint representation of motion more "separable," significantly easing the task of source separation.
  • Linearity is Sufficient: A major finding is that the brain does not require complex, non-linear machinery to perform marginalization. A properly weighted linear sum of multisensory activity is sufficient to approximate the mathematically ideal solution.
  • Readout Strategy: ALM decoder weights aligned with the neurons' vestibular preferences. This suggests a biologically plausible mechanism where the brain decodes heading by prioritizing neurons whose vestibular signals corroborate the visual input.

Results: Dramatic Error Reduction

The impact of the ALM decoder was substantial. Standard decoding methods faced with object motion resulted in a Root Mean Square Error (RMSE) of approximately 73°. The ALM decoder reduced this error to just ~3.7° in the bimodal (visual + vestibular) condition. Furthermore, adding vestibular signals halved the error compared to the visual-only condition (7.3° vs 3.7°). Crucially, the same neural population could accurately represent either heading or object motion simply by adjusting the linear readout weights.

Implications and Future Directions

While this study demonstrates that the information required for marginalization exists in MSTd and can be extracted linearly, several questions remain. The experiments focused on 2D motion in the frontoparallel plane; real-world 3D navigation involves additional complexities like "looming." Additionally, while MSTd is well-suited for large objects, other areas like MSTl might be specialized for smaller, more typical objects. Finally, future research using causal manipulations like optogenetics will be needed to confirm that downstream brain areas actually utilize this ALM-like readout strategy.

Summary updated

Abstract

We use visual image motion to judge the movement of objects, as well as our own movements through the environment. Generally, image motion components caused by object motion and self-motion are confounded in the retinal image. Thus, to estimate heading, the brain would ideally marginalize out the effects of object motion (or vice versa), but little is known about how this is accomplished neurally. Behavioral studies suggest that vestibular signals play a role in dissociating object motion and self-motion, and recent computational work suggests that a linear decoder can approximate marginalization by taking advantage of diverse multisensory representations. By measuring responses of MSTd neurons in two male rhesus monkeys and by applying a recently-developed method to approximate marginalization by linear population decoding, we tested the hypothesis that vestibular signals help to dissociate self-motion and object motion. We show that vestibular signals stabilize tuning for heading in neurons with congruent visual and vestibular heading preferences, whereas they stabilize tuning for object motion in neurons with discrepant preferences. Thus, vestibular signals enhance the separability of joint tuning for object motion and self-motion. We further show that a linear decoder, designed to approximate marginalization, allows the population to represent either self-motion or object motion with good accuracy. Decoder weights are broadly consistent with a readout strategy, suggested by recent computational work, in which responses are decoded according to the vestibular preferences of multisensory neurons. These results demonstrate, at both single neuron and population levels, that vestibular signals help to dissociate self-motion and object motion.<b>SIGNIFICANCE STATEMENT</b> The brain often needs to estimate one property of a changing environment while ignoring others. This can be difficult because multiple properties of the environment may be confounded in sensory signals. The brain can solve this problem by marginalizing over irrelevant properties to estimate the property-of-interest. We explore this problem in the context of self-motion and object motion, which are inherently confounded in the retinal image. We examine how diversity in a population of multisensory neurons may be exploited to decode self-motion and object motion from the population activity of neurons in macaque area MSTd.
BibTeX
@article{sasaki2017dissociation,
  author = {Ryo Sasaki and Dora E. Angelaki and Gregory C. DeAngelis},
  title = {{Dissociation of Self-Motion and Object Motion by Linear Population Decoding That Approximates Marginalization}},
  journal = {Journal of Neuroscience},
  year = {2017},
  doi = {10.1523/jneurosci.1177-17.2017},
}
Figure