Role of visual and non-visual cues in constructing a rotation-invariant representation of heading in parietal cortex

Adhira Sunkara, Gregory C. DeAngelis, Dora E. AngelakieLife 2015 · DOI

TL;DR

This paper reveals that the brain can accurately determine our direction of travel (heading) even when our eyes or head are moving, not just by using internal "efference copy" signals, but crucially by analyzing visual cues in the environment. Specifically, neurons in the macaque ventral intraparietal area (VIP) use both local motion parallax and global perspective distortions in optic flow to achieve this rotation-invariant heading representation, demonstrating a sophisticated visual computation for navigation.

What Problem Does This Paper Address?

When we move through the world, our brains need to figure out where we're going – our "heading." This is usually done by analyzing "optic flow," the pattern of visual motion on our retinas. However, our heading perception gets complicated when we also move our eyes or head, as these movements add their own rotational patterns to the optic flow. The prevailing idea was that the brain relies heavily on non-visual signals, like efference copies (internal signals sent to muscles), to account for these head and eye movements and thus perceive heading accurately. This paper investigates whether visual cues alone can achieve this "rotation-invariant" heading representation, challenging the exclusive reliance on non-visual signals.

Methods

The researchers trained monkeys to perform tasks involving simulated self-motion. They used two main experimental conditions:

  1. Real Pursuit (RP): Monkeys tracked a moving target, which involved actual eye movements. This condition provided both visual optic flow and non-visual efference copy signals.
  2. Simulated Pursuit (SP): Monkeys fixated a stationary target while the visual stimulus mimicked the optic flow patterns of translation and eye rotation, but without actual eye movement. This condition isolated the role of visual cues.

They presented stimuli in two environments: a 3D cloud of dots (providing both local motion parallax and global perspective cues) and a fronto-parallel plane of dots (providing global perspective cues but no local motion parallax). Neural activity was recorded from neurons in the ventral intraparietal area (VIP). They analyzed heading tuning curves – how a neuron's firing rate changes with different directions of simulated self-motion – to see if these responses were invariant to rotations or if they simply reflected the combined visual motion. A novel analysis method was developed to account for complex changes in tuning curve shape (skew and bandwidth) beyond simple shifts.

Datasets

Not explicitly listed as datasets in the traditional machine learning sense. The study generated novel visual stimuli (3D dot clouds and fronto-parallel planes of dots) to simulate self-motion with varying degrees of translation and rotation. These stimuli were presented to macaque monkeys, and neural responses were recorded.

Key Insights and Ideas

  • Visual Cues are Powerful: The brain can achieve rotation-invariant heading perception using visual cues alone, not just relying on internal motor commands. This is a significant finding that expands our understanding of visual processing for navigation.
  • VIP's Role: The ventral intraparietal area (VIP) is a key player in this process, with a subpopulation of neurons showing rotation-invariant heading responses even when only visual cues are present.
  • Two Visual Cues at Play: Both local motion parallax (from depth differences in the scene) and global perspective cues (distortions in the overall flow pattern) contribute to disentangling translation from rotation.
  • Multi-sensory Integration: While visual cues are crucial, the study also suggests that VIP neurons integrate visual and non-visual rotation signals, hinting at a sophisticated multi-sensory processing system.
  • Reference Frame Nuance: Rotation tolerance doesn't necessarily imply a head-centered reference frame. VIP neurons seem to represent heading in an intermediate reference frame, closer to eye-centered, which is consistent with how visual information changes on the retina during eye movements.

Key Results

  • A significant portion of VIP neurons (around 20-30%) showed rotation-invariant heading responses, meaning their response to heading direction didn't change much when rotations were added, even in the simulated pursuit condition (visual cues only).
  • In the 3D cloud experiment, median heading shifts were significantly smaller than expected if neurons simply represented the resultant optic flow, indicating partial compensation for rotation.
  • In the fronto-parallel plane experiment (lacking local motion parallax), VIP neurons still showed significant compensation for rotations, highlighting the role of global perspective cues.
  • Neural responses to pure rotation stimuli (visual vs. non-visual) were significantly correlated, supporting the idea of multi-sensory convergence for rotation information.
  • Temporal response patterns suggested an intermediate reference frame for heading representation in VIP, closer to eye-centered than head-centered.

Limitations and Future Work

  • Limited Rotation Types: The study primarily focused on horizontal (yaw axis) rotations at a single constant velocity. Future work should investigate responses to different rotation velocities, axes, and combinations of rotations.
  • Testing Other Sensory Inputs: While the study highlights visual cues, it acknowledges that vestibular inputs (from the inner ear) also contribute to self-motion perception. Future research could explore how these are integrated with visual and efference copy signals.
  • Decoding VIP Population: The study suggests that depending on how VIP population activity is decoded, it could support near-perfect behavioral compensation. Further research could explore these decoding strategies.
  • Real-World Complexity: The stimuli, while sophisticated, are still simplified representations of real-world navigation. Investigating responses to more complex, naturalistic environments would be valuable.

Summary updated

Abstract

As we navigate through the world, eye and head movements add rotational velocity patterns to the retinal image. When such rotations accompany observer translation, the rotational velocity patterns must be discounted to accurately perceive heading. The conventional view holds that this computation requires efference copies of self-generated eye/head movements. Here we demonstrate that the brain implements an alternative solution in which retinal velocity patterns are themselves used to dissociate translations from rotations. These results reveal a novel role for visual cues in achieving a rotation-invariant representation of heading in the macaque ventral intraparietal area. Specifically, we show that the visual system utilizes both local motion parallax cues and global perspective distortions to estimate heading in the presence of rotations. These findings further suggest that the brain is capable of performing complex computations to infer eye movements and discount their sensory consequences based solely on visual cues.
BibTeX
@inproceedings{sunkara2015role,
  author = {Adhira Sunkara and Gregory C. DeAngelis and Dora E. Angelaki},
  title = {{Role of visual and non-visual cues in constructing a rotation-invariant representation of heading in parietal cortex}},
  booktitle = {eLife},
  year = {2015},
  doi = {10.7554/elife.04693},
}
Figure